2023-10-18T11:55:36.191 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-18T11:55:36.193 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832 --name pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} -- /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/orig.config.yaml 2023-10-18T11:55:36.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T11:55:36.355 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832 branch: wip-batrick-testing-20231016.203825 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7430832' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20231016.203825 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231016.203825 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7703 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb sleep_before_teardown: 0 subset: 153/256 suite: fs suite_branch: wip-batrick-testing-20231016.203825 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_23:59:58 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T11:55:36.356 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa; will attempt to use it 2023-10-18T11:55:36.357 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks 2023-10-18T11:55:36.357 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T11:55:36.359 INFO:teuthology.task.internal:Checking packages... 2023-10-18T11:55:36.382 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f7420d7f8e55ceea786dcbf536c338ce920c21eb' 2023-10-18T11:55:36.382 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T11:55:36.383 INFO:teuthology.packaging:ref: None 2023-10-18T11:55:36.383 INFO:teuthology.packaging:tag: None 2023-10-18T11:55:36.383 INFO:teuthology.packaging:branch: wip-batrick-testing-20231016.203825 2023-10-18T11:55:36.383 INFO:teuthology.packaging:sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T11:55:36.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20231016.203825 2023-10-18T11:55:36.527 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6786.gf7420d7f 2023-10-18T11:55:36.527 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T11:55:36.533 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T11:55:36.534 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T11:55:36.547 INFO:teuthology.task.internal:Saving configuration 2023-10-18T11:55:36.568 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T11:55:36.574 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T11:55:36.598 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 11:50:08.724412', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T11:55:36.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 11:50:08.728216', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T11:55:36.646 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 11:50:08.726455', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T11:55:36.646 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T11:55:36.652 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-18T11:55:36.652 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-18T11:55:36.653 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-18T11:55:36.653 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T11:55:36.754 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff6d999dca0>, signals=[15]) 2023-10-18T11:55:36.755 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T11:55:36.763 INFO:teuthology.task.internal:Opening connections... 2023-10-18T11:55:36.763 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T11:55:36.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T11:55:36.832 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2023-10-18T11:55:36.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T11:55:36.900 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-10-18T11:55:36.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T11:55:36.967 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T11:55:36.973 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2023-10-18T11:55:36.995 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2023-10-18T11:55:36.995 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2023-10-18T11:55:37.054 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T11:55:37.054 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8.6 (Ootpa)" 2023-10-18T11:55:37.054 INFO:teuthology.orchestra.run.smithi070.stdout:ID="rhel" 2023-10-18T11:55:37.055 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="fedora" 2023-10-18T11:55:37.055 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8.6" 2023-10-18T11:55:37.055 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2023-10-18T11:55:37.055 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-18T11:55:37.055 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-10-18T11:55:37.056 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-18T11:55:37.056 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T11:55:37.056 INFO:teuthology.orchestra.run.smithi070.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T11:55:37.056 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T11:55:37.056 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T11:55:37.057 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T11:55:37.057 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-18T11:55:37.057 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T11:55:37.057 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-18T11:55:37.058 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-10-18T11:55:37.082 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2023-10-18T11:55:37.103 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2023-10-18T11:55:37.103 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2023-10-18T11:55:37.161 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T11:55:37.161 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8.6 (Ootpa)" 2023-10-18T11:55:37.162 INFO:teuthology.orchestra.run.smithi152.stdout:ID="rhel" 2023-10-18T11:55:37.162 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="fedora" 2023-10-18T11:55:37.162 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8.6" 2023-10-18T11:55:37.162 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8" 2023-10-18T11:55:37.163 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-18T11:55:37.163 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2023-10-18T11:55:37.163 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-18T11:55:37.163 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T11:55:37.163 INFO:teuthology.orchestra.run.smithi152.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T11:55:37.164 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T11:55:37.164 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T11:55:37.164 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T11:55:37.164 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-18T11:55:37.165 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T11:55:37.165 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-18T11:55:37.165 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2023-10-18T11:55:37.191 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-10-18T11:55:37.211 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-10-18T11:55:37.211 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-10-18T11:55:37.269 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T11:55:37.270 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2023-10-18T11:55:37.270 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-10-18T11:55:37.270 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-10-18T11:55:37.270 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2023-10-18T11:55:37.270 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-10-18T11:55:37.271 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-18T11:55:37.271 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-10-18T11:55:37.271 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-18T11:55:37.271 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T11:55:37.271 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T11:55:37.271 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T11:55:37.272 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T11:55:37.272 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T11:55:37.272 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-18T11:55:37.272 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T11:55:37.272 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-18T11:55:37.273 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-10-18T11:55:37.290 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T11:55:37.300 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T11:55:37.305 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T11:55:37.306 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2023-10-18T11:55:37.308 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2023-10-18T11:55:37.311 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-10-18T11:55:37.329 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T11:55:37.335 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T11:55:37.335 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2023-10-18T11:55:37.367 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2023-10-18T11:55:37.369 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-10-18T11:55:37.403 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T11:55:37.480 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T11:55:37.494 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-18T11:55:37.495 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-18T11:55:37.495 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-18T11:55:37.496 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T11:55:37.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:55:37.515 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-18T11:55:37.682 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T11:55:37.682 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T11:55:37.682 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-10-18T11:55:37.702 INFO:teuthology.orchestra.run.smithi070.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-18T11:55:37.702 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T11:55:37.702 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T11:55:37.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum upgrade -y linux-firmware 2023-10-18T11:55:38.987 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-10-18T11:55:38.988 INFO:teuthology.orchestra.run.smithi070.stdout:Unable to read consumer identity 2023-10-18T11:55:40.529 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-10-18T11:55:41.205 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-10-18T11:55:44.722 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 622 kB/s | 24 kB 00:00 2023-10-18T11:55:45.455 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-10-18T11:55:45.456 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2023-10-18T11:55:45.456 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-10-18T11:55:45.585 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T11:55:45.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T11:55:45.812 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ 2023-10-18T11:55:45.812 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-18T11:55:45.815 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi070.front.sepia.ceph.com to complete... 2023-10-18T11:56:00.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18 11:56:00 URL:https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [383161052/383161052] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-18T11:56:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:FINISHED --2023-10-18 11:56:00-- 2023-10-18T11:56:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:Total wall clock time: 14s 2023-10-18T11:56:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:Downloaded: 1 files, 365M in 14s (25.8 MB/s) 2023-10-18T11:56:00.344 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T11:56:00.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-18T11:56:01.951 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying... ######################################## 2023-10-18T11:56:02.021 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing... ######################################## 2023-10-18T11:56:02.122 INFO:teuthology.orchestra.run.smithi070.stdout:Updating / installing... 2023-10-18T11:56:40.401 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-18T11:56:45.201 INFO:teuthology.orchestra.run.smithi070.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-18T11:57:04.771 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-18T11:57:04.772 DEBUG:teuthology.orchestra.run.smithi070:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-18T11:57:04.827 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T11:57:04.828 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-18T11:57:04.828 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T11:57:04.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-18T11:57:04.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-18T11:57:04.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-18T11:57:04.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-18T11:57:04.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-18T11:57:04.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T11:57:04.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T11:57:04.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T11:57:04.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T11:57:04.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T11:57:04.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T11:57:04.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T11:57:04.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-18T11:57:04.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T11:57:04.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T11:57:04.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T11:57:04.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T11:57:04.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T11:57:04.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T11:57:04.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T11:57:04.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T11:57:04.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T11:57:04.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T11:57:04.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T11:57:04.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T11:57:04.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-18T11:57:04.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-18T11:57:04.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-18T11:57:04.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T11:57:04.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-18T11:57:04.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T11:57:04.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T11:57:04.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T11:57:04.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-18T11:57:04.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T11:57:04.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T11:57:04.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T11:57:04.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-18T11:57:04.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T11:57:04.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T11:57:04.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T11:57:04.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T11:57:04.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T11:57:04.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T11:57:04.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T11:57:04.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T11:57:04.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T11:57:04.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T11:57:04.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T11:57:04.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T11:57:04.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T11:57:04.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T11:57:04.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T11:57:04.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T11:57:04.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T11:57:04.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T11:57:04.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T11:57:04.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T11:57:04.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T11:57:04.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T11:57:04.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T11:57:04.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T11:57:04.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T11:57:04.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T11:57:04.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T11:57:04.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T11:57:04.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T11:57:04.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T11:57:04.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T11:57:04.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T11:57:04.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T11:57:04.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T11:57:04.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T11:57:04.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T11:57:04.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T11:57:04.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T11:57:04.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T11:57:04.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T11:57:04.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T11:57:04.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T11:57:04.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T11:57:04.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T11:57:04.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T11:57:04.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-18T11:57:04.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-18T11:57:04.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T11:57:04.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T11:57:04.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-18T11:57:04.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T11:57:04.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T11:57:04.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T11:57:04.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T11:57:04.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T11:57:04.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-18T11:57:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T11:57:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T11:57:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-18T11:57:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T11:57:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T11:57:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T11:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T11:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T11:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T11:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T11:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T11:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T11:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T11:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T11:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T11:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T11:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T11:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T11:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T11:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T11:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T11:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T11:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T11:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T11:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T11:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T11:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T11:57:04.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T11:57:04.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T11:57:04.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T11:57:04.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T11:57:04.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T11:57:04.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T11:57:04.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T11:57:04.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T11:57:04.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T11:57:04.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T11:57:04.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T11:57:04.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-18T11:57:04.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T11:57:04.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T11:57:04.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T11:57:04.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T11:57:04.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T11:57:04.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T11:57:04.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T11:57:04.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T11:57:04.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T11:57:04.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T11:57:04.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T11:57:04.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T11:57:04.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T11:57:04.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T11:57:04.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T11:57:04.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T11:57:04.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T11:57:04.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-18T11:57:04.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T11:57:04.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-18T11:57:04.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-18T11:57:04.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-18T11:57:04.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-18T11:57:04.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-18T11:57:04.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T11:57:04.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-18T11:57:04.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-18T11:57:04.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T11:57:04.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-18T11:57:04.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-18T11:57:04.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T11:57:04.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T11:57:04.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-18T11:57:04.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-18T11:57:04.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T11:57:04.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T11:57:04.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-18T11:57:04.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T11:57:04.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T11:57:04.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T11:57:04.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-18T11:57:04.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-18T11:57:04.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T11:57:04.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-18T11:57:04.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-18T11:57:04.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T11:57:04.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-18T11:57:04.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T11:57:04.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-18T11:57:04.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T11:57:04.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T11:57:04.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T11:57:04.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T11:57:04.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T11:57:04.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-18T11:57:04.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T11:57:04.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T11:57:04.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T11:57:04.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T11:57:04.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T11:57:04.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T11:57:04.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T11:57:04.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-18T11:57:04.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T11:57:04.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T11:57:04.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T11:57:04.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-18T11:57:04.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-18T11:57:04.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T11:57:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T11:57:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T11:57:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-18T11:57:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-18T11:57:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-18T11:57:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-18T11:57:04.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-18T11:57:04.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-18T11:57:04.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T11:57:04.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T11:57:04.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-18T11:57:04.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-18T11:57:04.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T11:57:04.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-18T11:57:04.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-18T11:57:04.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T11:57:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T11:57:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-18T11:57:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-18T11:57:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-18T11:57:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T11:57:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-18T11:57:04.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T11:57:04.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T11:57:04.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T11:57:04.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-18T11:57:04.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T11:57:04.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-18T11:57:04.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T11:57:04.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-18T11:57:04.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-18T11:57:04.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-18T11:57:04.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-18T11:57:04.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T11:57:04.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-18T11:57:04.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T11:57:04.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-18T11:57:04.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T11:57:04.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T11:57:04.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T11:57:04.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T11:57:04.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T11:57:04.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T11:57:04.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T11:57:04.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T11:57:04.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T11:57:04.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T11:57:04.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T11:57:04.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T11:57:04.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T11:57:04.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-18T11:57:04.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-18T11:57:04.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T11:57:04.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-18T11:57:04.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T11:57:04.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T11:57:04.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-18T11:57:04.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T11:57:04.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T11:57:04.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T11:57:04.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T11:57:04.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T11:57:04.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T11:57:04.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T11:57:04.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T11:57:04.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T11:57:04.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T11:57:04.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T11:57:04.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T11:57:04.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T11:57:04.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T11:57:04.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T11:57:04.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T11:57:04.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T11:57:04.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T11:57:04.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T11:57:04.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T11:57:04.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T11:57:04.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T11:57:04.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T11:57:04.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T11:57:04.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T11:57:04.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T11:57:04.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T11:57:04.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T11:57:04.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T11:57:04.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T11:57:04.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T11:57:04.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T11:57:04.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T11:57:04.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T11:57:04.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T11:57:04.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T11:57:04.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T11:57:04.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T11:57:04.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T11:57:04.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T11:57:04.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T11:57:04.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T11:57:04.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T11:57:04.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T11:57:04.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T11:57:04.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T11:57:04.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T11:57:04.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T11:57:04.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T11:57:04.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T11:57:04.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T11:57:04.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T11:57:04.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T11:57:04.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T11:57:04.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T11:57:04.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T11:57:04.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T11:57:04.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T11:57:04.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T11:57:04.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T11:57:04.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T11:57:04.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T11:57:04.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T11:57:04.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T11:57:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-18T11:57:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T11:57:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T11:57:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T11:57:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-18T11:57:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T11:57:04.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T11:57:04.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T11:57:04.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T11:57:04.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T11:57:04.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T11:57:04.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T11:57:04.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T11:57:04.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T11:57:04.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T11:57:04.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T11:57:04.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T11:57:04.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T11:57:04.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T11:57:04.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T11:57:04.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T11:57:04.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T11:57:04.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T11:57:04.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T11:57:04.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T11:57:04.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T11:57:04.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T11:57:04.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T11:57:04.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T11:57:04.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T11:57:04.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T11:57:04.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T11:57:04.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T11:57:04.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T11:57:04.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T11:57:04.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T11:57:04.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T11:57:04.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T11:57:04.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T11:57:04.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T11:57:04.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T11:57:04.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T11:57:04.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T11:57:04.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T11:57:04.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T11:57:04.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T11:57:04.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T11:57:04.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T11:57:04.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T11:57:04.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T11:57:04.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T11:57:04.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T11:57:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T11:57:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T11:57:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T11:57:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T11:57:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T11:57:04.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T11:57:04.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T11:57:04.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T11:57:04.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T11:57:04.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T11:57:04.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T11:57:04.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T11:57:04.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T11:57:04.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T11:57:04.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T11:57:04.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T11:57:04.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T11:57:04.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T11:57:04.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T11:57:04.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T11:57:04.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T11:57:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T11:57:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T11:57:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T11:57:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T11:57:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T11:57:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T11:57:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T11:57:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T11:57:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T11:57:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T11:57:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T11:57:04.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T11:57:04.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T11:57:04.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T11:57:04.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T11:57:04.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T11:57:04.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T11:57:04.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T11:57:04.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T11:57:04.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T11:57:04.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-18T11:57:04.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T11:57:04.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T11:57:04.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T11:57:04.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T11:57:04.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T11:57:04.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T11:57:04.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T11:57:04.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T11:57:04.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T11:57:04.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T11:57:04.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T11:57:04.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T11:57:04.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T11:57:04.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T11:57:04.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T11:57:04.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T11:57:04.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T11:57:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T11:57:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T11:57:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T11:57:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T11:57:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T11:57:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T11:57:04.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T11:57:04.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T11:57:04.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T11:57:04.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T11:57:04.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T11:57:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T11:57:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T11:57:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T11:57:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T11:57:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T11:57:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T11:57:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T11:57:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T11:57:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T11:57:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T11:57:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T11:57:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T11:57:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T11:57:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-18T11:57:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-18T11:57:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T11:57:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T11:57:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-18T11:57:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T11:57:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T11:57:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T11:57:04.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T11:57:04.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T11:57:04.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T11:57:04.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T11:57:04.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T11:57:04.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T11:57:04.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T11:57:04.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T11:57:04.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T11:57:04.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T11:57:04.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T11:57:04.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T11:57:04.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T11:57:04.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T11:57:04.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T11:57:04.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T11:57:04.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T11:57:04.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T11:57:04.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T11:57:04.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T11:57:04.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-18T11:57:04.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T11:57:04.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-18T11:57:04.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T11:57:04.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T11:57:04.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-18T11:57:04.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-18T11:57:04.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T11:57:04.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T11:57:04.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T11:57:04.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T11:57:04.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T11:57:04.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T11:57:04.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T11:57:04.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-18T11:57:04.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-18T11:57:04.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T11:57:04.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-18T11:57:04.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T11:57:04.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-18T11:57:04.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T11:57:04.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-18T11:57:04.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T11:57:04.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-18T11:57:04.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-18T11:57:04.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T11:57:04.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-18T11:57:04.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T11:57:04.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-18T11:57:04.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T11:57:04.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-18T11:57:04.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T11:57:04.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-18T11:57:04.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T11:57:04.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-18T11:57:04.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-18T11:57:04.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T11:57:04.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T11:57:04.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T11:57:04.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T11:57:04.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-18T11:57:04.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T11:57:04.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-18T11:57:04.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T11:57:04.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T11:57:04.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T11:57:04.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T11:57:04.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T11:57:04.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T11:57:04.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T11:57:04.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T11:57:04.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-18T11:57:04.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T11:57:04.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-18T11:57:04.928 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T11:57:04.928 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T11:57:04.928 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T11:57:04.928 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T11:57:04.928 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T11:57:04.929 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-18T11:57:04.929 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-18T11:57:04.929 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T11:57:04.929 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-18T11:57:04.929 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T11:57:04.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T11:57:04.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T11:57:04.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T11:57:04.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-18T11:57:04.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T11:57:04.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T11:57:04.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T11:57:04.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T11:57:04.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T11:57:04.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T11:57:04.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T11:57:04.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T11:57:04.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-18T11:57:04.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-18T11:57:04.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T11:57:04.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T11:57:04.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T11:57:04.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T11:57:04.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T11:57:04.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T11:57:04.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T11:57:04.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T11:57:04.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T11:57:04.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T11:57:04.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T11:57:04.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T11:57:04.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T11:57:04.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T11:57:04.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T11:57:04.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T11:57:04.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T11:57:04.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T11:57:04.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T11:57:04.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-18T11:57:04.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T11:57:04.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T11:57:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T11:57:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T11:57:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T11:57:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-18T11:57:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-18T11:57:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T11:57:04.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T11:57:04.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T11:57:04.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T11:57:04.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T11:57:04.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T11:57:04.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T11:57:04.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-18T11:57:04.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T11:57:04.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T11:57:04.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-18T11:57:04.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T11:57:04.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T11:57:04.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T11:57:04.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T11:57:04.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-18T11:57:04.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-18T11:57:04.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T11:57:04.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T11:57:04.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T11:57:04.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T11:57:04.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-18T11:57:04.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T11:57:04.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-18T11:57:04.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T11:57:04.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T11:57:04.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T11:57:04.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T11:57:04.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T11:57:04.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T11:57:04.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T11:57:04.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T11:57:04.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-18T11:57:04.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T11:57:04.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T11:57:04.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-18T11:57:04.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T11:57:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T11:57:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T11:57:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-18T11:57:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-18T11:57:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-18T11:57:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-18T11:57:04.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-18T11:57:04.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-18T11:57:04.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T11:57:04.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T11:57:04.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T11:57:04.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T11:57:04.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-18T11:57:04.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T11:57:04.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-18T11:57:04.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-18T11:57:04.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T11:57:04.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-18T11:57:04.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-18T11:57:04.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T11:57:04.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T11:57:04.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T11:57:04.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-18T11:57:04.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T11:57:04.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T11:57:04.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T11:57:04.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-18T11:57:04.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-18T11:57:04.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T11:57:04.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T11:57:04.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T11:57:04.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T11:57:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-18T11:57:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T11:57:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T11:57:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T11:57:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T11:57:04.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-18T11:57:04.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T11:57:04.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-18T11:57:04.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T11:57:04.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-18T11:57:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T11:57:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-18T11:57:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T11:57:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-18T11:57:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T11:57:04.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T11:57:04.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T11:57:04.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-18T11:57:04.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-18T11:57:04.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T11:57:04.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-18T11:57:04.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T11:57:04.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-18T11:57:04.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T11:57:04.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-18T11:57:04.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T11:57:04.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T11:57:04.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-18T11:57:04.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T11:57:04.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T11:57:04.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-18T11:57:04.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T11:57:04.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T11:57:04.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-18T11:57:04.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T11:57:04.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T11:57:04.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-18T11:57:04.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-18T11:57:04.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T11:57:04.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-18T11:57:04.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-18T11:57:04.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T11:57:04.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-18T11:57:04.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T11:57:04.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-18T11:57:04.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T11:57:04.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-18T11:57:04.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T11:57:04.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-18T11:57:04.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T11:57:04.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-18T11:57:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-18T11:57:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T11:57:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-18T11:57:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-18T11:57:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T11:57:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T11:57:04.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T11:57:04.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T11:57:04.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T11:57:04.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T11:57:04.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-18T11:57:04.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T11:57:04.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T11:57:04.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T11:57:04.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-18T11:57:04.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-18T11:57:04.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T11:57:04.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T11:57:04.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T11:57:04.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-18T11:57:04.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T11:57:04.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-18T11:57:04.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T11:57:04.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-18T11:57:04.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T11:57:04.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-18T11:57:04.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T11:57:04.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-18T11:57:04.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T11:57:04.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-18T11:57:04.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T11:57:04.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-18T11:57:04.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T11:57:04.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-18T11:57:04.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T11:57:04.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-18T11:57:04.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T11:57:04.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-18T11:57:04.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T11:57:04.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T11:57:04.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-18T11:57:04.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T11:57:04.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T11:57:04.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T11:57:04.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-18T11:57:04.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-18T11:57:04.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T11:57:04.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T11:57:04.970 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-18T11:57:04.970 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-18T11:57:04.970 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T11:57:04.970 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-18T11:57:04.970 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T11:57:04.971 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-18T11:57:04.971 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-18T11:57:04.971 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T11:57:04.971 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-18T11:57:04.971 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T11:57:04.972 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T11:57:04.972 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-18T11:57:04.972 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T11:57:04.972 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-18T11:57:04.972 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T11:57:04.973 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-18T11:57:04.973 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T11:57:04.973 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T11:57:04.973 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-18T11:57:04.973 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-18T11:57:04.974 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T11:57:04.974 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T11:57:04.974 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-18T11:57:04.974 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T11:57:04.974 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-18T11:57:04.975 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T11:57:04.975 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-18T11:57:04.975 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T11:57:04.975 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T11:57:04.975 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T11:57:04.975 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T11:57:04.976 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-18T11:57:04.976 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-18T11:57:04.976 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-18T11:57:04.976 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T11:57:04.976 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-18T11:57:04.977 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T11:57:04.977 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-18T11:57:04.977 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T11:57:04.977 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T11:57:04.977 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-18T11:57:04.978 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T11:57:04.978 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T11:57:04.978 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T11:57:04.978 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T11:57:04.978 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-18T11:57:04.979 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-18T11:57:04.979 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T11:57:04.979 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T11:57:04.979 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-18T11:57:04.979 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T11:57:04.979 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T11:57:04.980 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T11:57:04.980 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T11:57:04.980 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-18T11:57:04.980 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T11:57:04.980 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-18T11:57:04.981 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T11:57:04.981 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-18T11:57:04.981 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T11:57:04.981 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T11:57:04.981 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-18T11:57:04.982 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T11:57:04.982 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T11:57:04.982 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-18T11:57:04.982 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T11:57:04.982 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T11:57:04.983 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-18T11:57:04.983 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T11:57:04.983 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-18T11:57:04.983 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T11:57:04.983 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T11:57:04.984 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-18T11:57:04.984 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T11:57:04.984 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T11:57:04.984 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T11:57:04.984 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-18T11:57:04.984 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T11:57:04.985 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T11:57:04.985 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T11:57:04.985 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T11:57:04.985 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T11:57:04.985 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-18T11:57:04.986 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T11:57:04.986 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T11:57:04.986 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-18T11:57:04.986 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T11:57:04.986 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T11:57:04.987 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-18T11:57:04.987 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T11:57:04.987 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T11:57:04.987 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-18T11:57:04.987 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T11:57:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T11:57:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T11:57:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T11:57:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T11:57:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-18T11:57:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T11:57:04.989 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T11:57:04.989 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T11:57:04.989 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T11:57:04.989 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T11:57:04.989 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T11:57:04.990 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T11:57:04.990 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T11:57:04.990 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T11:57:04.990 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T11:57:04.990 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T11:57:04.991 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T11:57:04.991 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-18T11:57:04.991 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T11:57:04.991 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T11:57:04.991 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T11:57:04.992 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-18T11:57:04.992 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T11:57:04.992 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T11:57:04.992 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T11:57:04.992 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T11:57:04.992 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T11:57:04.993 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-18T11:57:04.993 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-18T11:57:04.993 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T11:57:04.993 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T11:57:04.993 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T11:57:04.994 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-18T11:57:04.994 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T11:57:04.994 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T11:57:04.994 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T11:57:04.994 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T11:57:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T11:57:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T11:57:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-18T11:57:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T11:57:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-18T11:57:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-18T11:57:04.996 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T11:57:04.996 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-18T11:57:04.996 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T11:57:04.996 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-18T11:57:04.996 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T11:57:04.997 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-18T11:57:04.997 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-18T11:57:04.997 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T11:57:04.997 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-18T11:57:04.997 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-18T11:57:04.998 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T11:57:04.998 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T11:57:04.998 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-18T11:57:04.998 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T11:57:04.998 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T11:57:04.998 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T11:57:04.999 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T11:57:04.999 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-18T11:57:04.999 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-18T11:57:04.999 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T11:57:04.999 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-18T11:57:05.000 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-18T11:57:05.000 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T11:57:05.000 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-18T11:57:05.000 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T11:57:05.000 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T11:57:05.000 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T11:57:05.001 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-18T11:57:05.001 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-18T11:57:05.001 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T11:57:05.001 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-18T11:57:05.001 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-18T11:57:05.002 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T11:57:05.002 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T11:57:05.002 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T11:57:05.002 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T11:57:05.002 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T11:57:05.003 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T11:57:05.003 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T11:57:05.003 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T11:57:05.003 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T11:57:05.003 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T11:57:05.003 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T11:57:05.004 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T11:57:05.004 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-18T11:57:05.004 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T11:57:05.004 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T11:57:05.004 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T11:57:05.005 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T11:57:05.005 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T11:57:05.005 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T11:57:05.005 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T11:57:05.005 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T11:57:05.006 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T11:57:05.006 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T11:57:05.006 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T11:57:05.006 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T11:57:05.006 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-18T11:57:05.007 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T11:57:05.007 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T11:57:05.007 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-18T11:57:05.007 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-18T11:57:05.007 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T11:57:05.007 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T11:57:05.008 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T11:57:05.008 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T11:57:05.008 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-18T11:57:05.008 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T11:57:05.008 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T11:57:05.009 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-18T11:57:05.009 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T11:57:05.009 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T11:57:05.009 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T11:57:05.009 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T11:57:05.010 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T11:57:05.010 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T11:57:05.010 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T11:57:05.010 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T11:57:05.010 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T11:57:05.011 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T11:57:05.011 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T11:57:05.011 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T11:57:05.011 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T11:57:05.011 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T11:57:05.011 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T11:57:05.012 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T11:57:05.012 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T11:57:05.012 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T11:57:05.012 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T11:57:05.012 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T11:57:05.013 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T11:57:05.013 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T11:57:05.013 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T11:57:05.013 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T11:57:05.013 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T11:57:05.014 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T11:57:05.014 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T11:57:05.014 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T11:57:05.014 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T11:57:05.014 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T11:57:05.014 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T11:57:05.015 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-18T11:57:05.015 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T11:57:05.015 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T11:57:05.015 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T11:57:05.015 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T11:57:05.016 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T11:57:05.016 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-18T11:57:05.016 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T11:57:05.016 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T11:57:05.016 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-18T11:57:05.016 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T11:57:05.017 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T11:57:05.017 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T11:57:05.017 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-18T11:57:05.017 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T11:57:05.017 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T11:57:05.018 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-18T11:57:05.018 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T11:57:05.018 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-18T11:57:05.018 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T11:57:05.018 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-18T11:57:05.019 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T11:57:05.019 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-18T11:57:05.019 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T11:57:05.019 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T11:57:05.020 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-18T11:57:05.020 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T11:57:05.020 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-18T11:57:05.020 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-18T11:57:05.021 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T11:57:05.021 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-18T11:57:05.021 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T11:57:05.021 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T11:57:05.022 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T11:57:05.022 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-18T11:57:05.022 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-18T11:57:05.022 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-18T11:57:05.023 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-18T11:57:05.023 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-18T11:57:05.023 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T11:57:05.023 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-18T11:57:05.023 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T11:57:05.024 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T11:57:05.024 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-18T11:57:05.024 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T11:57:05.024 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T11:57:05.024 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-18T11:57:05.025 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T11:57:05.025 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T11:57:05.025 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T11:57:05.025 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T11:57:05.025 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T11:57:05.025 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T11:57:05.026 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T11:57:05.026 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-18T11:57:05.026 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T11:57:05.026 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T11:57:05.026 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T11:57:05.027 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-18T11:57:05.027 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T11:57:05.027 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T11:57:05.027 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T11:57:05.027 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-18T11:57:05.028 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T11:57:05.028 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-18T11:57:05.028 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T11:57:05.028 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-18T11:57:05.028 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T11:57:05.028 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T11:57:05.029 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T11:57:05.029 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T11:57:05.029 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-18T11:57:05.029 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T11:57:05.029 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-18T11:57:05.030 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T11:57:05.030 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T11:57:05.030 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-18T11:57:05.030 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T11:57:05.030 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T11:57:05.031 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T11:57:05.031 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-18T11:57:05.031 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T11:57:05.031 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T11:57:05.031 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-18T11:57:05.031 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T11:57:05.032 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T11:57:05.032 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T11:57:05.032 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T11:57:05.032 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T11:57:05.032 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T11:57:05.033 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T11:57:05.033 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T11:57:05.033 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T11:57:05.033 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T11:57:05.033 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T11:57:05.033 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-18T11:57:05.034 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T11:57:05.034 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T11:57:05.034 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T11:57:05.034 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T11:57:05.034 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-18T11:57:05.035 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T11:57:05.035 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-18T11:57:05.035 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-18T11:57:05.035 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T11:57:05.035 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-18T11:57:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T11:57:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T11:57:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T11:57:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T11:57:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T11:57:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T11:57:05.037 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-18T11:57:05.037 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T11:57:05.037 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-18T11:57:05.037 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-18T11:57:05.037 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-18T11:57:05.038 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T11:57:05.038 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T11:57:05.038 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T11:57:05.038 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T11:57:05.038 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T11:57:05.039 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T11:57:05.039 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T11:57:05.039 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T11:57:05.039 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T11:57:05.039 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T11:57:05.040 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T11:57:05.040 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T11:57:05.040 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T11:57:05.040 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T11:57:05.040 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T11:57:05.041 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-18T11:57:05.041 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-18T11:57:05.041 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T11:57:05.041 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T11:57:05.041 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T11:57:05.041 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T11:57:05.042 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-18T11:57:05.042 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-18T11:57:05.042 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T11:57:05.042 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T11:57:05.042 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-18T11:57:05.043 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T11:57:05.043 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-18T11:57:05.043 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-18T11:57:05.043 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T11:57:05.043 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-18T11:57:05.043 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T11:57:05.044 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-18T11:57:05.044 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T11:57:05.044 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T11:57:05.044 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T11:57:05.044 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T11:57:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T11:57:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-18T11:57:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-18T11:57:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T11:57:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T11:57:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T11:57:05.046 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T11:57:05.046 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-18T11:57:05.046 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T11:57:05.046 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-18T11:57:05.046 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T11:57:05.047 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T11:57:05.047 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-18T11:57:05.047 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T11:57:05.047 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T11:57:05.047 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T11:57:05.048 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T11:57:05.048 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T11:57:05.048 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T11:57:05.048 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T11:57:05.048 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T11:57:05.048 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T11:57:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T11:57:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-18T11:57:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T11:57:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T11:57:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T11:57:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T11:57:05.050 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T11:57:05.050 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T11:57:05.050 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T11:57:05.050 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-18T11:57:05.051 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T11:57:05.051 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-18T11:57:05.051 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T11:57:05.051 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T11:57:05.051 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T11:57:05.051 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T11:57:05.052 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T11:57:05.052 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T11:57:05.052 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T11:57:05.052 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T11:57:05.052 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T11:57:05.053 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T11:57:05.053 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T11:57:05.053 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T11:57:05.053 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T11:57:05.053 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T11:57:05.053 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T11:57:05.054 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T11:57:05.054 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T11:57:05.054 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T11:57:05.054 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T11:57:05.054 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T11:57:05.055 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T11:57:05.055 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T11:57:05.055 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T11:57:05.055 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T11:57:05.055 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T11:57:05.055 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T11:57:05.056 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T11:57:05.056 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T11:57:05.056 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T11:57:05.056 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T11:57:05.056 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T11:57:05.057 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T11:57:05.057 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T11:57:05.057 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T11:57:05.057 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T11:57:05.057 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T11:57:05.057 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T11:57:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T11:57:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T11:57:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T11:57:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T11:57:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T11:57:05.059 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T11:57:05.059 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T11:57:05.059 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T11:57:05.059 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-18T11:57:05.059 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T11:57:05.059 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T11:57:05.060 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T11:57:05.060 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T11:57:05.060 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T11:57:05.060 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T11:57:05.060 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T11:57:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T11:57:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T11:57:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T11:57:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T11:57:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T11:57:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T11:57:05.062 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T11:57:05.062 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T11:57:05.062 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-18T11:57:05.062 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T11:57:05.062 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T11:57:05.063 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T11:57:05.063 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-18T11:57:05.063 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-18T11:57:05.063 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T11:57:05.063 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T11:57:05.064 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T11:57:05.064 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T11:57:05.064 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-18T11:57:05.064 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T11:57:05.064 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T11:57:05.065 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T11:57:05.065 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T11:57:05.065 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-18T11:57:05.065 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-18T11:57:05.065 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T11:57:05.066 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T11:57:05.066 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T11:57:05.066 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-18T11:57:05.066 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-18T11:57:05.066 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-18T11:57:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-18T11:57:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-18T11:57:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-18T11:57:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-18T11:57:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-18T11:57:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-18T11:57:05.068 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-18T11:57:05.068 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T11:57:05.068 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T11:57:05.068 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-18T11:57:05.068 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T11:57:05.069 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T11:57:05.069 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T11:57:05.069 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T11:57:05.069 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T11:57:05.069 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T11:57:05.070 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T11:57:05.070 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T11:57:05.070 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T11:57:05.070 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T11:57:05.070 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T11:57:05.070 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T11:57:05.071 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-18T11:57:05.071 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T11:57:05.071 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T11:57:05.071 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T11:57:05.071 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T11:57:05.072 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T11:57:05.072 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T11:57:05.072 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T11:57:05.072 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T11:57:05.072 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T11:57:05.072 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T11:57:05.073 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T11:57:05.073 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T11:57:05.073 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T11:57:05.073 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T11:57:05.073 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T11:57:05.073 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T11:57:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T11:57:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T11:57:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T11:57:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T11:57:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T11:57:05.075 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T11:57:05.075 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T11:57:05.075 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T11:57:05.075 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T11:57:05.075 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T11:57:05.075 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-18T11:57:05.076 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-18T11:57:05.076 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T11:57:05.076 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T11:57:05.076 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T11:57:05.076 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-18T11:57:05.077 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T11:57:05.077 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T11:57:05.077 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-18T11:57:05.077 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T11:57:05.077 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-18T11:57:05.077 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-18T11:57:05.078 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T11:57:05.078 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-18T11:57:05.078 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T11:57:05.078 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-18T11:57:05.078 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T11:57:05.079 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T11:57:05.079 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-18T11:57:05.079 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T11:57:05.079 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-18T11:57:05.079 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T11:57:05.079 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-18T11:57:05.080 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T11:57:05.080 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-18T11:57:05.080 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T11:57:05.080 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-18T11:57:05.080 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T11:57:05.080 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-18T11:57:05.081 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T11:57:05.081 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-18T11:57:05.081 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-18T11:57:05.081 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-18T11:57:05.081 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T11:57:05.082 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-18T11:57:05.082 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-18T11:57:05.082 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T11:57:05.082 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-18T11:57:05.082 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-18T11:57:05.083 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T11:57:05.083 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-18T11:57:05.083 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T11:57:05.083 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-18T11:57:05.083 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-18T11:57:05.083 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T11:57:05.084 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-18T11:57:05.084 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T11:57:05.084 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-18T11:57:05.084 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T11:57:05.084 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-18T11:57:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T11:57:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-18T11:57:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T11:57:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-18T11:57:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T11:57:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-18T11:57:05.086 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T11:57:05.086 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-18T11:57:05.086 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T11:57:05.086 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-18T11:57:05.086 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-18T11:57:05.087 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T11:57:05.087 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-18T11:57:05.087 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T11:57:05.087 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-18T11:57:05.087 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T11:57:05.087 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T11:57:05.088 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T11:57:05.088 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T11:57:05.088 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-18T11:57:05.088 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T11:57:05.088 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T11:57:05.089 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-18T11:57:05.089 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T11:57:05.089 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-18T11:57:05.089 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T11:57:05.089 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-18T11:57:05.090 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T11:57:05.090 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T11:57:05.090 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T11:57:05.090 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T11:57:05.090 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T11:57:05.090 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T11:57:05.091 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T11:57:05.091 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T11:57:05.091 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T11:57:05.091 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T11:57:05.091 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T11:57:05.092 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T11:57:05.092 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T11:57:05.092 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T11:57:05.092 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T11:57:05.092 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T11:57:05.093 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T11:57:05.093 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T11:57:05.093 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T11:57:05.093 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T11:57:05.093 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T11:57:05.093 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T11:57:05.094 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T11:57:05.094 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T11:57:05.094 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T11:57:05.094 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T11:57:05.094 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T11:57:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T11:57:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T11:57:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T11:57:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T11:57:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T11:57:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T11:57:05.096 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T11:57:05.096 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T11:57:05.096 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T11:57:05.096 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T11:57:05.096 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T11:57:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T11:57:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T11:57:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T11:57:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T11:57:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T11:57:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T11:57:05.098 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T11:57:05.098 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T11:57:05.098 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T11:57:05.098 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T11:57:05.098 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T11:57:05.099 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T11:57:05.099 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-18T11:57:05.099 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-18T11:57:05.099 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T11:57:05.099 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-18T11:57:05.099 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T11:57:05.100 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-18T11:57:05.100 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T11:57:05.100 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T11:57:05.100 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T11:57:05.100 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T11:57:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T11:57:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-18T11:57:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T11:57:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-18T11:57:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T11:57:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-18T11:57:05.102 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T11:57:05.102 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-18T11:57:05.102 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T11:57:05.102 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-18T11:57:05.102 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-18T11:57:05.103 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T11:57:05.103 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-18T11:57:05.103 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T11:57:05.103 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T11:57:05.103 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-18T11:57:05.104 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T11:57:05.104 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-18T11:57:05.104 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T11:57:05.104 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-18T11:57:05.104 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T11:57:05.104 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-18T11:57:05.105 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T11:57:05.105 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-18T11:57:05.105 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T11:57:05.105 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-18T11:57:05.105 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-18T11:57:05.106 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-18T11:57:05.106 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T11:57:05.106 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-18T11:57:05.106 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-18T11:57:05.106 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-18T11:57:05.107 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-18T11:57:05.107 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T11:57:05.107 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T11:57:05.107 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-18T11:57:05.107 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T11:57:05.108 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T11:57:05.108 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T11:57:05.108 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T11:57:05.108 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-18T11:57:05.108 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T11:57:05.109 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T11:57:05.109 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-18T11:57:05.109 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T11:57:05.109 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-18T11:57:05.109 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T11:57:05.110 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-18T11:57:05.110 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T11:57:05.110 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T11:57:05.110 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T11:57:05.110 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T11:57:05.111 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-18T11:57:05.111 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-18T11:57:05.111 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-18T11:57:05.111 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T11:57:05.111 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-18T11:57:05.112 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-18T11:57:05.112 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T11:57:05.112 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T11:57:05.112 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T11:57:05.112 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T11:57:05.112 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T11:57:05.113 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T11:57:05.113 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T11:57:05.113 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T11:57:05.113 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-18T11:57:05.113 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T11:57:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T11:57:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T11:57:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T11:57:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T11:57:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-18T11:57:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T11:57:05.115 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-18T11:57:05.115 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T11:57:05.115 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T11:57:05.115 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T11:57:05.115 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T11:57:05.115 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-18T11:57:05.116 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T11:57:05.116 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-18T11:57:05.116 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T11:57:05.116 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T11:57:05.116 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-18T11:57:05.117 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T11:57:05.117 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T11:57:05.117 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T11:57:05.117 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T11:57:05.117 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T11:57:05.117 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T11:57:05.118 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T11:57:05.118 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T11:57:05.118 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T11:57:05.118 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T11:57:05.118 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T11:57:05.119 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T11:57:05.119 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T11:57:05.119 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T11:57:05.119 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T11:57:05.119 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T11:57:05.119 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T11:57:05.120 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T11:57:05.120 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T11:57:05.120 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T11:57:05.120 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T11:57:05.120 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-18T11:57:05.120 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T11:57:05.121 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T11:57:05.121 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-18T11:57:05.121 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T11:57:05.121 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-18T11:57:05.121 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-18T11:57:05.121 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T11:57:05.122 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-18T11:57:05.122 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T11:57:05.122 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T11:57:05.122 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T11:57:05.122 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T11:57:05.123 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-18T11:57:05.123 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-18T11:57:05.123 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-18T11:57:05.123 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T11:57:05.123 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T11:57:05.124 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T11:57:05.124 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T11:57:05.124 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T11:57:05.124 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-18T11:57:05.124 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-18T11:57:05.124 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T11:57:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T11:57:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T11:57:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-18T11:57:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T11:57:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T11:57:05.125 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T11:57:05.126 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T11:57:05.126 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T11:57:05.126 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T11:57:05.126 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T11:57:05.126 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T11:57:05.126 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T11:57:05.127 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T11:57:05.127 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-18T11:57:05.127 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T11:57:05.127 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T11:57:05.127 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T11:57:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T11:57:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T11:57:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T11:57:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T11:57:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T11:57:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T11:57:05.129 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T11:57:05.129 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T11:57:05.129 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T11:57:05.129 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T11:57:05.129 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T11:57:05.129 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T11:57:05.130 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T11:57:05.130 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T11:57:05.130 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T11:57:05.130 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T11:57:05.130 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T11:57:05.131 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T11:57:05.131 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T11:57:05.131 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T11:57:05.131 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T11:57:05.131 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T11:57:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T11:57:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T11:57:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T11:57:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T11:57:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T11:57:05.133 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T11:57:05.133 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T11:57:05.133 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T11:57:05.133 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T11:57:05.133 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T11:57:05.133 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T11:57:05.134 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T11:57:05.134 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T11:57:05.134 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-18T11:57:05.134 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T11:57:05.134 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T11:57:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T11:57:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T11:57:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T11:57:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T11:57:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T11:57:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-18T11:57:05.136 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T11:57:05.136 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T11:57:05.136 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T11:57:05.136 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T11:57:05.136 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T11:57:05.136 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T11:57:05.137 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T11:57:05.137 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T11:57:05.137 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T11:57:05.137 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T11:57:05.137 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T11:57:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T11:57:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T11:57:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T11:57:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T11:57:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T11:57:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T11:57:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T11:57:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T11:57:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T11:57:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T11:57:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T11:57:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T11:57:05.140 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T11:57:05.140 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T11:57:05.140 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T11:57:05.140 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T11:57:05.140 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-18T11:57:05.141 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T11:57:05.141 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-18T11:57:05.141 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T11:57:05.141 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T11:57:05.141 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T11:57:05.141 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T11:57:05.142 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T11:57:05.142 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T11:57:05.142 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T11:57:05.142 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-18T11:57:05.142 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T11:57:05.143 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-18T11:57:05.143 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T11:57:05.143 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-18T11:57:05.143 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T11:57:05.143 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-18T11:57:05.144 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-18T11:57:05.144 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T11:57:05.144 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T11:57:05.144 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T11:57:05.144 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T11:57:05.144 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T11:57:05.145 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T11:57:05.145 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T11:57:05.145 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T11:57:05.145 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T11:57:05.145 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T11:57:05.146 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T11:57:05.146 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T11:57:05.146 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T11:57:05.146 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T11:57:05.146 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T11:57:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T11:57:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-18T11:57:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T11:57:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T11:57:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T11:57:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T11:57:05.148 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T11:57:05.148 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T11:57:05.148 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T11:57:05.148 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T11:57:05.148 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T11:57:05.148 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T11:57:05.149 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T11:57:05.149 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T11:57:05.149 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T11:57:05.149 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T11:57:05.150 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T11:57:05.150 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T11:57:05.150 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T11:57:05.150 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T11:57:05.150 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T11:57:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T11:57:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T11:57:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T11:57:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T11:57:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T11:57:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T11:57:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T11:57:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T11:57:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T11:57:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T11:57:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T11:57:05.153 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T11:57:05.153 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T11:57:05.153 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T11:57:05.153 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T11:57:05.153 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T11:57:05.154 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T11:57:05.154 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T11:57:05.154 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T11:57:05.154 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T11:57:05.154 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T11:57:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T11:57:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T11:57:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T11:57:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T11:57:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T11:57:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T11:57:05.156 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T11:57:05.156 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T11:57:05.156 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T11:57:05.156 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T11:57:05.156 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T11:57:05.157 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T11:57:05.157 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T11:57:05.157 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T11:57:05.157 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T11:57:05.157 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T11:57:05.157 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T11:57:05.158 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T11:57:05.158 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-18T11:57:05.158 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T11:57:05.158 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T11:57:05.158 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T11:57:05.159 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T11:57:05.159 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-18T11:57:05.159 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T11:57:05.159 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T11:57:05.159 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T11:57:05.159 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T11:57:05.160 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T11:57:05.160 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T11:57:05.160 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T11:57:05.160 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T11:57:05.160 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T11:57:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T11:57:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T11:57:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T11:57:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T11:57:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T11:57:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T11:57:05.162 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T11:57:05.162 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T11:57:05.162 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T11:57:05.162 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T11:57:05.162 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T11:57:05.163 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T11:57:05.163 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T11:57:05.163 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T11:57:05.163 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T11:57:05.163 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T11:57:05.164 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T11:57:05.164 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T11:57:05.164 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T11:57:05.164 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T11:57:05.164 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T11:57:05.164 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T11:57:05.165 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T11:57:05.165 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T11:57:05.165 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T11:57:05.165 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T11:57:05.165 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T11:57:05.165 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T11:57:05.166 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T11:57:05.166 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T11:57:05.166 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T11:57:05.166 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T11:57:05.166 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T11:57:05.166 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T11:57:05.167 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T11:57:05.167 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T11:57:05.167 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T11:57:05.167 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T11:57:05.167 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T11:57:05.168 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T11:57:05.168 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T11:57:05.168 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T11:57:05.168 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T11:57:05.168 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T11:57:05.168 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T11:57:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T11:57:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T11:57:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T11:57:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T11:57:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T11:57:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T11:57:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T11:57:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T11:57:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T11:57:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T11:57:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T11:57:05.171 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T11:57:05.171 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T11:57:05.171 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T11:57:05.171 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T11:57:05.171 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T11:57:05.172 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T11:57:05.172 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-18T11:57:05.172 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T11:57:05.172 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-18T11:57:05.172 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T11:57:05.172 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-18T11:57:05.173 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T11:57:05.173 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T11:57:05.173 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T11:57:05.173 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T11:57:05.173 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-18T11:57:05.173 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T11:57:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-18T11:57:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T11:57:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T11:57:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T11:57:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-18T11:57:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T11:57:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T11:57:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T11:57:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T11:57:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T11:57:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T11:57:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T11:57:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T11:57:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T11:57:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T11:57:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T11:57:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T11:57:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T11:57:05.177 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T11:57:05.177 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T11:57:05.177 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T11:57:05.177 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T11:57:05.177 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T11:57:05.177 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T11:57:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T11:57:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T11:57:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T11:57:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T11:57:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T11:57:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T11:57:05.179 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T11:57:05.179 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T11:57:05.179 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T11:57:05.179 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T11:57:05.179 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T11:57:05.179 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T11:57:05.180 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T11:57:05.180 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T11:57:05.180 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T11:57:05.180 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T11:57:05.180 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T11:57:05.180 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T11:57:05.181 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T11:57:05.181 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T11:57:05.181 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T11:57:05.181 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T11:57:05.181 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T11:57:05.182 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T11:57:05.182 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T11:57:05.182 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-18T11:57:05.182 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T11:57:05.182 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T11:57:05.182 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-18T11:57:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-18T11:57:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T11:57:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T11:57:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T11:57:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-18T11:57:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T11:57:05.184 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-18T11:57:05.184 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T11:57:05.184 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T11:57:05.184 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-18T11:57:05.184 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T11:57:05.184 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-18T11:57:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T11:57:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T11:57:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T11:57:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T11:57:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T11:57:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-18T11:57:05.186 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T11:57:05.186 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-18T11:57:05.186 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-18T11:57:05.186 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-18T11:57:05.186 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T11:57:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T11:57:05.188 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T11:57:05.188 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T11:57:05.188 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T11:57:05.188 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T11:57:05.188 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T11:57:05.189 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T11:57:05.189 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T11:57:05.189 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T11:57:05.189 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T11:57:05.189 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T11:57:05.190 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T11:57:05.190 DEBUG:teuthology.orchestra.run.smithi070:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-18T11:57:05.218 INFO:teuthology.orchestra.run.smithi070.stdout:postinstall scriptlet (using /bin/sh): 2023-10-18T11:57:05.219 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-18T11:57:05.219 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-18T11:57:05.219 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T11:57:05.219 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T11:57:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T11:57:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T11:57:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-10-18T11:57:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-18T11:57:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-18T11:57:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-18T11:57:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-18T11:57:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-18T11:57:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-10-18T11:57:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-18T11:57:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-18T11:57:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-18T11:57:05.222 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-10-18T11:57:05.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2023-10-18T11:57:05.302 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2023-10-18T11:57:05.303 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2023-10-18T11:57:05.303 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.02 2023-10-18T11:57:05.303 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 123.el8 2023-10-18T11:57:05.303 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2023-10-18T11:57:05.303 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-18T11:57:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:Group : System Environment/Base 2023-10-18T11:57:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 9265450 2023-10-18T11:57:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2023-10-18T11:57:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-18T11:57:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-18T11:57:05.305 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-18T11:57:05.305 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-18T11:57:05.305 INFO:teuthology.orchestra.run.smithi070.stdout:Relocations : (not relocatable) 2023-10-18T11:57:05.305 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : Red Hat, Inc. 2023-10-18T11:57:05.305 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : Red Hat, Inc. 2023-10-18T11:57:05.306 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T11:57:05.306 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2023-10-18T11:57:05.306 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2023-10-18T11:57:05.306 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T11:57:05.306 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T11:57:05.307 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T11:57:05.307 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T11:57:05.307 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2023-10-18T11:57:05.307 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T11:57:05.307 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T11:57:05.308 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T11:57:05.308 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 6.6.0-rc5-g47672119149b 2023-10-18T11:57:05.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T11:57:10.101 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2023-10-18T11:57:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:done 2023-10-18T11:57:11.963 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2023-10-18T11:57:11.986 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.ISNMrXDfk5 2023-10-18T11:57:11.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ISNMrXDfk5 2023-10-18T11:57:12.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.ISNMrXDfk5 2023-10-18T11:57:12.218 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.ISNMrXDfk5 is 6KB 2023-10-18T11:57:12.266 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.ISNMrXDfk5 2023-10-18T11:57:12.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T11:57:12.347 INFO:teuthology.orchestra.run.smithi070.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-18T11:57:12.347 INFO:teuthology.orchestra.run.smithi070.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-18T11:57:12.348 INFO:teuthology.orchestra.run.smithi070.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b.conf 2023-10-18T11:57:12.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b 2023-10-18T11:57:12.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2023-10-18T11:57:42.501 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-18T11:57:42.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-18T11:57:42.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T11:58:42.505 DEBUG:teuthology.orchestra.remote:timed out 2023-10-18T11:58:51.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-18T11:58:51.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T11:58:59.262 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-10-18T11:58:59.687 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-18T11:58:59.687 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-18T11:58:59.688 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-10-18T11:58:59.731 INFO:teuthology.orchestra.run.smithi070.stdout:6.6.0-rc5-g47672119149b 2023-10-18T11:58:59.731 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T11:58:59.731 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T11:58:59.732 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-18T11:59:00.733 DEBUG:teuthology.parallel:result is None 2023-10-18T11:59:00.734 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T11:59:00.742 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T11:59:00.743 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T11:59:00.748 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T11:59:00.751 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T11:59:00.787 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T11:59:00.794 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T11:59:00.801 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T11:59:00.802 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T11:59:00.829 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T11:59:00.833 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T11:59:00.895 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T11:59:00.901 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T11:59:00.901 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T11:59:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:59:00.920 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T11:59:00.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:59:00.939 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T11:59:00.944 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T11:59:00.963 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T11:59:00.998 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T11:59:01.018 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T11:59:01.042 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T11:59:01.064 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T11:59:01.072 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T11:59:01.122 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T11:59:01.126 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T11:59:01.133 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T11:59:01.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T11:59:01.138 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T11:59:01.140 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T11:59:01.205 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T11:59:01.217 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T11:59:01.218 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T11:59:01.224 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T11:59:01.227 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T11:59:01.266 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T11:59:01.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T11:59:01.419 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T11:59:01.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T11:59:01.558 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T11:59:01.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T11:59:01.628 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T11:59:01.676 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T11:59:01.729 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T11:59:01.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T11:59:01.861 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T11:59:01.862 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T11:59:01.948 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T11:59:01.996 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T11:59:02.056 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T11:59:02.100 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T11:59:02.145 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T11:59:02.146 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T11:59:02.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2023-10-18T11:59:02.218 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2023-10-18T11:59:02.221 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-10-18T11:59:02.275 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T11:59:02.316 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T11:59:02.331 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T11:59:02.685 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T11:59:02.694 INFO:teuthology.task.internal:Starting timer... 2023-10-18T11:59:02.694 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T11:59:02.704 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T11:59:02.713 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-18T11:59:02.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2023-10-18T11:59:02.827 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2023-10-18T11:59:02.830 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2023-10-18T11:59:02.894 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2023-10-18T11:59:02.895 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-10-18T11:59:02.946 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2023-10-18T11:59:02.948 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T11:59:02.948 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2023-10-18T11:59:03.017 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2023-10-18T11:59:03.018 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2023-10-18T11:59:03.042 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2023-10-18T11:59:03.042 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-10-18T11:59:03.061 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-10-18T11:59:03.062 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2023-10-18T11:59:03.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T11:59:03.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:59:03.131 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T11:59:03.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:59:03.159 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T11:59:03.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:59:03.187 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T11:59:03.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2023-10-18T11:59:03.255 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2023-10-18T11:59:03.281 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-10-18T11:59:03.308 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T11:59:03.320 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-18T11:59:03.326 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-18T11:59:03.433 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T11:59:03.451 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T11:59:03.452 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi152.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T12:11:33.867 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-10-18T12:11:33.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-18T12:11:33.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T12:11:33.960 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-10-18T12:11:34.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-18T12:11:34.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-10-18T12:11:34.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T12:11:34.145 DEBUG:teuthology.orchestra.run.smithi152:> true 2023-10-18T12:11:34.212 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-10-18T12:11:34.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-18T12:11:34.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T12:11:34.281 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-10-18T12:11:34.348 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-18T12:11:34.350 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T12:11:34.361 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T12:11:34.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T12:11:34.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:11:34.366 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T12:11:34.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:11:34.369 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T12:11:34.370 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:11:34.404 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T12:11:34.421 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T12:11:34.422 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T12:11:34.435 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T12:11:34.436 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T12:11:34.469 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T12:11:34.492 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2023-10-18T12:11:34.501 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-10-18T12:11:34.504 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2023-10-18T12:11:34.513 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-10-18T12:11:34.520 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T12:11:34.527 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T12:11:34.534 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T12:11:34.541 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T12:11:34.565 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2023-10-18T12:11:34.594 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2023-10-18T12:11:34.609 INFO:teuthology.orchestra.run.smithi152.stderr:bash: ntpq: command not found 2023-10-18T12:11:34.610 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-10-18T12:11:34.614 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:11:34.614 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2023-10-18T12:11:34.614 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.615 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.615 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.615 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.616 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:11:34.616 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-10-18T12:11:34.617 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.617 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.617 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.617 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.628 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T12:11:34.661 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T12:11:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:bash: ntpq: command not found 2023-10-18T12:11:34.756 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:11:34.756 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-10-18T12:11:34.757 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.757 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.757 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.757 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:11:34.759 INFO:teuthology.run_tasks:Running task pexec... 2023-10-18T12:11:34.769 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-18T12:11:34.769 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:11:34.770 DEBUG:teuthology.orchestra.run.smithi070:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-18T12:11:34.770 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi152.front.sepia.ceph.com 2023-10-18T12:11:34.771 DEBUG:teuthology.orchestra.run.smithi152:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-18T12:11:34.771 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi186.front.sepia.ceph.com 2023-10-18T12:11:34.771 DEBUG:teuthology.orchestra.run.smithi186:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-18T12:11:35.145 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-18T12:11:35.147 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-10-18T12:11:35.251 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-10-18T12:11:36.087 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:04:51 ago on Wed 18 Oct 2023 12:06:44 PM UTC. 2023-10-18T12:11:36.092 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:04:59 ago on Wed 18 Oct 2023 12:06:36 PM UTC. 2023-10-18T12:11:36.346 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:48 ago on Wed 18 Oct 2023 12:09:47 PM UTC. 2023-10-18T12:11:37.824 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-18T12:11:37.825 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-18T12:11:37.825 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2023-10-18T12:11:37.825 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-18T12:11:37.825 INFO:teuthology.orchestra.run.smithi186.stdout:Resetting modules: 2023-10-18T12:11:37.825 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools 2023-10-18T12:11:37.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:37.826 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-18T12:11:37.826 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-18T12:11:37.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:37.977 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-10-18T12:11:37.978 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-10-18T12:11:37.978 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2023-10-18T12:11:37.978 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-10-18T12:11:37.979 INFO:teuthology.orchestra.run.smithi152.stdout:Resetting modules: 2023-10-18T12:11:37.979 INFO:teuthology.orchestra.run.smithi152.stdout: container-tools 2023-10-18T12:11:37.979 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:37.979 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-10-18T12:11:37.980 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-10-18T12:11:37.980 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-10-18T12:11:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-10-18T12:11:38.139 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2023-10-18T12:11:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-10-18T12:11:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:Resetting modules: 2023-10-18T12:11:38.140 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools 2023-10-18T12:11:38.140 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-10-18T12:11:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-10-18T12:11:38.140 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:38.783 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:02 ago on Wed 18 Oct 2023 12:06:36 PM UTC. 2023-10-18T12:11:38.936 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:04:54 ago on Wed 18 Oct 2023 12:06:44 PM UTC. 2023-10-18T12:11:39.386 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:51 ago on Wed 18 Oct 2023 12:09:47 PM UTC. 2023-10-18T12:11:43.465 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-18T12:11:43.935 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-18T12:11:44.062 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-10-18T12:11:44.206 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-10-18T12:11:44.544 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-10-18T12:11:44.807 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-10-18T12:11:44.876 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:08 ago on Wed 18 Oct 2023 12:06:36 PM UTC. 2023-10-18T12:11:45.531 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:05:01 ago on Wed 18 Oct 2023 12:06:44 PM UTC. 2023-10-18T12:11:46.456 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:58 ago on Wed 18 Oct 2023 12:09:47 PM UTC. 2023-10-18T12:11:46.638 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-18T12:11:46.731 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-18T12:11:46.732 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-10-18T12:11:46.732 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-18T12:11:46.733 INFO:teuthology.orchestra.run.smithi186.stdout:Installing group/module packages: 2023-10-18T12:11:46.733 INFO:teuthology.orchestra.run.smithi186.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-18T12:11:46.733 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-18T12:11:46.734 INFO:teuthology.orchestra.run.smithi186.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-18T12:11:46.734 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-18T12:11:46.734 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-18T12:11:46.735 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-18T12:11:46.735 INFO:teuthology.orchestra.run.smithi186.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-18T12:11:46.735 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-10-18T12:11:46.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-18T12:11:46.735 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-18T12:11:46.736 INFO:teuthology.orchestra.run.smithi186.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-18T12:11:46.736 INFO:teuthology.orchestra.run.smithi186.stdout:Installing module profiles: 2023-10-18T12:11:46.736 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools/common 2023-10-18T12:11:46.736 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:46.736 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-10-18T12:11:46.737 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools rhel8 2023-10-18T12:11:46.737 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:46.737 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-18T12:11:46.737 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-18T12:11:46.737 INFO:teuthology.orchestra.run.smithi186.stdout:Install 10 Packages 2023-10-18T12:11:46.738 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:46.738 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 18 M 2023-10-18T12:11:46.738 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 65 M 2023-10-18T12:11:46.739 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-10-18T12:11:46.933 INFO:teuthology.orchestra.run.smithi186.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 125 kB/s | 25 kB 00:00 2023-10-18T12:11:46.957 INFO:teuthology.orchestra.run.smithi186.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 182 kB/s | 41 kB 00:00 2023-10-18T12:11:46.983 INFO:teuthology.orchestra.run.smithi186.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 377 kB/s | 94 kB 00:00 2023-10-18T12:11:47.050 INFO:teuthology.orchestra.run.smithi186.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 422 kB/s | 49 kB 00:00 2023-10-18T12:11:47.083 INFO:teuthology.orchestra.run.smithi186.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.1 MB/s | 524 kB 00:00 2023-10-18T12:11:47.230 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-10-18T12:11:47.284 INFO:teuthology.orchestra.run.smithi186.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 27 MB/s | 8.0 MB 00:00 2023-10-18T12:11:47.309 INFO:teuthology.orchestra.run.smithi186.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 577 kB/s | 149 kB 00:00 2023-10-18T12:11:47.322 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================================================ 2023-10-18T12:11:47.322 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-10-18T12:11:47.322 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================================================ 2023-10-18T12:11:47.323 INFO:teuthology.orchestra.run.smithi152.stdout:Installing group/module packages: 2023-10-18T12:11:47.323 INFO:teuthology.orchestra.run.smithi152.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-18T12:11:47.323 INFO:teuthology.orchestra.run.smithi152.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-18T12:11:47.323 INFO:teuthology.orchestra.run.smithi152.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-18T12:11:47.323 INFO:teuthology.orchestra.run.smithi152.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-18T12:11:47.324 INFO:teuthology.orchestra.run.smithi152.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-18T12:11:47.324 INFO:teuthology.orchestra.run.smithi152.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-18T12:11:47.324 INFO:teuthology.orchestra.run.smithi152.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-18T12:11:47.324 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-10-18T12:11:47.324 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-18T12:11:47.324 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-18T12:11:47.325 INFO:teuthology.orchestra.run.smithi152.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-18T12:11:47.325 INFO:teuthology.orchestra.run.smithi152.stdout:Installing module profiles: 2023-10-18T12:11:47.325 INFO:teuthology.orchestra.run.smithi152.stdout: container-tools/common 2023-10-18T12:11:47.325 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:47.325 INFO:teuthology.orchestra.run.smithi152.stdout:Enabling module streams: 2023-10-18T12:11:47.326 INFO:teuthology.orchestra.run.smithi152.stdout: container-tools rhel8 2023-10-18T12:11:47.326 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:47.326 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-10-18T12:11:47.326 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================================================ 2023-10-18T12:11:47.326 INFO:teuthology.orchestra.run.smithi152.stdout:Install 10 Packages 2023-10-18T12:11:47.327 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:47.327 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 18 M 2023-10-18T12:11:47.328 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 65 M 2023-10-18T12:11:47.328 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-10-18T12:11:47.334 INFO:teuthology.orchestra.run.smithi186.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 849 kB/s | 213 kB 00:00 2023-10-18T12:11:47.426 INFO:teuthology.orchestra.run.smithi186.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-10-18T12:11:47.495 INFO:teuthology.orchestra.run.smithi152.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 552 kB/s | 94 kB 00:00 2023-10-18T12:11:47.520 INFO:teuthology.orchestra.run.smithi152.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 127 kB/s | 25 kB 00:00 2023-10-18T12:11:47.526 INFO:teuthology.orchestra.run.smithi186.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 29 MB/s | 7.0 MB 00:00 2023-10-18T12:11:47.526 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:11:47.527 INFO:teuthology.orchestra.run.smithi186.stdout:Total 23 MB/s | 18 MB 00:00 2023-10-18T12:11:47.553 INFO:teuthology.orchestra.run.smithi152.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 178 kB/s | 41 kB 00:00 2023-10-18T12:11:47.637 INFO:teuthology.orchestra.run.smithi152.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 347 kB/s | 49 kB 00:00 2023-10-18T12:11:47.679 INFO:teuthology.orchestra.run.smithi152.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.2 MB/s | 524 kB 00:00 2023-10-18T12:11:47.704 INFO:teuthology.orchestra.run.smithi152.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-10-18T12:11:47.728 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-10-18T12:11:47.730 INFO:teuthology.orchestra.run.smithi152.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-18T12:11:47.746 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-10-18T12:11:47.747 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-10-18T12:11:47.888 INFO:teuthology.orchestra.run.smithi152.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 24 MB/s | 8.0 MB 00:00 2023-10-18T12:11:47.930 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-10-18T12:11:47.934 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-10-18T12:11:47.955 INFO:teuthology.orchestra.run.smithi152.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.7 MB/s | 2.2 MB 00:00 2023-10-18T12:11:48.089 INFO:teuthology.orchestra.run.smithi152.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-10-18T12:11:48.089 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:11:48.090 INFO:teuthology.orchestra.run.smithi152.stdout:Total 24 MB/s | 18 MB 00:00 2023-10-18T12:11:48.290 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-10-18T12:11:48.308 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-10-18T12:11:48.309 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-10-18T12:11:48.313 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-10-18T12:11:48.411 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================================ 2023-10-18T12:11:48.411 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-10-18T12:11:48.412 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================================ 2023-10-18T12:11:48.412 INFO:teuthology.orchestra.run.smithi070.stdout:Installing group/module packages: 2023-10-18T12:11:48.412 INFO:teuthology.orchestra.run.smithi070.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-18T12:11:48.412 INFO:teuthology.orchestra.run.smithi070.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-18T12:11:48.413 INFO:teuthology.orchestra.run.smithi070.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-18T12:11:48.413 INFO:teuthology.orchestra.run.smithi070.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-18T12:11:48.413 INFO:teuthology.orchestra.run.smithi070.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-18T12:11:48.413 INFO:teuthology.orchestra.run.smithi070.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-18T12:11:48.413 INFO:teuthology.orchestra.run.smithi070.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-18T12:11:48.414 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2023-10-18T12:11:48.414 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-18T12:11:48.414 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-18T12:11:48.414 INFO:teuthology.orchestra.run.smithi070.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-18T12:11:48.414 INFO:teuthology.orchestra.run.smithi070.stdout:Installing module profiles: 2023-10-18T12:11:48.415 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools/common 2023-10-18T12:11:48.415 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:48.415 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling module streams: 2023-10-18T12:11:48.415 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools rhel8 2023-10-18T12:11:48.415 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:48.415 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-10-18T12:11:48.416 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================================ 2023-10-18T12:11:48.416 INFO:teuthology.orchestra.run.smithi070.stdout:Install 10 Packages 2023-10-18T12:11:48.416 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:48.417 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 18 M 2023-10-18T12:11:48.417 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 65 M 2023-10-18T12:11:48.417 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-10-18T12:11:48.445 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-10-18T12:11:48.489 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-10-18T12:11:48.493 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-10-18T12:11:48.585 INFO:teuthology.orchestra.run.smithi070.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 549 kB/s | 94 kB 00:00 2023-10-18T12:11:48.610 INFO:teuthology.orchestra.run.smithi070.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 208 kB/s | 41 kB 00:00 2023-10-18T12:11:48.629 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-18T12:11:48.636 INFO:teuthology.orchestra.run.smithi070.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 112 kB/s | 25 kB 00:00 2023-10-18T12:11:48.661 INFO:teuthology.orchestra.run.smithi070.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 652 kB/s | 49 kB 00:00 2023-10-18T12:11:48.728 INFO:teuthology.orchestra.run.smithi070.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.4 MB/s | 524 kB 00:00 2023-10-18T12:11:48.785 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T12:11:48.854 INFO:teuthology.orchestra.run.smithi070.stdout:(6/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-10-18T12:11:48.962 INFO:teuthology.orchestra.run.smithi070.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 497 kB/s | 149 kB 00:00 2023-10-18T12:11:48.987 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-18T12:11:49.010 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-10-18T12:11:49.173 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-18T12:11:49.187 INFO:teuthology.orchestra.run.smithi070.stdout:(8/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-10-18T12:11:49.190 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-18T12:11:49.313 INFO:teuthology.orchestra.run.smithi070.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 12 MB/s | 8.0 MB 00:00 2023-10-18T12:11:49.377 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T12:11:49.379 INFO:teuthology.orchestra.run.smithi070.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 5.3 MB/s | 2.2 MB 00:00 2023-10-18T12:11:49.380 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:11:49.380 INFO:teuthology.orchestra.run.smithi070.stdout:Total 19 MB/s | 18 MB 00:00 2023-10-18T12:11:49.555 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-18T12:11:49.563 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-18T12:11:49.716 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-10-18T12:11:49.739 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-10-18T12:11:49.740 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-10-18T12:11:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-18T12:11:50.164 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-18T12:11:50.270 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-10-18T12:11:50.275 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-10-18T12:11:50.325 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-18T12:11:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-10-18T12:11:50.936 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-18T12:11:51.034 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-18T12:11:51.192 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-18T12:11:51.216 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T12:11:51.405 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-18T12:11:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-18T12:11:51.567 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-18T12:11:51.682 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-18T12:11:51.786 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T12:11:51.802 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-18T12:11:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-18T12:11:52.055 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T12:11:52.055 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-18T12:11:52.055 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T12:11:52.056 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-18T12:11:52.056 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-18T12:11:52.056 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-18T12:11:52.056 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-18T12:11:52.057 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-18T12:11:52.057 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-18T12:11:52.057 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-18T12:11:52.057 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-18T12:11:52.234 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-18T12:11:52.411 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T12:11:52.673 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T12:11:52.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-18T12:11:52.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T12:11:52.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-18T12:11:52.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-18T12:11:52.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-18T12:11:52.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-18T12:11:52.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-18T12:11:52.674 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-18T12:11:52.675 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-18T12:11:52.849 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-18T12:11:53.201 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-18T12:11:53.202 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:16 ago on Wed 18 Oct 2023 12:06:36 PM UTC. 2023-10-18T12:11:53.779 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-18T12:11:53.828 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-18T12:11:53.828 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:05:09 ago on Wed 18 Oct 2023 12:06:44 PM UTC. 2023-10-18T12:11:54.067 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-18T12:11:54.263 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-18T12:11:54.495 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T12:11:54.897 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T12:11:54.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-18T12:11:54.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T12:11:54.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-18T12:11:54.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-18T12:11:54.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-18T12:11:54.899 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-18T12:11:54.899 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-18T12:11:54.899 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-18T12:11:54.899 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-18T12:11:55.918 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-10-18T12:11:56.122 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:56.122 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-10-18T12:11:56.122 INFO:teuthology.orchestra.run.smithi186.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:56.122 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T12:11:56.123 INFO:teuthology.orchestra.run.smithi186.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:56.123 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-18T12:11:56.123 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-18T12:11:56.123 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-18T12:11:56.123 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:56.123 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:56.124 INFO:teuthology.orchestra.run.smithi186.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T12:11:56.124 INFO:teuthology.orchestra.run.smithi186.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-18T12:11:56.124 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:11:56.124 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-18T12:11:56.287 DEBUG:teuthology.parallel:result is None 2023-10-18T12:11:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-18T12:11:56.442 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:08 ago on Wed 18 Oct 2023 12:09:47 PM UTC. 2023-10-18T12:11:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:Installed products updated. 2023-10-18T12:11:57.567 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-10-18T12:11:57.568 INFO:teuthology.orchestra.run.smithi152.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:57.568 INFO:teuthology.orchestra.run.smithi152.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T12:11:57.568 INFO:teuthology.orchestra.run.smithi152.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:57.568 INFO:teuthology.orchestra.run.smithi152.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-18T12:11:57.568 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-18T12:11:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-18T12:11:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T12:11:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-18T12:11:57.570 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:11:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-10-18T12:11:57.758 DEBUG:teuthology.parallel:result is None 2023-10-18T12:11:59.352 INFO:teuthology.orchestra.run.smithi070.stdout:Installed products updated. 2023-10-18T12:11:59.551 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:59.551 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2023-10-18T12:11:59.551 INFO:teuthology.orchestra.run.smithi070.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:59.552 INFO:teuthology.orchestra.run.smithi070.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T12:11:59.552 INFO:teuthology.orchestra.run.smithi070.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:59.552 INFO:teuthology.orchestra.run.smithi070.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-18T12:11:59.552 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-18T12:11:59.552 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-18T12:11:59.553 INFO:teuthology.orchestra.run.smithi070.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:59.553 INFO:teuthology.orchestra.run.smithi070.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T12:11:59.553 INFO:teuthology.orchestra.run.smithi070.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T12:11:59.553 INFO:teuthology.orchestra.run.smithi070.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-18T12:11:59.553 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:11:59.554 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-10-18T12:11:59.760 DEBUG:teuthology.parallel:result is None 2023-10-18T12:11:59.761 INFO:teuthology.run_tasks:Running task install... 2023-10-18T12:11:59.770 DEBUG:teuthology.task.install:project ceph 2023-10-18T12:11:59.770 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}} 2023-10-18T12:11:59.771 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'} 2023-10-18T12:11:59.771 INFO:teuthology.task.install:Using flavor: default 2023-10-18T12:11:59.777 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T12:11:59.778 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-18T12:11:59.778 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T12:11:59.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:11:59.781 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T12:11:59.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:11:59.782 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T12:11:59.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:12:00.692 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/ 2023-10-18T12:12:00.693 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T12:12:01.867 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/ 2023-10-18T12:12:01.868 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T12:12:02.014 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T12:12:02.015 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:12:02.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T12:12:02.093 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T12:12:02.094 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T12:12:02.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T12:12:02.325 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T12:12:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2023-10-18T12:12:02.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2023-10-18T12:12:02.868 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-10-18T12:12:03.356 INFO:teuthology.orchestra.run.smithi070.stdout:44 files removed 2023-10-18T12:12:03.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T12:12:03.862 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-10-18T12:12:06.174 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/ 2023-10-18T12:12:06.174 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T12:12:08.684 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T12:12:08.684 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:12:08.684 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T12:12:08.719 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T12:12:08.720 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T12:12:08.792 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T12:12:08.873 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T12:12:08.941 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-10-18T12:12:08.942 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-10-18T12:12:09.205 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 19 kB/s | 81 kB 00:04 2023-10-18T12:12:09.315 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-18T12:12:10.388 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-10-18T12:12:10.442 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T12:12:10.777 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-18T12:12:18.055 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 1.1 kB/s | 9.4 kB 00:08 2023-10-18T12:12:18.181 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 12 kB/s | 81 kB 00:06 2023-10-18T12:12:18.804 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T12:12:18.805 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:12:18.805 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T12:12:18.851 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T12:12:18.851 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T12:12:18.922 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T12:12:19.003 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T12:12:19.072 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2023-10-18T12:12:19.073 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2023-10-18T12:12:19.453 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-10-18T12:12:19.840 INFO:teuthology.orchestra.run.smithi152.stdout:44 files removed 2023-10-18T12:12:19.887 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T12:12:20.220 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-10-18T12:12:22.812 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 39 kB/s | 81 kB 00:02 2023-10-18T12:12:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 204 B/s | 1.5 kB 00:07 2023-10-18T12:12:25.771 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-10-18T12:12:29.631 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 16 MB 00:03 2023-10-18T12:12:33.883 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 306 kB/s | 24 kB 00:00 2023-10-18T12:12:34.965 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 53 MB 00:01 2023-10-18T12:12:42.235 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 498 B/s | 9.4 kB 00:19 2023-10-18T12:12:43.628 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 47 MB 00:00 2023-10-18T12:12:47.169 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 312 B/s | 1.5 kB 00:04 2023-10-18T12:12:47.319 INFO:teuthology.orchestra.run.smithi152.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-18T12:12:48.337 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-10-18T12:12:50.847 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 296 B/s | 9.4 kB 00:32 2023-10-18T12:12:51.862 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 408 kB/s | 24 kB 00:00 2023-10-18T12:12:52.847 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 776 B/s | 1.5 kB 00:01 2023-10-18T12:12:52.847 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-10-18T12:12:52.997 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-18T12:12:53.931 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-18T12:12:55.457 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T12:12:55.458 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T12:12:55.473 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-18T12:12:55.475 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-18T12:12:55.476 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-18T12:12:55.477 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-18T12:12:55.479 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-18T12:12:55.483 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-18T12:12:55.484 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-18T12:12:55.623 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-10-18T12:12:55.635 INFO:teuthology.orchestra.run.smithi070.stdout:=========================================================================================================================== 2023-10-18T12:12:55.636 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-10-18T12:12:55.636 INFO:teuthology.orchestra.run.smithi070.stdout:=========================================================================================================================== 2023-10-18T12:12:55.636 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2023-10-18T12:12:55.636 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-18T12:12:55.636 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-18T12:12:55.637 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.4 k 2023-10-18T12:12:55.637 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 5.4 M 2023-10-18T12:12:55.637 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 919 k 2023-10-18T12:12:55.637 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 167 k 2023-10-18T12:12:55.637 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 1.6 M 2023-10-18T12:12:55.638 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 132 k 2023-10-18T12:12:55.638 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 1.8 M 2023-10-18T12:12:55.638 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 7.4 M 2023-10-18T12:12:55.638 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 47 k 2023-10-18T12:12:55.638 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 14 M 2023-10-18T12:12:55.639 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 49 M 2023-10-18T12:12:55.639 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 261 k 2023-10-18T12:12:55.639 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 192 k 2023-10-18T12:12:55.639 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 261 k 2023-10-18T12:12:55.639 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 18 k 2023-10-18T12:12:55.640 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-18T12:12:55.640 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-18T12:12:55.640 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-18T12:12:55.641 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 31 k 2023-10-18T12:12:55.641 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 840 k 2023-10-18T12:12:55.641 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 139 k 2023-10-18T12:12:55.641 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-18T12:12:55.641 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-18T12:12:55.642 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-18T12:12:55.642 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-18T12:12:55.642 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-18T12:12:55.642 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-18T12:12:55.642 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 196 k 2023-10-18T12:12:55.643 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 370 k 2023-10-18T12:12:55.643 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 364 k 2023-10-18T12:12:55.643 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 107 k 2023-10-18T12:12:55.643 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 94 k 2023-10-18T12:12:55.643 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.6 M 2023-10-18T12:12:55.644 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 178 k 2023-10-18T12:12:55.644 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-18T12:12:55.644 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2023-10-18T12:12:55.644 INFO:teuthology.orchestra.run.smithi070.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-18T12:12:55.644 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T12:12:55.645 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T12:12:55.645 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2023-10-18T12:12:55.645 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-18T12:12:55.645 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 23 M 2023-10-18T12:12:55.645 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 23 k 2023-10-18T12:12:55.646 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 2.3 M 2023-10-18T12:12:55.646 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 242 k 2023-10-18T12:12:55.646 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 4.6 M 2023-10-18T12:12:55.646 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 18 M 2023-10-18T12:12:55.646 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 15 k 2023-10-18T12:12:55.647 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 24 k 2023-10-18T12:12:55.647 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T12:12:55.647 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T12:12:55.647 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-18T12:12:55.647 INFO:teuthology.orchestra.run.smithi070.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-18T12:12:55.647 INFO:teuthology.orchestra.run.smithi070.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-18T12:12:55.648 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 185 k 2023-10-18T12:12:55.648 INFO:teuthology.orchestra.run.smithi070.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-18T12:12:55.648 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-18T12:12:55.648 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T12:12:55.648 INFO:teuthology.orchestra.run.smithi070.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-18T12:12:55.649 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-18T12:12:55.649 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T12:12:55.649 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 518 k 2023-10-18T12:12:55.649 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-18T12:12:55.649 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.9 M 2023-10-18T12:12:55.650 INFO:teuthology.orchestra.run.smithi070.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-18T12:12:55.650 INFO:teuthology.orchestra.run.smithi070.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-18T12:12:55.650 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T12:12:55.650 INFO:teuthology.orchestra.run.smithi070.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-18T12:12:55.651 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T12:12:55.651 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T12:12:55.651 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T12:12:55.651 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-18T12:12:55.651 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-18T12:12:55.651 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-18T12:12:55.652 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-18T12:12:55.652 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-18T12:12:55.652 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T12:12:55.652 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T12:12:55.652 INFO:teuthology.orchestra.run.smithi070.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T12:12:55.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T12:12:55.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 45 k 2023-10-18T12:12:55.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 125 k 2023-10-18T12:12:55.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T12:12:55.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-18T12:12:55.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T12:12:55.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T12:12:55.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-18T12:12:55.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T12:12:55.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T12:12:55.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T12:12:55.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T12:12:55.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T12:12:55.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T12:12:55.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T12:12:55.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T12:12:55.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T12:12:55.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T12:12:55.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T12:12:55.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T12:12:55.657 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-18T12:12:55.657 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T12:12:55.657 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T12:12:55.657 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T12:12:55.657 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T12:12:55.658 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T12:12:55.658 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T12:12:55.658 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-18T12:12:55.658 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T12:12:55.658 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T12:12:55.659 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-18T12:12:55.659 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T12:12:55.659 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T12:12:55.659 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T12:12:55.659 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T12:12:55.659 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T12:12:55.660 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-18T12:12:55.660 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T12:12:55.660 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T12:12:55.660 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T12:12:55.660 INFO:teuthology.orchestra.run.smithi070.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T12:12:55.661 INFO:teuthology.orchestra.run.smithi070.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T12:12:55.661 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T12:12:55.661 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T12:12:55.661 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T12:12:55.661 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T12:12:55.662 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T12:12:55.662 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-18T12:12:55.662 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-18T12:12:55.662 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-18T12:12:55.662 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T12:12:55.663 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-18T12:12:55.663 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling module streams: 2023-10-18T12:12:55.663 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql 10 2023-10-18T12:12:55.663 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:12:55.663 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-10-18T12:12:55.663 INFO:teuthology.orchestra.run.smithi070.stdout:=========================================================================================================================== 2023-10-18T12:12:55.664 INFO:teuthology.orchestra.run.smithi070.stdout:Install 128 Packages 2023-10-18T12:12:55.664 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 3 Packages 2023-10-18T12:12:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:12:55.665 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 209 M 2023-10-18T12:12:55.665 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-10-18T12:12:55.882 INFO:teuthology.orchestra.run.smithi070.stdout:(1/131): ceph-18.0.0-6786.gf7420d7f.el8.x86_64. 40 kB/s | 6.4 kB 00:00 2023-10-18T12:12:56.615 INFO:teuthology.orchestra.run.smithi070.stdout:(2/131): ceph-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.2 MB/s | 919 kB 00:00 2023-10-18T12:12:56.707 INFO:teuthology.orchestra.run.smithi070.stdout:(3/131): ceph-immutable-object-cache-18.0.0-678 1.8 MB/s | 167 kB 00:00 2023-10-18T12:12:57.199 INFO:teuthology.orchestra.run.smithi070.stdout:(4/131): ceph-mds-18.0.0-6786.gf7420d7f.el8.x86 4.7 MB/s | 2.3 MB 00:00 2023-10-18T12:12:57.416 INFO:teuthology.orchestra.run.smithi070.stdout:(5/131): ceph-mgr-18.0.0-6786.gf7420d7f.el8.x86 7.5 MB/s | 1.6 MB 00:00 2023-10-18T12:12:57.431 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 361 kB/s | 24 kB 00:00 2023-10-18T12:12:57.917 INFO:teuthology.orchestra.run.smithi070.stdout:(6/131): ceph-mon-18.0.0-6786.gf7420d7f.el8.x86 9.3 MB/s | 4.6 MB 00:00 2023-10-18T12:12:58.325 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-10-18T12:12:58.650 INFO:teuthology.orchestra.run.smithi070.stdout:(7/131): ceph-common-18.0.0-6786.gf7420d7f.el8. 7.7 MB/s | 23 MB 00:02 2023-10-18T12:12:59.134 INFO:teuthology.orchestra.run.smithi070.stdout:(8/131): ceph-base-18.0.0-6786.gf7420d7f.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-10-18T12:12:59.259 INFO:teuthology.orchestra.run.smithi070.stdout:(9/131): ceph-selinux-18.0.0-6786.gf7420d7f.el8 195 kB/s | 24 kB 00:00 2023-10-18T12:12:59.759 INFO:teuthology.orchestra.run.smithi070.stdout:(10/131): ceph-osd-18.0.0-6786.gf7420d7f.el8.x8 9.7 MB/s | 18 MB 00:01 2023-10-18T12:12:59.917 INFO:teuthology.orchestra.run.smithi070.stdout:(11/131): ceph-radosgw-18.0.0-6786.gf7420d7f.el 11 MB/s | 14 MB 00:01 2023-10-18T12:12:59.943 INFO:teuthology.orchestra.run.smithi070.stdout:(12/131): cephfs-mirror-18.0.0-6786.gf7420d7f.e 1.4 MB/s | 261 kB 00:00 2023-10-18T12:12:59.959 INFO:teuthology.orchestra.run.smithi070.stdout:(13/131): libcephfs-devel-18.0.0-6786.gf7420d7f 745 kB/s | 31 kB 00:00 2023-10-18T12:13:00.010 INFO:teuthology.orchestra.run.smithi070.stdout:(14/131): libcephsqlite-18.0.0-6786.gf7420d7f.e 3.6 MB/s | 185 kB 00:00 2023-10-18T12:13:00.052 INFO:teuthology.orchestra.run.smithi070.stdout:(15/131): libcephfs2-18.0.0-6786.gf7420d7f.el8. 7.6 MB/s | 840 kB 00:00 2023-10-18T12:13:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:(16/131): librados-devel-18.0.0-6786.gf7420d7f. 2.3 MB/s | 139 kB 00:00 2023-10-18T12:13:00.110 INFO:teuthology.orchestra.run.smithi070.stdout:(17/131): libradosstriper1-18.0.0-6786.gf7420d7 8.7 MB/s | 518 kB 00:00 2023-10-18T12:13:00.186 INFO:teuthology.orchestra.run.smithi070.stdout:(18/131): python3-ceph-argparse-18.0.0-6786.gf7 597 kB/s | 45 kB 00:00 2023-10-18T12:13:00.261 INFO:teuthology.orchestra.run.smithi070.stdout:(19/131): python3-ceph-common-18.0.0-6786.gf742 1.6 MB/s | 125 kB 00:00 2023-10-18T12:13:00.337 INFO:teuthology.orchestra.run.smithi070.stdout:(20/131): python3-cephfs-18.0.0-6786.gf7420d7f. 2.6 MB/s | 196 kB 00:00 2023-10-18T12:13:00.420 INFO:teuthology.orchestra.run.smithi070.stdout:(21/131): python3-rados-18.0.0-6786.gf7420d7f.e 4.3 MB/s | 370 kB 00:00 2023-10-18T12:13:00.587 INFO:teuthology.orchestra.run.smithi070.stdout:(22/131): librgw2-18.0.0-6786.gf7420d7f.el8.x86 13 MB/s | 6.9 MB 00:00 2023-10-18T12:13:00.629 INFO:teuthology.orchestra.run.smithi070.stdout:(23/131): python3-rbd-18.0.0-6786.gf7420d7f.el8 1.7 MB/s | 364 kB 00:00 2023-10-18T12:13:00.654 INFO:teuthology.orchestra.run.smithi070.stdout:(24/131): python3-rgw-18.0.0-6786.gf7420d7f.el8 1.6 MB/s | 107 kB 00:00 2023-10-18T12:13:00.680 INFO:teuthology.orchestra.run.smithi070.stdout:(25/131): rbd-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.8 MB/s | 94 kB 00:00 2023-10-18T12:13:00.747 INFO:teuthology.orchestra.run.smithi070.stdout:(26/131): rbd-nbd-18.0.0-6786.gf7420d7f.el8.x86 2.6 MB/s | 178 kB 00:00 2023-10-18T12:13:00.839 INFO:teuthology.orchestra.run.smithi070.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6786.g 252 kB/s | 23 kB 00:00 2023-10-18T12:13:00.906 INFO:teuthology.orchestra.run.smithi070.stdout:(28/131): rbd-mirror-18.0.0-6786.gf7420d7f.el8. 14 MB/s | 3.6 MB 00:00 2023-10-18T12:13:00.931 INFO:teuthology.orchestra.run.smithi070.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.4 MB/s | 132 kB 00:00 2023-10-18T12:13:01.057 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-10-18T12:13:01.073 INFO:teuthology.orchestra.run.smithi070.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6786.gf7420 11 MB/s | 1.8 MB 00:00 2023-10-18T12:13:01.173 INFO:teuthology.orchestra.run.smithi070.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6786.gf7 2.4 MB/s | 242 kB 00:00 2023-10-18T12:13:01.257 INFO:teuthology.orchestra.run.smithi070.stdout:(32/131): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 571 kB/s | 47 kB 00:00 2023-10-18T12:13:01.363 INFO:teuthology.orchestra.run.smithi070.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6786.gf 142 kB/s | 15 kB 00:00 2023-10-18T12:13:01.466 INFO:teuthology.orchestra.run.smithi070.stdout:(34/131): ceph-volume-18.0.0-6786.gf7420d7f.el8 2.5 MB/s | 261 kB 00:00 2023-10-18T12:13:01.558 INFO:teuthology.orchestra.run.smithi070.stdout:(35/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-10-18T12:13:01.583 INFO:teuthology.orchestra.run.smithi070.stdout:(36/131): cephadm-18.0.0-6786.gf7420d7f.el8.noa 1.6 MB/s | 192 kB 00:00 2023-10-18T12:13:01.608 INFO:teuthology.orchestra.run.smithi070.stdout:(37/131): cephfs-top-18.0.0-6786.gf7420d7f.el8. 363 kB/s | 18 kB 00:00 2023-10-18T12:13:02.171 INFO:teuthology.orchestra.run.smithi070.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 105 kB/s | 59 kB 00:00 2023-10-18T12:13:02.375 INFO:teuthology.orchestra.run.smithi070.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 387 kB/s | 306 kB 00:00 2023-10-18T12:13:02.400 INFO:teuthology.orchestra.run.smithi070.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 327 kB/s | 75 kB 00:00 2023-10-18T12:13:02.609 INFO:teuthology.orchestra.run.smithi070.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 209 kB/s | 44 kB 00:00 2023-10-18T12:13:02.676 INFO:teuthology.orchestra.run.smithi070.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2023-10-18T12:13:02.835 INFO:teuthology.orchestra.run.smithi070.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 208 kB/s | 33 kB 00:00 2023-10-18T12:13:03.087 INFO:teuthology.orchestra.run.smithi070.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 388 kB/s | 185 kB 00:00 2023-10-18T12:13:03.111 INFO:teuthology.orchestra.run.smithi070.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 59 kB/s | 16 kB 00:00 2023-10-18T12:13:03.269 INFO:teuthology.orchestra.run.smithi070.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 948 kB/s | 173 kB 00:00 2023-10-18T12:13:03.295 INFO:teuthology.orchestra.run.smithi070.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-10-18T12:13:03.428 INFO:teuthology.orchestra.run.smithi070.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-10-18T12:13:03.454 INFO:teuthology.orchestra.run.smithi070.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 442 kB/s | 81 kB 00:00 2023-10-18T12:13:03.554 INFO:teuthology.orchestra.run.smithi070.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 141 kB/s | 18 kB 00:00 2023-10-18T12:13:03.688 INFO:teuthology.orchestra.run.smithi070.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2023-10-18T12:13:03.821 INFO:teuthology.orchestra.run.smithi070.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 442 kB/s | 59 kB 00:00 2023-10-18T12:13:03.855 INFO:teuthology.orchestra.run.smithi070.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 2.9 MB/s | 1.2 MB 00:00 2023-10-18T12:13:03.955 INFO:teuthology.orchestra.run.smithi070.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2023-10-18T12:13:03.997 INFO:teuthology.orchestra.run.smithi070.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-10-18T12:13:04.081 INFO:teuthology.orchestra.run.smithi070.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 125 kB/s | 16 kB 00:00 2023-10-18T12:13:04.123 INFO:teuthology.orchestra.run.smithi070.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 269 kB/s | 34 kB 00:00 2023-10-18T12:13:04.223 INFO:teuthology.orchestra.run.smithi070.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-10-18T12:13:04.249 INFO:teuthology.orchestra.run.smithi070.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2023-10-18T12:13:04.348 INFO:teuthology.orchestra.run.smithi070.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 149 kB/s | 19 kB 00:00 2023-10-18T12:13:04.374 INFO:teuthology.orchestra.run.smithi070.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 191 kB/s | 24 kB 00:00 2023-10-18T12:13:04.474 INFO:teuthology.orchestra.run.smithi070.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 234 kB/s | 29 kB 00:00 2023-10-18T12:13:04.499 INFO:teuthology.orchestra.run.smithi070.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-10-18T12:13:04.641 INFO:teuthology.orchestra.run.smithi070.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.5 MB/s | 255 kB 00:00 2023-10-18T12:13:04.702 INFO:teuthology.orchestra.run.smithi070.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-10-18T12:13:04.775 INFO:teuthology.orchestra.run.smithi070.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 458 kB/s | 61 kB 00:00 2023-10-18T12:13:04.817 INFO:teuthology.orchestra.run.smithi070.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 746 kB/s | 85 kB 00:00 2023-10-18T12:13:04.901 INFO:teuthology.orchestra.run.smithi070.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2023-10-18T12:13:04.926 INFO:teuthology.orchestra.run.smithi070.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-18T12:13:04.976 INFO:teuthology.orchestra.run.smithi070.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 941 kB/s | 47 kB 00:00 2023-10-18T12:13:05.018 INFO:teuthology.orchestra.run.smithi070.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-10-18T12:13:05.060 INFO:teuthology.orchestra.run.smithi070.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 7.2 MB/s | 1.7 MB 00:00 2023-10-18T12:13:05.114 INFO:teuthology.orchestra.run.smithi070.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 446 kB/s | 43 kB 00:00 2023-10-18T12:13:05.178 INFO:teuthology.orchestra.run.smithi070.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-18T12:13:05.203 INFO:teuthology.orchestra.run.smithi070.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 579 kB/s | 48 kB 00:00 2023-10-18T12:13:05.228 INFO:teuthology.orchestra.run.smithi070.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-18T12:13:05.245 INFO:teuthology.orchestra.run.smithi070.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-10-18T12:13:05.270 INFO:teuthology.orchestra.run.smithi070.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-10-18T12:13:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-10-18T12:13:05.346 INFO:teuthology.orchestra.run.smithi070.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-10-18T12:13:05.371 INFO:teuthology.orchestra.run.smithi070.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 281 kB/s | 19 kB 00:00 2023-10-18T12:13:05.397 INFO:teuthology.orchestra.run.smithi070.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-10-18T12:13:05.430 INFO:teuthology.orchestra.run.smithi070.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-10-18T12:13:05.456 INFO:teuthology.orchestra.run.smithi070.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-10-18T12:13:05.509 INFO:teuthology.orchestra.run.smithi070.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.6 MB/s | 605 kB 00:00 2023-10-18T12:13:05.548 INFO:teuthology.orchestra.run.smithi070.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-10-18T12:13:05.590 INFO:teuthology.orchestra.run.smithi070.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-10-18T12:13:05.623 INFO:teuthology.orchestra.run.smithi070.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-10-18T12:13:05.682 INFO:teuthology.orchestra.run.smithi070.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-10-18T12:13:05.707 INFO:teuthology.orchestra.run.smithi070.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 716 kB/s | 60 kB 00:00 2023-10-18T12:13:05.732 INFO:teuthology.orchestra.run.smithi070.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-18T12:13:05.758 INFO:teuthology.orchestra.run.smithi070.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-18T12:13:05.783 INFO:teuthology.orchestra.run.smithi070.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-10-18T12:13:05.825 INFO:teuthology.orchestra.run.smithi070.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-10-18T12:13:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-10-18T12:13:05.875 INFO:teuthology.orchestra.run.smithi070.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-18T12:13:05.943 INFO:teuthology.orchestra.run.smithi070.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-10-18T12:13:05.984 INFO:teuthology.orchestra.run.smithi070.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-10-18T12:13:06.027 INFO:teuthology.orchestra.run.smithi070.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-18T12:13:06.068 INFO:teuthology.orchestra.run.smithi070.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-10-18T12:13:06.110 INFO:teuthology.orchestra.run.smithi070.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 935 kB/s | 39 kB 00:00 2023-10-18T12:13:06.161 INFO:teuthology.orchestra.run.smithi070.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-10-18T12:13:06.211 INFO:teuthology.orchestra.run.smithi070.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-10-18T12:13:06.253 INFO:teuthology.orchestra.run.smithi070.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-10-18T12:13:06.316 INFO:teuthology.orchestra.run.smithi070.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.0 MB/s | 187 kB 00:00 2023-10-18T12:13:06.379 INFO:teuthology.orchestra.run.smithi070.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2023-10-18T12:13:06.434 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-10-18T12:13:06.537 INFO:teuthology.orchestra.run.smithi070.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2023-10-18T12:13:06.579 INFO:teuthology.orchestra.run.smithi070.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 645 kB/s | 27 kB 00:00 2023-10-18T12:13:06.621 INFO:teuthology.orchestra.run.smithi070.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-18T12:13:06.655 INFO:teuthology.orchestra.run.smithi070.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-10-18T12:13:06.763 INFO:teuthology.orchestra.run.smithi070.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-10-18T12:13:06.814 INFO:teuthology.orchestra.run.smithi070.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-10-18T12:13:06.856 INFO:teuthology.orchestra.run.smithi070.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-10-18T12:13:06.931 INFO:teuthology.orchestra.run.smithi070.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-10-18T12:13:06.956 INFO:teuthology.orchestra.run.smithi070.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 640 kB/s | 64 kB 00:00 2023-10-18T12:13:06.981 INFO:teuthology.orchestra.run.smithi070.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-18T12:13:07.007 INFO:teuthology.orchestra.run.smithi070.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-10-18T12:13:07.032 INFO:teuthology.orchestra.run.smithi070.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-18T12:13:07.066 INFO:teuthology.orchestra.run.smithi070.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-10-18T12:13:07.108 INFO:teuthology.orchestra.run.smithi070.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-18T12:13:07.241 INFO:teuthology.orchestra.run.smithi070.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 23 MB/s | 4.9 MB 00:00 2023-10-18T12:13:07.283 INFO:teuthology.orchestra.run.smithi070.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-18T12:13:07.358 INFO:teuthology.orchestra.run.smithi070.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-10-18T12:13:07.392 INFO:teuthology.orchestra.run.smithi070.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-10-18T12:13:07.451 INFO:teuthology.orchestra.run.smithi070.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-10-18T12:13:07.553 INFO:teuthology.orchestra.run.smithi070.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-10-18T12:13:07.643 INFO:teuthology.orchestra.run.smithi070.stdout:(127/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-10-18T12:13:07.893 INFO:teuthology.orchestra.run.smithi070.stdout:(128/131): librados2-18.0.0-6786.gf7420d7f.el8. 11 MB/s | 3.7 MB 00:00 2023-10-18T12:13:07.973 INFO:teuthology.orchestra.run.smithi070.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 9.8 MB/s | 798 kB 00:00 2023-10-18T12:13:08.169 INFO:teuthology.orchestra.run.smithi070.stdout:(130/131): librbd1-18.0.0-6786.gf7420d7f.el8.x8 7.1 MB/s | 3.7 MB 00:00 2023-10-18T12:13:10.636 INFO:teuthology.orchestra.run.smithi070.stdout:(131/131): ceph-test-18.0.0-6786.gf7420d7f.el8. 4.3 MB/s | 49 MB 00:11 2023-10-18T12:13:10.639 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:13:10.639 INFO:teuthology.orchestra.run.smithi070.stdout:Total 14 MB/s | 209 MB 00:14 2023-10-18T12:13:11.980 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T12:13:11.982 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T12:13:11.996 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-18T12:13:11.997 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-18T12:13:11.998 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-18T12:13:12.000 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-18T12:13:12.002 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-18T12:13:12.005 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-18T12:13:12.006 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-18T12:13:12.136 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-10-18T12:13:12.148 INFO:teuthology.orchestra.run.smithi152.stdout:=========================================================================================================================== 2023-10-18T12:13:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-10-18T12:13:12.149 INFO:teuthology.orchestra.run.smithi152.stdout:=========================================================================================================================== 2023-10-18T12:13:12.149 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-10-18T12:13:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-18T12:13:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-18T12:13:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.4 k 2023-10-18T12:13:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 5.4 M 2023-10-18T12:13:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 919 k 2023-10-18T12:13:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 167 k 2023-10-18T12:13:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 1.6 M 2023-10-18T12:13:12.151 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 132 k 2023-10-18T12:13:12.151 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 1.8 M 2023-10-18T12:13:12.151 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 7.4 M 2023-10-18T12:13:12.151 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 47 k 2023-10-18T12:13:12.151 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 14 M 2023-10-18T12:13:12.151 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 49 M 2023-10-18T12:13:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 261 k 2023-10-18T12:13:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 192 k 2023-10-18T12:13:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 261 k 2023-10-18T12:13:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 18 k 2023-10-18T12:13:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-18T12:13:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-18T12:13:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-18T12:13:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 31 k 2023-10-18T12:13:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 840 k 2023-10-18T12:13:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 139 k 2023-10-18T12:13:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-18T12:13:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-18T12:13:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-18T12:13:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-18T12:13:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-18T12:13:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-18T12:13:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 196 k 2023-10-18T12:13:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 370 k 2023-10-18T12:13:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 364 k 2023-10-18T12:13:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 107 k 2023-10-18T12:13:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 94 k 2023-10-18T12:13:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.6 M 2023-10-18T12:13:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 178 k 2023-10-18T12:13:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-18T12:13:12.156 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2023-10-18T12:13:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-18T12:13:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T12:13:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T12:13:12.157 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-10-18T12:13:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-18T12:13:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 23 M 2023-10-18T12:13:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 23 k 2023-10-18T12:13:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 2.3 M 2023-10-18T12:13:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 242 k 2023-10-18T12:13:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 4.6 M 2023-10-18T12:13:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 18 M 2023-10-18T12:13:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 15 k 2023-10-18T12:13:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 24 k 2023-10-18T12:13:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T12:13:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T12:13:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-18T12:13:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-18T12:13:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-18T12:13:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 185 k 2023-10-18T12:13:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-18T12:13:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-18T12:13:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T12:13:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-18T12:13:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-18T12:13:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T12:13:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 518 k 2023-10-18T12:13:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-18T12:13:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.9 M 2023-10-18T12:13:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-18T12:13:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-18T12:13:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T12:13:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-18T12:13:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T12:13:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T12:13:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T12:13:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-18T12:13:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-18T12:13:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-18T12:13:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-18T12:13:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-18T12:13:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T12:13:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T12:13:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T12:13:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T12:13:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 45 k 2023-10-18T12:13:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 125 k 2023-10-18T12:13:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T12:13:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-18T12:13:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T12:13:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T12:13:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-18T12:13:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T12:13:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T12:13:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T12:13:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T12:13:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T12:13:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T12:13:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T12:13:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T12:13:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T12:13:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T12:13:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T12:13:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T12:13:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-18T12:13:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T12:13:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T12:13:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T12:13:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T12:13:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T12:13:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T12:13:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-18T12:13:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T12:13:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T12:13:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-18T12:13:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T12:13:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T12:13:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T12:13:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T12:13:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T12:13:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-18T12:13:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T12:13:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T12:13:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T12:13:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T12:13:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T12:13:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T12:13:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T12:13:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T12:13:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T12:13:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T12:13:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-18T12:13:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-18T12:13:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-18T12:13:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T12:13:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-18T12:13:12.177 INFO:teuthology.orchestra.run.smithi152.stdout:Enabling module streams: 2023-10-18T12:13:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql 10 2023-10-18T12:13:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:13:12.178 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-10-18T12:13:12.178 INFO:teuthology.orchestra.run.smithi152.stdout:=========================================================================================================================== 2023-10-18T12:13:12.178 INFO:teuthology.orchestra.run.smithi152.stdout:Install 128 Packages 2023-10-18T12:13:12.178 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 3 Packages 2023-10-18T12:13:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:13:12.179 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 209 M 2023-10-18T12:13:12.180 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-10-18T12:13:12.325 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-10-18T12:13:12.459 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-10-18T12:13:12.460 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-10-18T12:13:12.849 INFO:teuthology.orchestra.run.smithi152.stdout:(1/131): ceph-18.0.0-6786.gf7420d7f.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-10-18T12:13:13.284 INFO:teuthology.orchestra.run.smithi152.stdout:(2/131): ceph-fuse-18.0.0-6786.gf7420d7f.el8.x8 2.1 MB/s | 919 kB 00:00 2023-10-18T12:13:13.400 INFO:teuthology.orchestra.run.smithi152.stdout:(3/131): ceph-immutable-object-cache-18.0.0-678 1.4 MB/s | 167 kB 00:00 2023-10-18T12:13:13.542 INFO:teuthology.orchestra.run.smithi152.stdout:(4/131): ceph-base-18.0.0-6786.gf7420d7f.el8.x8 5.9 MB/s | 5.4 MB 00:00 2023-10-18T12:13:13.767 INFO:teuthology.orchestra.run.smithi152.stdout:(5/131): ceph-mgr-18.0.0-6786.gf7420d7f.el8.x86 7.2 MB/s | 1.6 MB 00:00 2023-10-18T12:13:14.001 INFO:teuthology.orchestra.run.smithi152.stdout:(6/131): ceph-mds-18.0.0-6786.gf7420d7f.el8.x86 3.9 MB/s | 2.3 MB 00:00 2023-10-18T12:13:14.031 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-10-18T12:13:14.036 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-10-18T12:13:14.276 INFO:teuthology.orchestra.run.smithi152.stdout:(7/131): ceph-mon-18.0.0-6786.gf7420d7f.el8.x86 9.1 MB/s | 4.6 MB 00:00 2023-10-18T12:13:15.401 INFO:teuthology.orchestra.run.smithi152.stdout:(8/131): ceph-common-18.0.0-6786.gf7420d7f.el8. 8.1 MB/s | 23 MB 00:02 2023-10-18T12:13:15.460 INFO:teuthology.orchestra.run.smithi152.stdout:(9/131): ceph-selinux-18.0.0-6786.gf7420d7f.el8 418 kB/s | 24 kB 00:00 2023-10-18T12:13:15.602 INFO:teuthology.orchestra.run.smithi152.stdout:(10/131): ceph-radosgw-18.0.0-6786.gf7420d7f.el 10 MB/s | 14 MB 00:01 2023-10-18T12:13:15.694 INFO:teuthology.orchestra.run.smithi152.stdout:(11/131): cephfs-mirror-18.0.0-6786.gf7420d7f.e 2.8 MB/s | 261 kB 00:00 2023-10-18T12:13:15.911 INFO:teuthology.orchestra.run.smithi152.stdout:(12/131): ceph-osd-18.0.0-6786.gf7420d7f.el8.x8 9.4 MB/s | 18 MB 00:01 2023-10-18T12:13:15.936 INFO:teuthology.orchestra.run.smithi152.stdout:(13/131): libcephfs-devel-18.0.0-6786.gf7420d7f 128 kB/s | 31 kB 00:00 2023-10-18T12:13:16.011 INFO:teuthology.orchestra.run.smithi152.stdout:(14/131): libcephsqlite-18.0.0-6786.gf7420d7f.e 2.4 MB/s | 185 kB 00:00 2023-10-18T12:13:16.045 INFO:teuthology.orchestra.run.smithi152.stdout:(15/131): libcephfs2-18.0.0-6786.gf7420d7f.el8. 6.1 MB/s | 840 kB 00:00 2023-10-18T12:13:16.070 INFO:teuthology.orchestra.run.smithi152.stdout:(16/131): librados-devel-18.0.0-6786.gf7420d7f. 2.3 MB/s | 139 kB 00:00 2023-10-18T12:13:16.137 INFO:teuthology.orchestra.run.smithi152.stdout:(17/131): libradosstriper1-18.0.0-6786.gf7420d7 5.5 MB/s | 518 kB 00:00 2023-10-18T12:13:16.221 INFO:teuthology.orchestra.run.smithi152.stdout:(18/131): python3-ceph-argparse-18.0.0-6786.gf7 536 kB/s | 45 kB 00:00 2023-10-18T12:13:16.313 INFO:teuthology.orchestra.run.smithi152.stdout:(19/131): python3-ceph-common-18.0.0-6786.gf742 1.3 MB/s | 125 kB 00:00 2023-10-18T12:13:16.405 INFO:teuthology.orchestra.run.smithi152.stdout:(20/131): python3-cephfs-18.0.0-6786.gf7420d7f. 2.1 MB/s | 196 kB 00:00 2023-10-18T12:13:16.514 INFO:teuthology.orchestra.run.smithi152.stdout:(21/131): python3-rados-18.0.0-6786.gf7420d7f.e 3.3 MB/s | 370 kB 00:00 2023-10-18T12:13:16.614 INFO:teuthology.orchestra.run.smithi152.stdout:(22/131): librgw2-18.0.0-6786.gf7420d7f.el8.x86 13 MB/s | 6.9 MB 00:00 2023-10-18T12:13:16.639 INFO:teuthology.orchestra.run.smithi152.stdout:(23/131): python3-rbd-18.0.0-6786.gf7420d7f.el8 2.8 MB/s | 364 kB 00:00 2023-10-18T12:13:16.681 INFO:teuthology.orchestra.run.smithi152.stdout:(24/131): python3-rgw-18.0.0-6786.gf7420d7f.el8 1.6 MB/s | 107 kB 00:00 2023-10-18T12:13:16.715 INFO:teuthology.orchestra.run.smithi152.stdout:(25/131): rbd-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.2 MB/s | 94 kB 00:00 2023-10-18T12:13:16.799 INFO:teuthology.orchestra.run.smithi152.stdout:(26/131): rbd-nbd-18.0.0-6786.gf7420d7f.el8.x86 2.1 MB/s | 178 kB 00:00 2023-10-18T12:13:16.891 INFO:teuthology.orchestra.run.smithi152.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6786.g 251 kB/s | 23 kB 00:00 2023-10-18T12:13:17.000 INFO:teuthology.orchestra.run.smithi152.stdout:(28/131): rbd-mirror-18.0.0-6786.gf7420d7f.el8. 11 MB/s | 3.6 MB 00:00 2023-10-18T12:13:17.025 INFO:teuthology.orchestra.run.smithi152.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 990 kB/s | 132 kB 00:00 2023-10-18T12:13:17.259 INFO:teuthology.orchestra.run.smithi152.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6786.gf7420 6.8 MB/s | 1.8 MB 00:00 2023-10-18T12:13:17.367 INFO:teuthology.orchestra.run.smithi152.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6786.gf7 2.2 MB/s | 242 kB 00:00 2023-10-18T12:13:17.459 INFO:teuthology.orchestra.run.smithi152.stdout:(32/131): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 517 kB/s | 47 kB 00:00 2023-10-18T12:13:17.643 INFO:teuthology.orchestra.run.smithi152.stdout:(33/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-10-18T12:13:17.660 INFO:teuthology.orchestra.run.smithi152.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6786.gf 75 kB/s | 15 kB 00:00 2023-10-18T12:13:17.682 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T12:13:17.683 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T12:13:17.698 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-18T12:13:17.699 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-18T12:13:17.700 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-18T12:13:17.702 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-18T12:13:17.704 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-18T12:13:17.707 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-18T12:13:17.708 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-18T12:13:17.762 INFO:teuthology.orchestra.run.smithi152.stdout:(35/131): ceph-volume-18.0.0-6786.gf7420d7f.el8 2.2 MB/s | 261 kB 00:00 2023-10-18T12:13:17.794 INFO:teuthology.orchestra.run.smithi152.stdout:(36/131): cephadm-18.0.0-6786.gf7420d7f.el8.noa 1.4 MB/s | 192 kB 00:00 2023-10-18T12:13:17.828 INFO:teuthology.orchestra.run.smithi152.stdout:(37/131): cephfs-top-18.0.0-6786.gf7420d7f.el8. 277 kB/s | 18 kB 00:00 2023-10-18T12:13:17.840 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-18T12:13:17.852 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-18T12:13:17.853 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-10-18T12:13:17.853 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-18T12:13:17.853 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-10-18T12:13:17.853 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-18T12:13:17.853 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-18T12:13:17.854 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.4 k 2023-10-18T12:13:17.854 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 5.4 M 2023-10-18T12:13:17.854 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 919 k 2023-10-18T12:13:17.854 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 167 k 2023-10-18T12:13:17.855 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 1.6 M 2023-10-18T12:13:17.855 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 132 k 2023-10-18T12:13:17.855 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 1.8 M 2023-10-18T12:13:17.855 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 7.4 M 2023-10-18T12:13:17.855 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 47 k 2023-10-18T12:13:17.856 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 14 M 2023-10-18T12:13:17.856 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 49 M 2023-10-18T12:13:17.856 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 261 k 2023-10-18T12:13:17.856 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 192 k 2023-10-18T12:13:17.856 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 261 k 2023-10-18T12:13:17.857 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 18 k 2023-10-18T12:13:17.857 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-18T12:13:17.857 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-18T12:13:17.857 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-18T12:13:17.857 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 31 k 2023-10-18T12:13:17.858 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 840 k 2023-10-18T12:13:17.858 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 139 k 2023-10-18T12:13:17.858 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-18T12:13:17.858 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-18T12:13:17.858 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-18T12:13:17.859 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-18T12:13:17.859 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-18T12:13:17.859 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-18T12:13:17.859 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 196 k 2023-10-18T12:13:17.859 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 370 k 2023-10-18T12:13:17.860 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 364 k 2023-10-18T12:13:17.860 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 107 k 2023-10-18T12:13:17.860 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 94 k 2023-10-18T12:13:17.860 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.6 M 2023-10-18T12:13:17.861 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 178 k 2023-10-18T12:13:17.861 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-18T12:13:17.861 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-10-18T12:13:17.861 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-18T12:13:17.861 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T12:13:17.861 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T12:13:17.862 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-10-18T12:13:17.862 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-18T12:13:17.862 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 23 M 2023-10-18T12:13:17.862 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 23 k 2023-10-18T12:13:17.863 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 2.3 M 2023-10-18T12:13:17.863 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 242 k 2023-10-18T12:13:17.863 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 4.6 M 2023-10-18T12:13:17.863 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 18 M 2023-10-18T12:13:17.863 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 15 k 2023-10-18T12:13:17.864 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 24 k 2023-10-18T12:13:17.864 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T12:13:17.864 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T12:13:17.864 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-18T12:13:17.864 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-18T12:13:17.865 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-18T12:13:17.865 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 185 k 2023-10-18T12:13:17.865 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-18T12:13:17.865 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-18T12:13:17.865 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T12:13:17.866 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-18T12:13:17.866 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-18T12:13:17.866 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T12:13:17.866 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 518 k 2023-10-18T12:13:17.866 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-18T12:13:17.867 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.9 M 2023-10-18T12:13:17.867 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-18T12:13:17.867 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-18T12:13:17.867 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T12:13:17.867 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-18T12:13:17.868 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T12:13:17.868 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T12:13:17.868 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T12:13:17.868 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-18T12:13:17.868 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-18T12:13:17.868 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-18T12:13:17.869 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-18T12:13:17.869 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-18T12:13:17.869 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T12:13:17.869 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T12:13:17.869 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T12:13:17.870 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T12:13:17.870 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 45 k 2023-10-18T12:13:17.870 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 125 k 2023-10-18T12:13:17.870 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T12:13:17.870 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-18T12:13:17.871 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T12:13:17.871 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T12:13:17.871 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-18T12:13:17.871 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T12:13:17.872 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T12:13:17.872 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T12:13:17.872 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T12:13:17.872 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T12:13:17.876 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T12:13:17.877 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T12:13:17.877 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T12:13:17.877 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T12:13:17.877 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T12:13:17.877 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T12:13:17.878 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T12:13:17.878 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-18T12:13:17.878 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T12:13:17.878 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T12:13:17.878 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T12:13:17.879 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T12:13:17.879 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T12:13:17.879 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T12:13:17.879 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-18T12:13:17.879 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T12:13:17.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T12:13:17.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-18T12:13:17.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T12:13:17.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T12:13:17.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T12:13:17.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T12:13:17.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T12:13:17.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-18T12:13:17.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T12:13:17.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T12:13:17.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T12:13:17.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T12:13:17.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T12:13:17.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T12:13:17.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T12:13:17.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T12:13:17.883 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T12:13:17.883 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T12:13:17.883 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-18T12:13:17.883 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-18T12:13:17.883 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-18T12:13:17.884 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T12:13:17.884 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-18T12:13:17.884 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-10-18T12:13:17.884 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2023-10-18T12:13:17.884 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:13:17.885 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-18T12:13:17.885 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-18T12:13:17.885 INFO:teuthology.orchestra.run.smithi186.stdout:Install 128 Packages 2023-10-18T12:13:17.885 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 3 Packages 2023-10-18T12:13:17.885 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:13:17.886 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 209 M 2023-10-18T12:13:17.886 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-10-18T12:13:18.151 INFO:teuthology.orchestra.run.smithi186.stdout:(1/131): ceph-18.0.0-6786.gf7420d7f.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-10-18T12:13:18.178 INFO:teuthology.orchestra.run.smithi152.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2023-10-18T12:13:18.320 INFO:teuthology.orchestra.run.smithi152.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 582 kB/s | 306 kB 00:00 2023-10-18T12:13:18.396 INFO:teuthology.orchestra.run.smithi152.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 345 kB/s | 75 kB 00:00 2023-10-18T12:13:18.476 INFO:teuthology.orchestra.run.smithi186.stdout:(2/131): ceph-fuse-18.0.0-6786.gf7420d7f.el8.x8 2.8 MB/s | 919 kB 00:00 2023-10-18T12:13:18.513 INFO:teuthology.orchestra.run.smithi152.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2023-10-18T12:13:18.538 INFO:teuthology.orchestra.run.smithi152.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-10-18T12:13:18.559 INFO:teuthology.orchestra.run.smithi186.stdout:(3/131): ceph-immutable-object-cache-18.0.0-678 2.0 MB/s | 167 kB 00:00 2023-10-18T12:13:18.647 INFO:teuthology.orchestra.run.smithi152.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-10-18T12:13:18.705 INFO:teuthology.orchestra.run.smithi152.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 964 kB/s | 185 kB 00:00 2023-10-18T12:13:18.726 INFO:teuthology.orchestra.run.smithi186.stdout:(4/131): ceph-base-18.0.0-6786.gf7420d7f.el8.x8 7.8 MB/s | 5.4 MB 00:00 2023-10-18T12:13:18.755 INFO:teuthology.orchestra.run.smithi152.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-10-18T12:13:18.814 INFO:teuthology.orchestra.run.smithi152.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.5 MB/s | 173 kB 00:00 2023-10-18T12:13:18.865 INFO:teuthology.orchestra.run.smithi152.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-10-18T12:13:18.923 INFO:teuthology.orchestra.run.smithi152.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 745 kB/s | 81 kB 00:00 2023-10-18T12:13:18.935 INFO:teuthology.orchestra.run.smithi186.stdout:(5/131): ceph-mgr-18.0.0-6786.gf7420d7f.el8.x86 7.8 MB/s | 1.6 MB 00:00 2023-10-18T12:13:18.974 INFO:teuthology.orchestra.run.smithi152.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-10-18T12:13:19.057 INFO:teuthology.orchestra.run.smithi152.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 131 kB/s | 18 kB 00:00 2023-10-18T12:13:19.117 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-10-18T12:13:19.152 INFO:teuthology.orchestra.run.smithi186.stdout:(6/131): ceph-mds-18.0.0-6786.gf7420d7f.el8.x86 3.9 MB/s | 2.3 MB 00:00 2023-10-18T12:13:19.166 INFO:teuthology.orchestra.run.smithi152.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-10-18T12:13:19.200 INFO:teuthology.orchestra.run.smithi152.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.2 MB/s | 1.2 MB 00:00 2023-10-18T12:13:19.275 INFO:teuthology.orchestra.run.smithi152.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2023-10-18T12:13:19.286 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-18T12:13:19.301 INFO:teuthology.orchestra.run.smithi152.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 600 kB/s | 60 kB 00:00 2023-10-18T12:13:19.377 INFO:teuthology.orchestra.run.smithi186.stdout:(7/131): ceph-mon-18.0.0-6786.gf7420d7f.el8.x86 10 MB/s | 4.6 MB 00:00 2023-10-18T12:13:19.384 INFO:teuthology.orchestra.run.smithi152.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-10-18T12:13:19.401 INFO:teuthology.orchestra.run.smithi152.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-10-18T12:13:19.485 INFO:teuthology.orchestra.run.smithi152.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-10-18T12:13:19.487 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-18T12:13:19.510 INFO:teuthology.orchestra.run.smithi152.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-10-18T12:13:19.585 INFO:teuthology.orchestra.run.smithi152.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 641 kB/s | 64 kB 00:00 2023-10-18T12:13:19.619 INFO:teuthology.orchestra.run.smithi152.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-10-18T12:13:19.645 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T12:13:19.694 INFO:teuthology.orchestra.run.smithi152.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-10-18T12:13:19.728 INFO:teuthology.orchestra.run.smithi152.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-10-18T12:13:19.758 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T12:13:19.803 INFO:teuthology.orchestra.run.smithi152.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-10-18T12:13:19.837 INFO:teuthology.orchestra.run.smithi152.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-10-18T12:13:19.912 INFO:teuthology.orchestra.run.smithi152.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-10-18T12:13:19.946 INFO:teuthology.orchestra.run.smithi152.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-10-18T12:13:19.987 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T12:13:20.013 INFO:teuthology.orchestra.run.smithi152.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 851 kB/s | 85 kB 00:00 2023-10-18T12:13:20.055 INFO:teuthology.orchestra.run.smithi152.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2023-10-18T12:13:20.081 INFO:teuthology.orchestra.run.smithi152.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-18T12:13:20.142 INFO:teuthology.orchestra.run.smithi152.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 805 kB/s | 47 kB 00:00 2023-10-18T12:13:20.181 INFO:teuthology.orchestra.run.smithi152.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-10-18T12:13:20.223 INFO:teuthology.orchestra.run.smithi152.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-18T12:13:20.265 INFO:teuthology.orchestra.run.smithi152.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-10-18T12:13:20.333 INFO:teuthology.orchestra.run.smithi152.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 5.4 MB/s | 1.7 MB 00:00 2023-10-18T12:13:20.440 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T12:13:20.711 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T12:13:20.833 INFO:teuthology.orchestra.run.smithi152.stdout:(75/131): ceph-test-18.0.0-6786.gf7420d7f.el8.x 9.1 MB/s | 49 MB 00:05 2023-10-18T12:13:20.866 INFO:teuthology.orchestra.run.smithi152.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 80 kB/s | 48 kB 00:00 2023-10-18T12:13:20.892 INFO:teuthology.orchestra.run.smithi152.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 156 kB/s | 87 kB 00:00 2023-10-18T12:13:20.918 INFO:teuthology.orchestra.run.smithi152.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 547 kB/s | 43 kB 00:00 2023-10-18T12:13:20.944 INFO:teuthology.orchestra.run.smithi186.stdout:(8/131): ceph-radosgw-18.0.0-6786.gf7420d7f.el8 8.9 MB/s | 14 MB 00:01 2023-10-18T12:13:20.967 INFO:teuthology.orchestra.run.smithi152.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2023-10-18T12:13:20.993 INFO:teuthology.orchestra.run.smithi152.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.2 MB/s | 229 kB 00:00 2023-10-18T12:13:21.002 INFO:teuthology.orchestra.run.smithi186.stdout:(9/131): ceph-selinux-18.0.0-6786.gf7420d7f.el8 419 kB/s | 24 kB 00:00 2023-10-18T12:13:21.018 INFO:teuthology.orchestra.run.smithi152.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 371 kB/s | 19 kB 00:00 2023-10-18T12:13:21.035 INFO:teuthology.orchestra.run.smithi152.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-10-18T12:13:21.069 INFO:teuthology.orchestra.run.smithi152.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-10-18T12:13:21.085 INFO:teuthology.orchestra.run.smithi152.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 357 kB/s | 18 kB 00:00 2023-10-18T12:13:21.119 INFO:teuthology.orchestra.run.smithi152.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.4 MB/s | 559 kB 00:00 2023-10-18T12:13:21.145 INFO:teuthology.orchestra.run.smithi152.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-18T12:13:21.178 INFO:teuthology.orchestra.run.smithi152.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-10-18T12:13:21.185 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T12:13:21.195 INFO:teuthology.orchestra.run.smithi152.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-10-18T12:13:21.221 INFO:teuthology.orchestra.run.smithi152.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-10-18T12:13:21.246 INFO:teuthology.orchestra.run.smithi152.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-10-18T12:13:21.271 INFO:teuthology.orchestra.run.smithi152.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-18T12:13:21.297 INFO:teuthology.orchestra.run.smithi152.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-18T12:13:21.322 INFO:teuthology.orchestra.run.smithi152.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-10-18T12:13:21.353 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:13:21.356 INFO:teuthology.orchestra.run.smithi152.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-10-18T12:13:21.381 INFO:teuthology.orchestra.run.smithi152.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-10-18T12:13:21.395 INFO:teuthology.orchestra.run.smithi186.stdout:(10/131): ceph-common-18.0.0-6786.gf7420d7f.el8 6.7 MB/s | 23 MB 00:03 2023-10-18T12:13:21.407 INFO:teuthology.orchestra.run.smithi152.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-18T12:13:21.449 INFO:teuthology.orchestra.run.smithi152.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-18T12:13:21.499 INFO:teuthology.orchestra.run.smithi152.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2023-10-18T12:13:21.533 INFO:teuthology.orchestra.run.smithi152.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 648 kB/s | 54 kB 00:00 2023-10-18T12:13:21.547 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:13:21.570 INFO:teuthology.orchestra.run.smithi186.stdout:(11/131): ceph-osd-18.0.0-6786.gf7420d7f.el8.x8 7.4 MB/s | 18 MB 00:02 2023-10-18T12:13:21.604 INFO:teuthology.orchestra.run.smithi152.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.4 MB/s | 259 kB 00:00 2023-10-18T12:13:21.642 INFO:teuthology.orchestra.run.smithi152.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-10-18T12:13:21.667 INFO:teuthology.orchestra.run.smithi152.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 624 kB/s | 39 kB 00:00 2023-10-18T12:13:21.703 INFO:teuthology.orchestra.run.smithi186.stdout:(12/131): cephfs-mirror-18.0.0-6786.gf7420d7f.e 847 kB/s | 261 kB 00:00 2023-10-18T12:13:21.717 INFO:teuthology.orchestra.run.smithi152.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-10-18T12:13:21.751 INFO:teuthology.orchestra.run.smithi152.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-10-18T12:13:21.753 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:13:21.754 INFO:teuthology.orchestra.run.smithi186.stdout:(13/131): libcephfs-devel-18.0.0-6786.gf7420d7f 169 kB/s | 31 kB 00:00 2023-10-18T12:13:21.777 INFO:teuthology.orchestra.run.smithi152.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-18T12:13:21.802 INFO:teuthology.orchestra.run.smithi152.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-18T12:13:21.821 INFO:teuthology.orchestra.run.smithi186.stdout:(14/131): libcephfs2-18.0.0-6786.gf7420d7f.el8. 7.0 MB/s | 840 kB 00:00 2023-10-18T12:13:21.843 INFO:teuthology.orchestra.run.smithi152.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-18T12:13:21.846 INFO:teuthology.orchestra.run.smithi186.stdout:(15/131): libcephsqlite-18.0.0-6786.gf7420d7f.e 2.0 MB/s | 185 kB 00:00 2023-10-18T12:13:21.877 INFO:teuthology.orchestra.run.smithi152.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 800 kB/s | 27 kB 00:00 2023-10-18T12:13:21.888 INFO:teuthology.orchestra.run.smithi186.stdout:(16/131): librados-devel-18.0.0-6786.gf7420d7f. 2.0 MB/s | 139 kB 00:00 2023-10-18T12:13:21.895 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:13:21.919 INFO:teuthology.orchestra.run.smithi152.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-18T12:13:21.961 INFO:teuthology.orchestra.run.smithi152.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-10-18T12:13:21.997 INFO:teuthology.orchestra.run.smithi186.stdout:(17/131): libradosstriper1-18.0.0-6786.gf7420d7 3.4 MB/s | 518 kB 00:00 2023-10-18T12:13:22.061 INFO:teuthology.orchestra.run.smithi152.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-10-18T12:13:22.075 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T12:13:22.080 INFO:teuthology.orchestra.run.smithi186.stdout:(18/131): python3-ceph-argparse-18.0.0-6786.gf7 537 kB/s | 45 kB 00:00 2023-10-18T12:13:22.112 INFO:teuthology.orchestra.run.smithi152.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-10-18T12:13:22.154 INFO:teuthology.orchestra.run.smithi152.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-10-18T12:13:22.182 INFO:teuthology.orchestra.run.smithi186.stdout:(19/131): python3-ceph-common-18.0.0-6786.gf742 1.2 MB/s | 125 kB 00:00 2023-10-18T12:13:22.234 INFO:teuthology.orchestra.run.smithi152.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-10-18T12:13:22.263 INFO:teuthology.orchestra.run.smithi152.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 589 kB/s | 64 kB 00:00 2023-10-18T12:13:22.281 INFO:teuthology.orchestra.run.smithi186.stdout:(20/131): python3-cephfs-18.0.0-6786.gf7420d7f. 1.9 MB/s | 196 kB 00:00 2023-10-18T12:13:22.288 INFO:teuthology.orchestra.run.smithi152.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.3 MB/s | 126 kB 00:00 2023-10-18T12:13:22.291 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-18T12:13:22.313 INFO:teuthology.orchestra.run.smithi152.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-10-18T12:13:22.338 INFO:teuthology.orchestra.run.smithi152.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-18T12:13:22.373 INFO:teuthology.orchestra.run.smithi186.stdout:(21/131): python3-rados-18.0.0-6786.gf7420d7f.e 4.0 MB/s | 370 kB 00:00 2023-10-18T12:13:22.381 INFO:teuthology.orchestra.run.smithi152.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-18T12:13:22.423 INFO:teuthology.orchestra.run.smithi152.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-18T12:13:22.551 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-18T12:13:22.573 INFO:teuthology.orchestra.run.smithi152.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-10-18T12:13:22.615 INFO:teuthology.orchestra.run.smithi152.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-18T12:13:22.657 INFO:teuthology.orchestra.run.smithi152.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.2 MB/s | 354 kB 00:00 2023-10-18T12:13:22.824 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-18T12:13:22.924 INFO:teuthology.orchestra.run.smithi152.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 8.3 MB/s | 4.9 MB 00:00 2023-10-18T12:13:22.948 INFO:teuthology.orchestra.run.smithi186.stdout:(22/131): librgw2-18.0.0-6786.gf7420d7f.el8.x86 6.5 MB/s | 6.9 MB 00:01 2023-10-18T12:13:22.973 INFO:teuthology.orchestra.run.smithi186.stdout:(23/131): python3-rbd-18.0.0-6786.gf7420d7f.el8 607 kB/s | 364 kB 00:00 2023-10-18T12:13:23.015 INFO:teuthology.orchestra.run.smithi186.stdout:(24/131): python3-rgw-18.0.0-6786.gf7420d7f.el8 1.6 MB/s | 107 kB 00:00 2023-10-18T12:13:23.032 INFO:teuthology.orchestra.run.smithi186.stdout:(25/131): rbd-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.6 MB/s | 94 kB 00:00 2023-10-18T12:13:23.083 INFO:teuthology.orchestra.run.smithi152.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 7.2 MB/s | 4.7 MB 00:00 2023-10-18T12:13:23.116 INFO:teuthology.orchestra.run.smithi152.stdout:(126/131): postgresql-contrib-10.21-2.module+el 4.2 MB/s | 808 kB 00:00 2023-10-18T12:13:23.124 INFO:teuthology.orchestra.run.smithi186.stdout:(26/131): rbd-nbd-18.0.0-6786.gf7420d7f.el8.x86 1.9 MB/s | 178 kB 00:00 2023-10-18T12:13:23.184 INFO:teuthology.orchestra.run.smithi152.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-10-18T12:13:23.216 INFO:teuthology.orchestra.run.smithi186.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6786.g 252 kB/s | 23 kB 00:00 2023-10-18T12:13:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-18T12:13:23.275 INFO:teuthology.orchestra.run.smithi152.stdout:(128/131): postgresql-server-10.21-2.module+el8 8.2 MB/s | 5.1 MB 00:00 2023-10-18T12:13:23.308 INFO:teuthology.orchestra.run.smithi186.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.4 MB/s | 132 kB 00:00 2023-10-18T12:13:23.334 INFO:teuthology.orchestra.run.smithi152.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-18T12:13:23.600 INFO:teuthology.orchestra.run.smithi186.stdout:(29/131): rbd-mirror-18.0.0-6786.gf7420d7f.el8. 6.1 MB/s | 3.6 MB 00:00 2023-10-18T12:13:23.701 INFO:teuthology.orchestra.run.smithi186.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6786.gf7420 4.5 MB/s | 1.8 MB 00:00 2023-10-18T12:13:23.801 INFO:teuthology.orchestra.run.smithi186.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6786.gf7 2.4 MB/s | 242 kB 00:00 2023-10-18T12:13:23.884 INFO:teuthology.orchestra.run.smithi186.stdout:(32/131): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 569 kB/s | 47 kB 00:00 2023-10-18T12:13:23.985 INFO:teuthology.orchestra.run.smithi186.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6786.gf 150 kB/s | 15 kB 00:00 2023-10-18T12:13:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-18T12:13:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T12:13:24.309 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T12:13:24.442 INFO:teuthology.orchestra.run.smithi152.stdout:(130/131): librbd1-18.0.0-6786.gf7420d7f.el8.x8 2.9 MB/s | 3.7 MB 00:01 2023-10-18T12:13:24.477 INFO:teuthology.orchestra.run.smithi186.stdout:(34/131): ceph-volume-18.0.0-6786.gf7420d7f.el8 531 kB/s | 261 kB 00:00 2023-10-18T12:13:24.485 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T12:13:24.560 INFO:teuthology.orchestra.run.smithi186.stdout:(35/131): cephadm-18.0.0-6786.gf7420d7f.el8.noa 2.3 MB/s | 192 kB 00:00 2023-10-18T12:13:24.661 INFO:teuthology.orchestra.run.smithi186.stdout:(36/131): cephfs-top-18.0.0-6786.gf7420d7f.el8. 181 kB/s | 18 kB 00:00 2023-10-18T12:13:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T12:13:24.936 INFO:teuthology.orchestra.run.smithi186.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-10-18T12:13:24.942 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-18T12:13:25.003 INFO:teuthology.orchestra.run.smithi186.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 885 kB/s | 59 kB 00:00 2023-10-18T12:13:25.086 INFO:teuthology.orchestra.run.smithi186.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2023-10-18T12:13:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-18T12:13:25.220 INFO:teuthology.orchestra.run.smithi186.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-10-18T12:13:25.278 INFO:teuthology.orchestra.run.smithi186.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 751 kB/s | 44 kB 00:00 2023-10-18T12:13:25.329 INFO:teuthology.orchestra.run.smithi186.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-10-18T12:13:25.346 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-18T12:13:25.371 INFO:teuthology.orchestra.run.smithi186.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 794 kB/s | 33 kB 00:00 2023-10-18T12:13:25.405 INFO:teuthology.orchestra.run.smithi186.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 491 kB/s | 16 kB 00:00 2023-10-18T12:13:25.471 INFO:teuthology.orchestra.run.smithi186.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-10-18T12:13:25.485 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T12:13:25.563 INFO:teuthology.orchestra.run.smithi186.stdout:(46/131): ceph-mgr-diskprediction-local-18.0.0- 3.8 MB/s | 7.4 MB 00:01 2023-10-18T12:13:25.589 INFO:teuthology.orchestra.run.smithi186.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-10-18T12:13:25.656 INFO:teuthology.orchestra.run.smithi186.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-10-18T12:13:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T12:13:25.723 INFO:teuthology.orchestra.run.smithi186.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-10-18T12:13:25.827 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T12:13:25.906 INFO:teuthology.orchestra.run.smithi186.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 237 kB/s | 81 kB 00:00 2023-10-18T12:13:25.940 INFO:teuthology.orchestra.run.smithi186.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2023-10-18T12:13:25.998 INFO:teuthology.orchestra.run.smithi186.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 530 kB/s | 49 kB 00:00 2023-10-18T12:13:26.015 INFO:teuthology.orchestra.run.smithi186.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 780 kB/s | 59 kB 00:00 2023-10-18T12:13:26.099 INFO:teuthology.orchestra.run.smithi186.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-10-18T12:13:26.124 INFO:teuthology.orchestra.run.smithi186.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 480 kB/s | 60 kB 00:00 2023-10-18T12:13:26.166 INFO:teuthology.orchestra.run.smithi186.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 232 kB/s | 16 kB 00:00 2023-10-18T12:13:26.208 INFO:teuthology.orchestra.run.smithi186.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-10-18T12:13:26.242 INFO:teuthology.orchestra.run.smithi186.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-10-18T12:13:26.309 INFO:teuthology.orchestra.run.smithi186.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 640 kB/s | 64 kB 00:00 2023-10-18T12:13:26.334 INFO:teuthology.orchestra.run.smithi186.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-10-18T12:13:26.376 INFO:teuthology.orchestra.run.smithi186.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-10-18T12:13:26.401 INFO:teuthology.orchestra.run.smithi186.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 437 kB/s | 29 kB 00:00 2023-10-18T12:13:26.435 INFO:teuthology.orchestra.run.smithi186.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 471 kB/s | 27 kB 00:00 2023-10-18T12:13:26.469 INFO:teuthology.orchestra.run.smithi186.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-10-18T12:13:26.499 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T12:13:26.527 INFO:teuthology.orchestra.run.smithi186.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-18T12:13:26.594 INFO:teuthology.orchestra.run.smithi186.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2023-10-18T12:13:26.620 INFO:teuthology.orchestra.run.smithi186.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-10-18T12:13:26.653 INFO:teuthology.orchestra.run.smithi186.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 393 kB/s | 23 kB 00:00 2023-10-18T12:13:26.678 INFO:teuthology.orchestra.run.smithi186.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-18T12:13:26.694 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T12:13:26.871 INFO:teuthology.orchestra.run.smithi186.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 244 kB/s | 47 kB 00:00 2023-10-18T12:13:26.913 INFO:teuthology.orchestra.run.smithi186.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2023-10-18T12:13:26.938 INFO:teuthology.orchestra.run.smithi186.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-10-18T12:13:26.980 INFO:teuthology.orchestra.run.smithi186.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 639 kB/s | 43 kB 00:00 2023-10-18T12:13:27.005 INFO:teuthology.orchestra.run.smithi186.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.5 MB/s | 220 kB 00:00 2023-10-18T12:13:27.030 INFO:teuthology.orchestra.run.smithi186.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 957 kB/s | 48 kB 00:00 2023-10-18T12:13:27.047 INFO:teuthology.orchestra.run.smithi186.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-10-18T12:13:27.073 INFO:teuthology.orchestra.run.smithi186.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-10-18T12:13:27.098 INFO:teuthology.orchestra.run.smithi186.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-10-18T12:13:27.123 INFO:teuthology.orchestra.run.smithi186.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-10-18T12:13:27.157 INFO:teuthology.orchestra.run.smithi186.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-10-18T12:13:27.174 INFO:teuthology.orchestra.run.smithi186.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 372 kB/s | 19 kB 00:00 2023-10-18T12:13:27.199 INFO:teuthology.orchestra.run.smithi186.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-10-18T12:13:27.233 INFO:teuthology.orchestra.run.smithi186.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-10-18T12:13:27.234 INFO:teuthology.orchestra.run.smithi152.stdout:(131/131): librados2-18.0.0-6786.gf7420d7f.el8. 931 kB/s | 3.7 MB 00:04 2023-10-18T12:13:27.237 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:13:27.237 INFO:teuthology.orchestra.run.smithi152.stdout:Total 14 MB/s | 209 MB 00:15 2023-10-18T12:13:27.250 INFO:teuthology.orchestra.run.smithi186.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 356 kB/s | 18 kB 00:00 2023-10-18T12:13:27.284 INFO:teuthology.orchestra.run.smithi186.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-10-18T12:13:27.309 INFO:teuthology.orchestra.run.smithi186.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-18T12:13:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T12:13:27.351 INFO:teuthology.orchestra.run.smithi186.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-10-18T12:13:27.401 INFO:teuthology.orchestra.run.smithi186.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 20 MB/s | 2.3 MB 00:00 2023-10-18T12:13:27.427 INFO:teuthology.orchestra.run.smithi186.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-10-18T12:13:27.452 INFO:teuthology.orchestra.run.smithi186.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-10-18T12:13:27.477 INFO:teuthology.orchestra.run.smithi186.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-18T12:13:27.502 INFO:teuthology.orchestra.run.smithi186.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-18T12:13:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T12:13:27.546 INFO:teuthology.orchestra.run.smithi186.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 564 kB/s | 39 kB 00:00 2023-10-18T12:13:27.595 INFO:teuthology.orchestra.run.smithi186.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.9 MB/s | 644 kB 00:00 2023-10-18T12:13:27.620 INFO:teuthology.orchestra.run.smithi186.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.3 MB/s | 171 kB 00:00 2023-10-18T12:13:27.646 INFO:teuthology.orchestra.run.smithi186.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-18T12:13:27.687 INFO:teuthology.orchestra.run.smithi186.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-18T12:13:27.721 INFO:teuthology.orchestra.run.smithi186.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-18T12:13:27.763 INFO:teuthology.orchestra.run.smithi186.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-18T12:13:27.805 INFO:teuthology.orchestra.run.smithi186.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-10-18T12:13:27.846 INFO:teuthology.orchestra.run.smithi186.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 932 kB/s | 39 kB 00:00 2023-10-18T12:13:27.897 INFO:teuthology.orchestra.run.smithi186.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-10-18T12:13:27.955 INFO:teuthology.orchestra.run.smithi186.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 9.0 MB/s | 538 kB 00:00 2023-10-18T12:13:28.082 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-10-18T12:13:28.114 INFO:teuthology.orchestra.run.smithi186.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-10-18T12:13:28.139 INFO:teuthology.orchestra.run.smithi186.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 855 kB/s | 157 kB 00:00 2023-10-18T12:13:28.165 INFO:teuthology.orchestra.run.smithi186.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-18T12:13:28.198 INFO:teuthology.orchestra.run.smithi186.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-18T12:13:28.203 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-10-18T12:13:28.203 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-10-18T12:13:28.232 INFO:teuthology.orchestra.run.smithi186.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 796 kB/s | 27 kB 00:00 2023-10-18T12:13:28.274 INFO:teuthology.orchestra.run.smithi186.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-18T12:13:28.328 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T12:13:28.357 INFO:teuthology.orchestra.run.smithi186.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-10-18T12:13:28.424 INFO:teuthology.orchestra.run.smithi186.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 292 kB/s | 44 kB 00:00 2023-10-18T12:13:28.483 INFO:teuthology.orchestra.run.smithi186.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.6 MB/s | 457 kB 00:00 2023-10-18T12:13:28.550 INFO:teuthology.orchestra.run.smithi186.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-10-18T12:13:28.576 INFO:teuthology.orchestra.run.smithi186.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-10-18T12:13:28.601 INFO:teuthology.orchestra.run.smithi186.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-18T12:13:28.618 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-18T12:13:28.626 INFO:teuthology.orchestra.run.smithi186.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-18T12:13:28.651 INFO:teuthology.orchestra.run.smithi186.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-10-18T12:13:28.677 INFO:teuthology.orchestra.run.smithi186.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-18T12:13:28.710 INFO:teuthology.orchestra.run.smithi186.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-10-18T12:13:28.752 INFO:teuthology.orchestra.run.smithi186.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-18T12:13:28.836 INFO:teuthology.orchestra.run.smithi186.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 31 MB/s | 4.9 MB 00:00 2023-10-18T12:13:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-18T12:13:28.877 INFO:teuthology.orchestra.run.smithi186.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-18T12:13:28.920 INFO:teuthology.orchestra.run.smithi186.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-18T12:13:28.978 INFO:teuthology.orchestra.run.smithi186.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-10-18T12:13:29.029 INFO:teuthology.orchestra.run.smithi186.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-10-18T12:13:29.112 INFO:teuthology.orchestra.run.smithi186.stdout:(126/131): postgresql-server-10.21-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-10-18T12:13:29.146 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-18T12:13:29.154 INFO:teuthology.orchestra.run.smithi186.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-10-18T12:13:29.330 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-10-18T12:13:29.334 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-10-18T12:13:29.367 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 26/134 2023-10-18T12:13:29.506 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 27/134 2023-10-18T12:13:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 28/134 2023-10-18T12:13:29.926 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T12:13:30.138 INFO:teuthology.orchestra.run.smithi186.stdout:(128/131): librados2-18.0.0-6786.gf7420d7f.el8. 3.7 MB/s | 3.7 MB 00:01 2023-10-18T12:13:30.180 INFO:teuthology.orchestra.run.smithi186.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2023-10-18T12:13:30.388 INFO:teuthology.orchestra.run.smithi186.stdout:(130/131): librbd1-18.0.0-6786.gf7420d7f.el8.x8 3.0 MB/s | 3.7 MB 00:01 2023-10-18T12:13:31.230 INFO:teuthology.orchestra.run.smithi186.stdout:(131/131): ceph-test-18.0.0-6786.gf7420d7f.el8. 4.8 MB/s | 49 MB 00:10 2023-10-18T12:13:31.233 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:13:31.234 INFO:teuthology.orchestra.run.smithi186.stdout:Total 16 MB/s | 209 MB 00:13 2023-10-18T12:13:31.972 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T12:13:32.050 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-10-18T12:13:32.171 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-10-18T12:13:32.172 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-10-18T12:13:33.125 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-18T12:13:33.304 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-10-18T12:13:33.308 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-10-18T12:13:33.461 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-18T12:13:33.706 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-18T12:13:33.977 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-18T12:13:34.186 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-18T12:13:34.197 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-10-18T12:13:34.362 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-18T12:13:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-18T12:13:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 36/134 2023-10-18T12:13:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-18T12:13:34.715 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T12:13:34.766 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 37/134 2023-10-18T12:13:34.828 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T12:13:34.925 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-18T12:13:35.056 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T12:13:35.113 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-18T12:13:35.316 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T12:13:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T12:13:35.489 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T12:13:35.653 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-18T12:13:35.689 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T12:13:35.897 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-18T12:13:36.066 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T12:13:36.108 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T12:13:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T12:13:36.281 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:13:36.398 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:13:36.435 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-18T12:13:36.572 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:13:36.663 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-18T12:13:36.664 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:13:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-18T12:13:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T12:13:36.962 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-18T12:13:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-18T12:13:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-18T12:13:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-18T12:13:37.330 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-18T12:13:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-18T12:13:37.572 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-18T12:13:37.797 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-18T12:13:37.836 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-18T12:13:38.038 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-18T12:13:38.139 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-10-18T12:13:38.164 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T12:13:38.260 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-18T12:13:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T12:13:38.321 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-18T12:13:38.422 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T12:13:38.546 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-18T12:13:38.573 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T12:13:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-18T12:13:38.715 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T12:13:38.768 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-18T12:13:38.803 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T12:13:38.823 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-18T12:13:38.978 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-18T12:13:38.998 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T12:13:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-18T12:13:39.292 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-18T12:13:39.410 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T12:13:39.446 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-18T12:13:39.615 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T12:13:39.622 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T12:13:39.681 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-18T12:13:39.757 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T12:13:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-18T12:13:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T12:13:40.028 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T12:13:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-18T12:13:40.192 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:13:40.313 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:13:40.384 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-18T12:13:40.482 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:13:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T12:13:40.562 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:13:40.590 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-18T12:13:40.680 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T12:13:40.711 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T12:13:40.788 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-18T12:13:40.906 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-18T12:13:41.112 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-18T12:13:41.136 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-18T12:13:41.280 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T12:13:41.350 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-18T12:13:41.420 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-18T12:13:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T12:13:41.713 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-18T12:13:41.756 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-18T12:13:42.001 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-18T12:13:42.038 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-18T12:13:42.164 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T12:13:42.165 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T12:13:42.183 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-18T12:13:42.269 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T12:13:42.408 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-18T12:13:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-18T12:13:42.413 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T12:13:42.544 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T12:13:42.603 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-18T12:13:42.672 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-18T12:13:42.850 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-18T12:13:42.906 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-18T12:13:42.995 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 26/134 2023-10-18T12:13:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-18T12:13:43.063 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-18T12:13:43.108 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 27/134 2023-10-18T12:13:43.232 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-18T12:13:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-18T12:13:43.309 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 28/134 2023-10-18T12:13:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-18T12:13:43.430 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-18T12:13:43.538 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T12:13:43.564 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T12:13:43.679 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-18T12:13:43.709 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T12:13:43.853 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-18T12:13:43.872 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T12:13:44.051 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-18T12:13:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-18T12:13:44.494 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T12:13:44.522 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-18T12:13:44.655 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T12:13:44.771 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-18T12:13:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-18T12:13:45.256 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T12:13:45.314 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-18T12:13:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T12:13:45.430 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T12:13:45.540 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-18T12:13:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-18T12:13:45.977 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-18T12:13:46.146 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T12:13:46.340 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-18T12:13:46.362 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-18T12:13:46.562 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-18T12:13:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-18T12:13:46.851 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-18T12:13:46.906 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-18T12:13:47.012 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 26/134 2023-10-18T12:13:47.140 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-18T12:13:47.141 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 27/134 2023-10-18T12:13:47.299 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-18T12:13:47.343 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 28/134 2023-10-18T12:13:47.391 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-18T12:13:47.592 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-18T12:13:47.597 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T12:13:47.606 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-18T12:13:47.801 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 36/134 2023-10-18T12:13:47.877 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-18T12:13:47.977 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 37/134 2023-10-18T12:13:48.128 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-18T12:13:48.216 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-18T12:13:48.272 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-18T12:13:48.454 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T12:13:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-18T12:13:48.553 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T12:13:48.757 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-18T12:13:48.758 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-18T12:13:48.949 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-18T12:13:49.096 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T12:13:49.194 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T12:13:49.377 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T12:13:49.391 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-18T12:13:49.549 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-18T12:13:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-18T12:13:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-18T12:13:49.913 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-18T12:13:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T12:13:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T12:13:50.081 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-18T12:13:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-18T12:13:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 93/134 2023-10-18T12:13:50.400 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-18T12:13:50.401 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-18T12:13:50.616 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-18T12:13:50.721 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-18T12:13:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-18T12:13:51.017 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-18T12:13:51.239 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-18T12:13:51.348 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-18T12:13:51.451 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-18T12:13:51.582 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-18T12:13:51.623 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-18T12:13:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 36/134 2023-10-18T12:13:51.826 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-18T12:13:51.961 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 37/134 2023-10-18T12:13:52.026 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-18T12:13:52.111 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-18T12:13:52.264 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-18T12:13:52.329 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-18T12:13:52.445 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T12:13:52.530 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T12:13:52.570 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-18T12:13:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-18T12:13:52.771 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-18T12:13:52.966 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-18T12:13:53.000 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-18T12:13:53.114 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T12:13:53.174 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-18T12:13:53.193 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T12:13:53.323 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-18T12:13:53.482 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-18T12:13:53.629 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-18T12:13:53.749 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-18T12:13:53.883 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-18T12:13:53.928 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-18T12:13:54.105 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-18T12:13:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-18T12:13:54.315 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-18T12:13:54.371 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-18T12:13:54.465 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-18T12:13:54.536 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-18T12:13:54.652 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-18T12:13:54.714 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-18T12:13:54.858 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-18T12:13:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-18T12:13:55.185 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-18T12:13:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:13:55.242 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-18T12:13:55.315 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-18T12:13:55.446 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-18T12:13:55.514 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-18T12:13:55.686 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-18T12:13:55.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-18T12:13:55.739 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:13:55.881 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-18T12:13:56.428 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-18T12:13:56.428 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-18T12:13:56.428 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-18T12:13:56.429 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-18T12:13:56.429 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-18T12:13:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:13:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-18T12:13:56.609 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-18T12:13:56.712 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-18T12:13:56.782 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T12:13:56.872 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-18T12:13:56.883 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-18T12:13:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T12:13:57.062 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T12:13:57.076 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-18T12:13:57.083 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-18T12:13:57.241 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-18T12:13:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-18T12:13:57.415 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-18T12:13:57.544 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-18T12:13:57.701 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-18T12:13:57.737 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-18T12:13:57.921 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-18T12:13:58.019 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-18T12:13:58.276 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-18T12:13:58.522 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-18T12:13:58.661 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-18T12:13:58.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-18T12:13:59.060 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-18T12:13:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-18T12:13:59.310 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-18T12:13:59.331 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-18T12:13:59.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-18T12:13:59.602 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-18T12:13:59.614 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-18T12:13:59.811 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-18T12:13:59.903 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-18T12:14:00.030 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-18T12:14:00.221 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-18T12:14:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-18T12:14:00.406 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-18T12:14:00.455 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-18T12:14:00.558 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-18T12:14:00.771 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-18T12:14:01.007 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-18T12:14:01.184 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-18T12:14:01.243 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-18T12:14:01.405 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-18T12:14:01.460 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T12:14:01.530 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T12:14:01.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-18T12:14:01.703 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 93/134 2023-10-18T12:14:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-18T12:14:02.017 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-18T12:14:03.135 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-18T12:14:03.416 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-18T12:14:03.637 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-18T12:14:03.925 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-18T12:14:04.248 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-18T12:14:04.480 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-18T12:14:05.278 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-18T12:14:05.423 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:14:05.444 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T12:14:05.515 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T12:14:05.694 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 93/134 2023-10-18T12:14:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:14:06.577 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:14:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T12:14:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T12:14:07.118 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T12:14:09.441 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:14:09.669 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:14:10.266 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T12:14:10.511 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T12:14:10.592 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T12:14:10.718 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T12:14:17.495 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T12:14:17.496 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2023-10-18T12:14:17.496 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2023-10-18T12:14:17.496 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /dev 2023-10-18T12:14:17.496 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /run 2023-10-18T12:14:17.496 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2023-10-18T12:14:17.497 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2023-10-18T12:14:17.497 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2023-10-18T12:14:17.497 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2023-10-18T12:14:17.497 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /dev 2023-10-18T12:14:17.497 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:14:17.925 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T12:14:19.646 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T12:14:19.843 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T12:14:20.111 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T12:14:20.290 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T12:14:20.811 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T12:14:21.065 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T12:14:21.106 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T12:14:23.951 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T12:14:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T12:14:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 102/134 2023-10-18T12:14:24.510 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 103/134 2023-10-18T12:14:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T12:14:24.958 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T12:14:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T12:14:26.166 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T12:14:26.167 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2023-10-18T12:14:26.167 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2023-10-18T12:14:26.167 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /dev 2023-10-18T12:14:26.167 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /run 2023-10-18T12:14:26.167 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2023-10-18T12:14:26.168 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2023-10-18T12:14:26.168 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2023-10-18T12:14:26.168 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2023-10-18T12:14:26.168 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /dev 2023-10-18T12:14:26.168 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:14:26.262 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T12:14:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T12:14:26.524 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T12:14:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T12:14:26.954 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 107/134 2023-10-18T12:14:27.177 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T12:14:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T12:14:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T12:14:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T12:14:28.117 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T12:14:28.299 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T12:14:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T12:14:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T12:14:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 110/134 2023-10-18T12:14:29.129 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T12:14:29.162 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T12:14:29.191 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T12:14:29.314 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T12:14:29.332 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T12:14:29.380 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T12:14:29.457 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T12:14:29.523 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T12:14:29.523 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-10-18T12:14:29.524 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-10-18T12:14:29.524 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-10-18T12:14:29.524 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-10-18T12:14:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-10-18T12:14:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-10-18T12:14:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-10-18T12:14:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-10-18T12:14:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-10-18T12:14:29.526 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:14:29.621 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T12:14:29.866 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T12:14:30.040 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T12:14:30.221 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T12:14:31.024 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T12:14:31.226 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T12:14:31.337 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 102/134 2023-10-18T12:14:31.459 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T12:14:31.546 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 103/134 2023-10-18T12:14:31.624 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T12:14:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T12:14:31.844 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T12:14:31.952 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T12:14:32.007 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T12:14:32.283 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T12:14:32.480 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T12:14:32.650 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T12:14:32.667 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T12:14:33.249 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T12:14:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T12:14:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T12:14:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 107/134 2023-10-18T12:14:34.244 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T12:14:34.280 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T12:14:34.430 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T12:14:34.441 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T12:14:34.607 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T12:14:34.784 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T12:14:34.921 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 102/134 2023-10-18T12:14:35.149 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 103/134 2023-10-18T12:14:35.256 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T12:14:35.323 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T12:14:35.545 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T12:14:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 110/134 2023-10-18T12:14:35.815 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T12:14:36.012 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T12:14:36.046 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T12:14:36.188 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T12:14:36.230 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T12:14:36.388 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T12:14:36.482 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 115/134 2023-10-18T12:14:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T12:14:36.777 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T12:14:36.853 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-18T12:14:36.897 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T12:14:36.947 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T12:14:37.041 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T12:14:37.092 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 117/134 2023-10-18T12:14:37.323 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-18T12:14:37.382 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 107/134 2023-10-18T12:14:37.502 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T12:14:37.621 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T12:14:37.656 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T12:14:37.752 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T12:14:37.798 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T12:14:37.808 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T12:14:38.049 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-18T12:14:38.249 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 121/134 2023-10-18T12:14:38.395 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T12:14:39.136 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 110/134 2023-10-18T12:14:39.341 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T12:14:39.375 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T12:14:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T12:14:39.573 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T12:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T12:14:40.082 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T12:14:40.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T12:14:41.546 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T12:14:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:42.131 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:42.170 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:42.394 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 124/134 2023-10-18T12:14:42.438 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 115/134 2023-10-18T12:14:42.611 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 125/134 2023-10-18T12:14:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-18T12:14:42.952 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 126/134 2023-10-18T12:14:42.974 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 117/134 2023-10-18T12:14:43.217 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-18T12:14:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-18T12:14:43.378 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T12:14:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T12:14:43.571 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T12:14:43.670 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T12:14:43.791 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-18T12:14:43.884 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T12:14:43.913 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T12:14:43.949 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 121/134 2023-10-18T12:14:44.096 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T12:14:44.134 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T12:14:44.157 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T12:14:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-18T12:14:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:44.574 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:44.574 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T12:14:44.866 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T12:14:44.900 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T12:14:45.084 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:45.372 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:45.372 INFO:teuthology.orchestra.run.smithi070.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-18T12:14:45.373 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:14:45.805 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 115/134 2023-10-18T12:14:45.988 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 1/134 2023-10-18T12:14:45.988 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2/134 2023-10-18T12:14:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 3/134 2023-10-18T12:14:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 4/134 2023-10-18T12:14:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/134 2023-10-18T12:14:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:14:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:14:45.990 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T12:14:45.990 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T12:14:45.990 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 10/134 2023-10-18T12:14:45.990 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 11/134 2023-10-18T12:14:45.990 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 12/134 2023-10-18T12:14:45.991 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 13/134 2023-10-18T12:14:45.991 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 14/134 2023-10-18T12:14:45.991 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 15/134 2023-10-18T12:14:45.991 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 16/134 2023-10-18T12:14:45.991 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 17/134 2023-10-18T12:14:45.992 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 18/134 2023-10-18T12:14:45.992 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 19/134 2023-10-18T12:14:45.992 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/134 2023-10-18T12:14:45.992 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 21/134 2023-10-18T12:14:45.992 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 22/134 2023-10-18T12:14:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 23/134 2023-10-18T12:14:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 24/134 2023-10-18T12:14:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 25/134 2023-10-18T12:14:46.002 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 26/134 2023-10-18T12:14:46.002 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 27/134 2023-10-18T12:14:46.003 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 28/134 2023-10-18T12:14:46.003 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/134 2023-10-18T12:14:46.003 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 30/134 2023-10-18T12:14:46.003 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 31/134 2023-10-18T12:14:46.003 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/134 2023-10-18T12:14:46.004 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/134 2023-10-18T12:14:46.004 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 34/134 2023-10-18T12:14:46.004 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/134 2023-10-18T12:14:46.004 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 36/134 2023-10-18T12:14:46.004 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 37/134 2023-10-18T12:14:46.004 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 38/134 2023-10-18T12:14:46.005 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-18T12:14:46.005 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-18T12:14:46.005 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-18T12:14:46.005 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-18T12:14:46.005 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-18T12:14:46.006 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-18T12:14:46.006 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-18T12:14:46.006 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-18T12:14:46.006 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-18T12:14:46.006 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-18T12:14:46.007 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-18T12:14:46.007 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-18T12:14:46.007 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-18T12:14:46.007 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-18T12:14:46.007 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-18T12:14:46.007 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-18T12:14:46.008 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-18T12:14:46.008 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-18T12:14:46.008 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-18T12:14:46.008 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-18T12:14:46.008 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-18T12:14:46.009 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-18T12:14:46.009 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-18T12:14:46.009 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-18T12:14:46.009 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-18T12:14:46.009 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-18T12:14:46.010 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-18T12:14:46.010 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-18T12:14:46.010 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-18T12:14:46.010 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-18T12:14:46.010 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-18T12:14:46.010 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-18T12:14:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-18T12:14:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-18T12:14:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-18T12:14:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-18T12:14:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-18T12:14:46.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-18T12:14:46.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-18T12:14:46.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-18T12:14:46.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-18T12:14:46.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-18T12:14:46.013 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-18T12:14:46.013 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-18T12:14:46.013 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-18T12:14:46.013 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-18T12:14:46.013 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-18T12:14:46.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-18T12:14:46.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-18T12:14:46.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-18T12:14:46.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-18T12:14:46.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-18T12:14:46.015 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-18T12:14:46.015 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-18T12:14:46.015 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-18T12:14:46.015 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-18T12:14:46.015 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-18T12:14:46.015 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-18T12:14:46.016 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-18T12:14:46.016 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-18T12:14:46.016 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-18T12:14:46.016 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-18T12:14:46.017 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-18T12:14:46.017 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-18T12:14:46.017 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-18T12:14:46.017 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-18T12:14:46.017 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-18T12:14:46.017 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-18T12:14:46.018 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-18T12:14:46.018 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-18T12:14:46.018 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-18T12:14:46.018 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-18T12:14:46.018 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-18T12:14:46.019 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-18T12:14:46.019 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-18T12:14:46.019 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-18T12:14:46.019 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-18T12:14:46.019 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-18T12:14:46.019 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-18T12:14:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-18T12:14:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-18T12:14:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-18T12:14:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-18T12:14:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-18T12:14:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-18T12:14:46.021 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-18T12:14:46.021 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-18T12:14:46.021 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-18T12:14:46.021 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-18T12:14:46.021 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-18T12:14:46.022 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 129/134 2023-10-18T12:14:46.022 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-18T12:14:46.022 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 131/134 2023-10-18T12:14:46.022 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:46.022 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-18T12:14:46.200 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-18T12:14:46.342 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:46.374 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 117/134 2023-10-18T12:14:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:46.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-18T12:14:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:46.805 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T12:14:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 124/134 2023-10-18T12:14:47.003 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T12:14:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 125/134 2023-10-18T12:14:47.225 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-18T12:14:47.389 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 126/134 2023-10-18T12:14:47.401 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 121/134 2023-10-18T12:14:47.581 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T12:14:47.762 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-18T12:14:47.936 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T12:14:48.013 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T12:14:48.228 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T12:14:48.240 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T12:14:48.403 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T12:14:48.411 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T12:14:48.538 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-18T12:14:48.539 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:48.737 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:48.737 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:05 ago on Wed 18 Oct 2023 12:12:43 PM UTC. 2023-10-18T12:14:48.764 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:48.764 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T12:14:48.991 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T12:14:49.176 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 1/134 2023-10-18T12:14:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2/134 2023-10-18T12:14:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 3/134 2023-10-18T12:14:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 4/134 2023-10-18T12:14:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/134 2023-10-18T12:14:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:14:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:14:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T12:14:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T12:14:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 10/134 2023-10-18T12:14:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 11/134 2023-10-18T12:14:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 12/134 2023-10-18T12:14:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 13/134 2023-10-18T12:14:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 14/134 2023-10-18T12:14:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 15/134 2023-10-18T12:14:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 16/134 2023-10-18T12:14:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 17/134 2023-10-18T12:14:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 18/134 2023-10-18T12:14:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 19/134 2023-10-18T12:14:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/134 2023-10-18T12:14:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 21/134 2023-10-18T12:14:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 22/134 2023-10-18T12:14:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 23/134 2023-10-18T12:14:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 24/134 2023-10-18T12:14:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 25/134 2023-10-18T12:14:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 26/134 2023-10-18T12:14:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 27/134 2023-10-18T12:14:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 28/134 2023-10-18T12:14:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/134 2023-10-18T12:14:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 30/134 2023-10-18T12:14:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 31/134 2023-10-18T12:14:49.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/134 2023-10-18T12:14:49.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/134 2023-10-18T12:14:49.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 34/134 2023-10-18T12:14:49.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/134 2023-10-18T12:14:49.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 36/134 2023-10-18T12:14:49.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 37/134 2023-10-18T12:14:49.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 38/134 2023-10-18T12:14:49.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-18T12:14:49.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-18T12:14:49.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-18T12:14:49.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-18T12:14:49.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-18T12:14:49.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-18T12:14:49.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-18T12:14:49.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-18T12:14:49.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-18T12:14:49.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-18T12:14:49.692 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-18T12:14:49.692 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-18T12:14:49.692 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-18T12:14:49.692 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-18T12:14:49.692 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-18T12:14:49.693 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-18T12:14:49.693 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-18T12:14:49.694 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-18T12:14:49.694 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-18T12:14:49.695 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-18T12:14:49.695 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-18T12:14:49.695 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-18T12:14:49.696 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-18T12:14:49.696 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-18T12:14:49.696 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-18T12:14:49.697 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-18T12:14:49.697 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-18T12:14:49.697 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-18T12:14:49.697 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-18T12:14:49.697 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-18T12:14:49.698 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-18T12:14:49.698 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-18T12:14:49.698 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-18T12:14:49.699 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-18T12:14:49.699 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-18T12:14:49.699 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-18T12:14:49.699 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-18T12:14:49.699 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-18T12:14:49.700 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-18T12:14:49.700 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-18T12:14:49.700 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-18T12:14:49.700 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-18T12:14:49.700 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-18T12:14:49.701 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-18T12:14:49.701 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-18T12:14:49.701 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-18T12:14:49.701 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-18T12:14:49.701 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-18T12:14:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-18T12:14:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-18T12:14:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-18T12:14:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-18T12:14:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-18T12:14:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-18T12:14:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-18T12:14:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-18T12:14:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-18T12:14:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-18T12:14:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-18T12:14:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-18T12:14:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-18T12:14:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-18T12:14:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-18T12:14:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-18T12:14:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-18T12:14:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-18T12:14:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-18T12:14:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-18T12:14:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-18T12:14:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-18T12:14:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-18T12:14:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-18T12:14:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-18T12:14:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-18T12:14:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-18T12:14:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-18T12:14:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-18T12:14:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-18T12:14:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-18T12:14:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-18T12:14:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-18T12:14:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-18T12:14:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-18T12:14:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-18T12:14:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-18T12:14:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-18T12:14:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-18T12:14:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-18T12:14:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-18T12:14:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-18T12:14:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 129/134 2023-10-18T12:14:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-18T12:14:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 131/134 2023-10-18T12:14:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-18T12:14:49.873 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:50.129 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:50.149 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T12:14:50.338 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 124/134 2023-10-18T12:14:50.595 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 125/134 2023-10-18T12:14:50.902 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 126/134 2023-10-18T12:14:51.272 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-18T12:14:51.454 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T12:14:51.537 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T12:14:51.754 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T12:14:51.766 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T12:14:51.913 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T12:14:51.921 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T12:14:52.031 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-18T12:14:52.032 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:52.196 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:52.196 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:52 ago on Wed 18 Oct 2023 12:13:00 PM UTC. 2023-10-18T12:14:52.216 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:52.216 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T12:14:52.468 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T12:14:52.695 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:53.203 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:53.203 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 1/134 2023-10-18T12:14:53.203 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2/134 2023-10-18T12:14:53.204 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 3/134 2023-10-18T12:14:53.204 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 4/134 2023-10-18T12:14:53.204 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/134 2023-10-18T12:14:53.204 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T12:14:53.204 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T12:14:53.205 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T12:14:53.205 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T12:14:53.205 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 10/134 2023-10-18T12:14:53.205 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 11/134 2023-10-18T12:14:53.205 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 12/134 2023-10-18T12:14:53.206 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 13/134 2023-10-18T12:14:53.206 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 14/134 2023-10-18T12:14:53.206 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 15/134 2023-10-18T12:14:53.206 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 16/134 2023-10-18T12:14:53.207 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 17/134 2023-10-18T12:14:53.207 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 18/134 2023-10-18T12:14:53.207 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 19/134 2023-10-18T12:14:53.207 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/134 2023-10-18T12:14:53.207 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 21/134 2023-10-18T12:14:53.208 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 22/134 2023-10-18T12:14:53.208 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 23/134 2023-10-18T12:14:53.208 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 24/134 2023-10-18T12:14:53.208 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 25/134 2023-10-18T12:14:53.208 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 26/134 2023-10-18T12:14:53.208 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 27/134 2023-10-18T12:14:53.209 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 28/134 2023-10-18T12:14:53.209 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/134 2023-10-18T12:14:53.209 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 30/134 2023-10-18T12:14:53.219 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 31/134 2023-10-18T12:14:53.220 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/134 2023-10-18T12:14:53.220 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/134 2023-10-18T12:14:53.220 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 34/134 2023-10-18T12:14:53.220 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/134 2023-10-18T12:14:53.221 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 36/134 2023-10-18T12:14:53.221 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 37/134 2023-10-18T12:14:53.221 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 38/134 2023-10-18T12:14:53.221 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-18T12:14:53.221 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-18T12:14:53.221 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-18T12:14:53.222 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-18T12:14:53.222 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-18T12:14:53.222 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-18T12:14:53.222 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-18T12:14:53.222 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-18T12:14:53.223 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-18T12:14:53.223 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-18T12:14:53.223 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-18T12:14:53.223 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-18T12:14:53.224 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-18T12:14:53.224 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-18T12:14:53.224 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-18T12:14:53.224 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-18T12:14:53.225 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-18T12:14:53.225 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-18T12:14:53.225 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-18T12:14:53.225 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-18T12:14:53.225 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-18T12:14:53.226 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-18T12:14:53.226 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-18T12:14:53.226 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-18T12:14:53.226 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-18T12:14:53.226 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-18T12:14:53.227 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-18T12:14:53.227 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-18T12:14:53.227 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-18T12:14:53.227 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-18T12:14:53.227 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-18T12:14:53.228 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-18T12:14:53.228 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-18T12:14:53.228 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-18T12:14:53.228 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-18T12:14:53.228 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-18T12:14:53.229 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-18T12:14:53.229 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-18T12:14:53.229 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-18T12:14:53.229 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-18T12:14:53.229 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-18T12:14:53.229 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-18T12:14:53.230 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-18T12:14:53.230 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-18T12:14:53.230 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-18T12:14:53.230 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-18T12:14:53.230 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-18T12:14:53.231 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-18T12:14:53.231 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-18T12:14:53.231 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-18T12:14:53.231 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-18T12:14:53.231 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-18T12:14:53.232 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-18T12:14:53.232 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-18T12:14:53.232 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-18T12:14:53.232 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-18T12:14:53.232 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-18T12:14:53.233 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-18T12:14:53.233 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-18T12:14:53.233 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-18T12:14:53.233 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-18T12:14:53.233 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-18T12:14:53.233 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-18T12:14:53.234 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-18T12:14:53.234 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-18T12:14:53.234 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-18T12:14:53.234 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-18T12:14:53.234 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-18T12:14:53.235 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-18T12:14:53.235 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-18T12:14:53.235 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-18T12:14:53.236 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-18T12:14:53.236 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-18T12:14:53.236 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-18T12:14:53.236 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-18T12:14:53.236 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-18T12:14:53.237 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-18T12:14:53.237 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-18T12:14:53.237 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-18T12:14:53.237 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-18T12:14:53.237 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-18T12:14:53.237 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-18T12:14:53.238 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-18T12:14:53.238 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-18T12:14:53.238 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-18T12:14:53.238 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-18T12:14:53.238 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-18T12:14:53.239 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-18T12:14:53.239 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-18T12:14:53.239 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-18T12:14:53.239 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 129/134 2023-10-18T12:14:53.239 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-18T12:14:53.240 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 131/134 2023-10-18T12:14:53.240 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T12:14:53.240 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-18T12:14:54.783 INFO:teuthology.orchestra.run.smithi070.stdout:Installed products updated. 2023-10-18T12:14:55.012 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:14:55.012 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2023-10-18T12:14:55.012 INFO:teuthology.orchestra.run.smithi070.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-18T12:14:55.013 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.013 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.013 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2023-10-18T12:14:55.013 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-18T12:14:55.013 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.1-7.el8.noarch 2023-10-18T12:14:55.014 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.0.4-10.el8.x86_64 2023-10-18T12:14:55.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.015 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.015 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.015 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.015 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.015 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.016 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.016 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.016 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.016 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.016 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.017 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.017 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.017 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.017 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.017 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.018 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.018 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.018 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.018 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:14:55.018 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-18T12:14:55.019 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.1-9.el8.x86_64 2023-10-18T12:14:55.019 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-18T12:14:55.019 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T12:14:55.019 INFO:teuthology.orchestra.run.smithi070.stdout: jq-1.6-3.el8.x86_64 2023-10-18T12:14:55.019 INFO:teuthology.orchestra.run.smithi070.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-18T12:14:55.019 INFO:teuthology.orchestra.run.smithi070.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-18T12:14:55.020 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-18T12:14:55.020 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.020 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.020 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.020 INFO:teuthology.orchestra.run.smithi070.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-18T12:14:55.021 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-18T12:14:55.021 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T12:14:55.021 INFO:teuthology.orchestra.run.smithi070.stdout: libpq-13.5-1.el8.x86_64 2023-10-18T12:14:55.021 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-18T12:14:55.021 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T12:14:55.022 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.022 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.022 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-18T12:14:55.022 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.022 INFO:teuthology.orchestra.run.smithi070.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-18T12:14:55.023 INFO:teuthology.orchestra.run.smithi070.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-18T12:14:55.023 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-18T12:14:55.023 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T12:14:55.023 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-18T12:14:55.023 INFO:teuthology.orchestra.run.smithi070.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-18T12:14:55.024 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T12:14:55.024 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T12:14:55.024 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-18T12:14:55.024 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-18T12:14:55.024 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-18T12:14:55.024 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-18T12:14:55.025 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-18T12:14:55.025 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-18T12:14:55.025 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:14:55.025 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:14:55.025 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:14:55.026 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-18T12:14:55.026 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-18T12:14:55.026 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T12:14:55.026 INFO:teuthology.orchestra.run.smithi070.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T12:14:55.026 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T12:14:55.027 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.027 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.027 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.027 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T12:14:55.027 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-18T12:14:55.028 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T12:14:55.028 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T12:14:55.028 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-18T12:14:55.028 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T12:14:55.028 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T12:14:55.028 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T12:14:55.029 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-18T12:14:55.029 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T12:14:55.029 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T12:14:55.029 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T12:14:55.029 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T12:14:55.030 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T12:14:55.030 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T12:14:55.030 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T12:14:55.030 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T12:14:55.030 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-18T12:14:55.031 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-18T12:14:55.031 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-18T12:14:55.031 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T12:14:55.031 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T12:14:55.031 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T12:14:55.032 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T12:14:55.032 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-18T12:14:55.032 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T12:14:55.032 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T12:14:55.032 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-18T12:14:55.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T12:14:55.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T12:14:55.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T12:14:55.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T12:14:55.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T12:14:55.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-18T12:14:55.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T12:14:55.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T12:14:55.035 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T12:14:55.035 INFO:teuthology.orchestra.run.smithi070.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T12:14:55.035 INFO:teuthology.orchestra.run.smithi070.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T12:14:55.035 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T12:14:55.035 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T12:14:55.036 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T12:14:55.036 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T12:14:55.036 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T12:14:55.036 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.036 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.037 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:14:55.037 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-18T12:14:55.037 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-18T12:14:55.037 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-18T12:14:55.037 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-18T12:14:55.037 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T12:14:55.038 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-18T12:14:55.038 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:14:55.039 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-10-18T12:14:55.329 DEBUG:teuthology.parallel:result is None 2023-10-18T12:14:55.713 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T12:14:55.713 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:50 ago on Wed 18 Oct 2023 12:13:05 PM UTC. 2023-10-18T12:15:00.026 INFO:teuthology.orchestra.run.smithi152.stdout:Installed products updated. 2023-10-18T12:15:00.240 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:15:00.240 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2023-10-18T12:15:00.241 INFO:teuthology.orchestra.run.smithi152.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-18T12:15:00.241 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.241 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.241 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-10-18T12:15:00.241 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-18T12:15:00.242 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.1-7.el8.noarch 2023-10-18T12:15:00.242 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.0.4-10.el8.x86_64 2023-10-18T12:15:00.242 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.242 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.242 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.243 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.243 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.243 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.243 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.243 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.246 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.246 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.246 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:00.247 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-18T12:15:00.247 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.1-9.el8.x86_64 2023-10-18T12:15:00.247 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-18T12:15:00.247 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T12:15:00.247 INFO:teuthology.orchestra.run.smithi152.stdout: jq-1.6-3.el8.x86_64 2023-10-18T12:15:00.248 INFO:teuthology.orchestra.run.smithi152.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-18T12:15:00.248 INFO:teuthology.orchestra.run.smithi152.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-18T12:15:00.248 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-18T12:15:00.248 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.248 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.249 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.249 INFO:teuthology.orchestra.run.smithi152.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-18T12:15:00.249 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-18T12:15:00.249 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T12:15:00.249 INFO:teuthology.orchestra.run.smithi152.stdout: libpq-13.5-1.el8.x86_64 2023-10-18T12:15:00.250 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-18T12:15:00.250 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T12:15:00.250 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.250 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.250 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-18T12:15:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-18T12:15:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-18T12:15:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-18T12:15:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T12:15:00.251 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-18T12:15:00.252 INFO:teuthology.orchestra.run.smithi152.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-18T12:15:00.252 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T12:15:00.252 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T12:15:00.252 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-18T12:15:00.252 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-18T12:15:00.253 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-18T12:15:00.253 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-18T12:15:00.253 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-18T12:15:00.253 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-18T12:15:00.253 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:15:00.254 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:15:00.254 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:15:00.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-18T12:15:00.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-18T12:15:00.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T12:15:00.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T12:15:00.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T12:15:00.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T12:15:00.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-18T12:15:00.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T12:15:00.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T12:15:00.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-18T12:15:00.257 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T12:15:00.257 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T12:15:00.257 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T12:15:00.257 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-18T12:15:00.258 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T12:15:00.258 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T12:15:00.258 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T12:15:00.258 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T12:15:00.258 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T12:15:00.259 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T12:15:00.259 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T12:15:00.259 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T12:15:00.259 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-18T12:15:00.259 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-18T12:15:00.260 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-18T12:15:00.260 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T12:15:00.260 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T12:15:00.260 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T12:15:00.260 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T12:15:00.260 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-18T12:15:00.261 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T12:15:00.261 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T12:15:00.261 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-18T12:15:00.261 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T12:15:00.261 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.262 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.262 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T12:15:00.262 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T12:15:00.262 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.262 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T12:15:00.263 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T12:15:00.263 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-18T12:15:00.263 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T12:15:00.263 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T12:15:00.263 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T12:15:00.264 INFO:teuthology.orchestra.run.smithi152.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T12:15:00.264 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T12:15:00.264 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T12:15:00.264 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T12:15:00.264 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T12:15:00.265 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T12:15:00.265 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T12:15:00.265 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.265 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.265 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:00.265 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-18T12:15:00.266 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-18T12:15:00.266 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-18T12:15:00.266 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-18T12:15:00.266 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T12:15:00.267 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-18T12:15:00.267 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-10-18T12:15:00.267 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-10-18T12:15:00.462 DEBUG:teuthology.parallel:result is None 2023-10-18T12:15:03.833 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-10-18T12:15:04.042 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:15:04.043 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-10-18T12:15:04.043 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-18T12:15:04.043 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.043 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.044 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-10-18T12:15:04.044 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-18T12:15:04.044 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-7.el8.noarch 2023-10-18T12:15:04.044 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2023-10-18T12:15:04.044 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.044 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.045 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.045 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.045 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.045 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.045 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.046 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.046 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.046 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.046 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.047 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.047 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.047 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.047 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.047 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.048 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.048 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.048 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.048 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.048 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.049 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.049 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T12:15:04.049 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-18T12:15:04.049 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2023-10-18T12:15:04.049 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-18T12:15:04.050 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T12:15:04.050 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-3.el8.x86_64 2023-10-18T12:15:04.050 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-18T12:15:04.050 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-18T12:15:04.050 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-18T12:15:04.051 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.051 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.051 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.051 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-18T12:15:04.051 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-18T12:15:04.052 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T12:15:04.052 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.5-1.el8.x86_64 2023-10-18T12:15:04.052 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-18T12:15:04.052 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T12:15:04.052 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.053 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.053 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-18T12:15:04.053 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.053 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-18T12:15:04.053 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-18T12:15:04.054 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-18T12:15:04.054 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T12:15:04.054 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-18T12:15:04.054 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-18T12:15:04.055 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T12:15:04.055 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T12:15:04.055 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-18T12:15:04.055 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-18T12:15:04.055 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-18T12:15:04.056 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-18T12:15:04.056 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-18T12:15:04.056 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-18T12:15:04.056 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:15:04.056 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:15:04.057 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T12:15:04.057 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-18T12:15:04.057 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-18T12:15:04.057 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T12:15:04.057 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T12:15:04.058 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T12:15:04.058 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.058 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.058 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.058 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T12:15:04.059 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-18T12:15:04.059 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T12:15:04.059 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T12:15:04.059 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-18T12:15:04.059 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T12:15:04.060 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T12:15:04.060 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T12:15:04.060 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-18T12:15:04.060 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T12:15:04.060 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T12:15:04.061 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T12:15:04.061 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T12:15:04.061 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T12:15:04.061 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T12:15:04.061 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T12:15:04.062 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T12:15:04.062 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-18T12:15:04.062 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-18T12:15:04.062 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-18T12:15:04.062 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T12:15:04.063 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T12:15:04.063 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T12:15:04.063 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T12:15:04.063 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-18T12:15:04.063 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T12:15:04.064 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T12:15:04.064 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-18T12:15:04.064 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T12:15:04.064 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.064 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.065 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T12:15:04.065 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T12:15:04.065 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.065 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T12:15:04.065 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T12:15:04.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-18T12:15:04.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T12:15:04.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T12:15:04.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T12:15:04.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T12:15:04.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T12:15:04.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T12:15:04.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T12:15:04.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T12:15:04.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T12:15:04.068 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T12:15:04.068 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.068 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.068 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T12:15:04.068 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-18T12:15:04.068 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-18T12:15:04.069 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-18T12:15:04.069 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-18T12:15:04.069 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T12:15:04.070 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-18T12:15:04.070 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:15:04.070 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-18T12:15:04.271 DEBUG:teuthology.parallel:result is None 2023-10-18T12:15:04.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:15:04.533 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T12:15:04.593 INFO:teuthology.orchestra.run.smithi070.stdout:18.0.0-6786.gf7420d7f.el8 2023-10-18T12:15:04.594 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el8 2023-10-18T12:15:04.594 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T12:15:04.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:15:04.817 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T12:15:04.845 INFO:teuthology.orchestra.run.smithi152.stdout:18.0.0-6786.gf7420d7f.el8 2023-10-18T12:15:04.845 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el8 2023-10-18T12:15:04.846 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T12:15:04.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:15:04.984 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T12:15:05.012 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-6786.gf7420d7f.el8 2023-10-18T12:15:05.012 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el8 2023-10-18T12:15:05.013 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T12:15:05.013 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T12:15:05.014 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:15:05.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T12:15:05.108 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:15:05.109 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T12:15:05.157 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:15:05.158 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T12:15:05.207 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T12:15:05.208 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:15:05.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T12:15:05.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T12:15:05.386 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:15:05.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T12:15:05.418 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T12:15:05.488 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:15:05.489 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T12:15:05.521 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T12:15:05.592 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T12:15:05.592 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:15:05.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T12:15:05.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T12:15:05.767 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:15:05.768 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T12:15:05.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T12:15:05.870 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:15:05.870 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T12:15:05.902 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T12:15:05.973 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T12:15:05.974 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:15:05.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T12:15:06.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T12:15:06.151 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:15:06.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T12:15:06.183 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T12:15:06.255 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:15:06.255 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T12:15:06.286 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T12:15:06.358 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-18T12:15:06.451 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'subvols': {'create': 2}} 2023-10-18T12:15:06.452 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:15:06.453 INFO:tasks.cephadm:Cluster fsid is f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:15:06.453 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-18T12:15:06.453 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.70', 'mon.b': '172.21.15.152', 'mon.c': '172.21.15.186'} 2023-10-18T12:15:06.454 INFO:tasks.cephadm:First mon is mon.a on smithi070 2023-10-18T12:15:06.454 INFO:tasks.cephadm:First mgr is x 2023-10-18T12:15:06.454 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-18T12:15:06.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2023-10-18T12:15:06.525 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2023-10-18T12:15:06.558 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2023-10-18T12:15:06.590 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-18T12:15:06.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:15:06.739 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73406/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6786-gf7420d7f', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6786.gf7420d7f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-16 23:26:05.686958', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231016.203825', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-10-18T12:15:06.975 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-batrick-testing-20231016.203825, sha1 f7420d7f8e55ceea786dcbf536c338ce920c21eb from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:15:06.976 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm 2023-10-18T12:15:06.976 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:07.702 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 18 12:15 /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:07.702 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.371 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 18 12:15 /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.372 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.868 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 18 12:15 /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.869 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.922 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.944 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-18T12:15:08.995 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb on all hosts... 2023-10-18T12:15:08.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb pull 2023-10-18T12:15:09.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb pull 2023-10-18T12:15:09.003 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb pull 2023-10-18T12:15:11.394 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T12:15:12.762 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T12:15:13.629 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T12:15:50.977 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-10-18T12:15:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:15:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120", 2023-10-18T12:15:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2023-10-18T12:15:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b60cde6693b3225f0a93b57dd34fe281c6ae755adca32bcc00f7ca131ec348" 2023-10-18T12:15:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-10-18T12:15:50.979 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-10-18T12:15:51.819 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-18T12:15:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:15:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120", 2023-10-18T12:15:51.820 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2023-10-18T12:15:51.820 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b60cde6693b3225f0a93b57dd34fe281c6ae755adca32bcc00f7ca131ec348" 2023-10-18T12:15:51.820 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2023-10-18T12:15:51.820 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-18T12:15:52.380 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:15:52.381 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:15:52.381 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120", 2023-10-18T12:15:52.381 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2023-10-18T12:15:52.382 INFO:teuthology.orchestra.run.smithi070.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b60cde6693b3225f0a93b57dd34fe281c6ae755adca32bcc00f7ca131ec348" 2023-10-18T12:15:52.382 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:15:52.382 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:15:52.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2023-10-18T12:15:52.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2023-10-18T12:15:52.516 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2023-10-18T12:15:52.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2023-10-18T12:15:52.621 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2023-10-18T12:15:52.654 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2023-10-18T12:15:52.684 INFO:tasks.cephadm:Writing seed config... 2023-10-18T12:15:52.703 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-18T12:15:52.703 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-18T12:15:52.704 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-18T12:15:52.704 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-18T12:15:52.704 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-18T12:15:52.705 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-18T12:15:52.705 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-18T12:15:52.705 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-18T12:15:52.706 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-18T12:15:52.706 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-18T12:15:52.706 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-18T12:15:52.707 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-18T12:15:52.707 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-18T12:15:52.707 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-18T12:15:52.708 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-18T12:15:52.708 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-18T12:15:52.708 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-18T12:15:52.708 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-18T12:15:52.709 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-18T12:15:52.709 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-18T12:15:52.709 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-18T12:15:52.710 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:15:52.711 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-18T12:15:52.754 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f94fde2a-6daf-11ee-8db6-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-10-18T12:15:52.755 DEBUG:teuthology.orchestra.run.smithi070:mon.a> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a.service 2023-10-18T12:15:52.802 DEBUG:teuthology.orchestra.run.smithi070:mgr.x> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x.service 2023-10-18T12:15:52.848 INFO:tasks.cephadm:Bootstrapping... 2023-10-18T12:15:52.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb -v bootstrap --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.70 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:15:52.868 INFO:journalctl@ceph.mon.a.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:15:52.915 INFO:journalctl@ceph.mgr.x.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:15:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-10-18T12:15:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', '-v', 'bootstrap', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.70', '--skip-admin-label'] 2023-10-18T12:15:53.250 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.1.1 2023-10-18T12:15:53.250 INFO:teuthology.orchestra.run.smithi070.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-18T12:15:53.251 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying podman|docker is present... 2023-10-18T12:15:53.423 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.1.1 2023-10-18T12:15:53.424 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying lvm2 is present... 2023-10-18T12:15:53.424 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying time synchronization is in place... 2023-10-18T12:15:53.437 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-18T12:15:53.438 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-18T12:15:53.450 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-18T12:15:53.451 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2023-10-18T12:15:53.464 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2023-10-18T12:15:53.477 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2023-10-18T12:15:53.477 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2023-10-18T12:15:53.478 INFO:teuthology.orchestra.run.smithi070.stdout:Repeating the final host check... 2023-10-18T12:15:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.1.1 2023-10-18T12:15:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-18T12:15:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl is present 2023-10-18T12:15:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:lvcreate is present 2023-10-18T12:15:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-18T12:15:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-18T12:15:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-18T12:15:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2023-10-18T12:15:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2023-10-18T12:15:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2023-10-18T12:15:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2023-10-18T12:15:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:Host looks OK 2023-10-18T12:15:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:Cluster fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:15:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:Acquiring lock 140131770857960 on /run/cephadm/f94fde2a-6daf-11ee-8db6-212e2dc638e7.lock 2023-10-18T12:15:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:Lock 140131770857960 acquired on /run/cephadm/f94fde2a-6daf-11ee-8db6-212e2dc638e7.lock 2023-10-18T12:15:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 3300 ... 2023-10-18T12:15:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 6789 ... 2023-10-18T12:15:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:Base mon IP(s) is [172.21.15.70:3300, 172.21.15.70:6789], mon addrv is [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2023-10-18T12:15:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-10-18T12:15:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.70 metric 100 2023-10-18T12:15:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-18T12:15:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-10-18T12:15:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-10-18T12:15:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-18T12:15:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-18T12:15:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-18T12:15:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-10-18T12:15:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 fe80::6f76:96b5:cb68:c2e6/64 scope link noprefixroute 2023-10-18T12:15:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-18T12:15:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2023-10-18T12:15:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2023-10-18T12:15:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-18T12:15:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-18T12:15:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T12:15:54.200 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120 2023-10-18T12:15:54.200 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T12:15:54.200 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Getting image source signatures 2023-10-18T12:15:54.201 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:bde102e0dc1fccb96f1164f2fe1ba9d3f5d7242e22d2b672b1db3aedff4d056f 2023-10-18T12:15:54.201 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:8ed15414cb45b8b895a21f106743ceee45272cee0d9e7e5b1f49bd1e877e4a11 2023-10-18T12:15:54.201 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying config sha256:34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120 2023-10-18T12:15:54.201 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-18T12:15:54.201 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Storing signatures 2023-10-18T12:15:55.477 INFO:teuthology.orchestra.run.smithi070.stdout:ceph: stdout ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev) 2023-10-18T12:15:55.477 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph version: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev) 2023-10-18T12:15:55.478 INFO:teuthology.orchestra.run.smithi070.stdout:Extracting ceph user uid/gid from container image... 2023-10-18T12:15:56.156 INFO:teuthology.orchestra.run.smithi070.stdout:stat: stdout 167 167 2023-10-18T12:15:56.156 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial keys... 2023-10-18T12:15:56.930 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQB8zC9ltfq+IhAAtJAtnowSguEAqtWXPvfdHw== 2023-10-18T12:15:57.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQB9zC9lxWb6EhAAI7g6ZOdrqFYCoHNCjoaNVA== 2023-10-18T12:15:58.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQB+zC9lOt0VDBAARnxKgHt2WJcYySIusFedmw== 2023-10-18T12:15:58.682 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial monmap... 2023-10-18T12:15:59.462 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-18T12:15:59.462 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-18T12:15:59.463 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:15:59.463 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-18T12:15:59.463 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool for a [v2:172.21.15.70:3300,v1:172.21.15.70:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-18T12:15:59.464 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = pacific 2023-10-18T12:15:59.464 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: set fsid to f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:15:59.464 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-18T12:15:59.465 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:15:59.465 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mon... 2023-10-18T12:16:01.752 INFO:teuthology.orchestra.run.smithi070.stdout:create mon.a on 2023-10-18T12:16:02.028 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-18T12:16:02.223 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-18T12:16:02.455 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7.target. 2023-10-18T12:16:02.456 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7.target. 2023-10-18T12:16:02.812 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a 2023-10-18T12:16:02.812 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a.service: Unit ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a.service not loaded. 2023-10-18T12:16:03.001 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7.target.wants/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@.service. 2023-10-18T12:16:03.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:03 smithi070 systemd[1]: Starting Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:16:03.809 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T12:16:04.044 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2023-10-18T12:16:04.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:03 smithi070 podman[98094]: 2023-10-18T12:16:04.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:03 smithi070 bash[98094]: 5f3f01764d495bc85a593a5e1c91c7783cd38c5ee504eb77055044f17b9da832 2023-10-18T12:16:04.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:03 smithi070 systemd[1]: Started Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:16:04.046 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T12:16:04.046 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2023-10-18T12:16:04.046 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2023-10-18T12:16:04.047 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2023-10-18T12:16:04.047 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon to start... 2023-10-18T12:16:04.047 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon... 2023-10-18T12:16:04.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:03 smithi070 ceph-mon[98134]: mkfs f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:04.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:03 smithi070 ceph-mon[98134]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-18T12:16:04.908 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout cluster: 2023-10-18T12:16:04.909 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout id: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-18T12:16:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout services: 2023-10-18T12:16:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.682112s) 2023-10-18T12:16:04.910 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-18T12:16:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-18T12:16:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:04.911 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout data: 2023-10-18T12:16:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-18T12:16:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-18T12:16:04.912 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-18T12:16:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pgs: 2023-10-18T12:16:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:mon is available 2023-10-18T12:16:04.913 INFO:teuthology.orchestra.run.smithi070.stdout:Assimilating anything we can from ceph.conf... 2023-10-18T12:16:05.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:04 smithi070 ceph-mon[98134]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-18T12:16:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:04 smithi070 ceph-mon[98134]: monmap e1: 1 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:16:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:04 smithi070 ceph-mon[98134]: fsmap 2023-10-18T12:16:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:04 smithi070 ceph-mon[98134]: osdmap e1: 0 total, 0 up, 0 in 2023-10-18T12:16:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:04 smithi070 ceph-mon[98134]: mgrmap e1: no daemons active 2023-10-18T12:16:05.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:04 smithi070 ceph-mon[98134]: from='client.? 172.21.15.70:0/3709785219' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-18T12:16:05.989 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:05.990 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2023-10-18T12:16:05.990 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:05.990 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2023-10-18T12:16:05.991 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-18T12:16:05.991 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-18T12:16:05.991 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-18T12:16:05.992 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-18T12:16:05.992 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:05.992 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-18T12:16:05.993 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-18T12:16:05.993 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:05.994 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2023-10-18T12:16:05.994 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-18T12:16:05.994 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-18T12:16:05.995 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-18T12:16:05.995 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-18T12:16:05.995 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-18T12:16:05.995 INFO:teuthology.orchestra.run.smithi070.stdout:Generating new minimal ceph.conf... 2023-10-18T12:16:06.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:05 smithi070 ceph-mon[98134]: from='client.? 172.21.15.70:0/606538825' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-18T12:16:06.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:05 smithi070 ceph-mon[98134]: from='client.? 172.21.15.70:0/606538825' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-18T12:16:06.966 INFO:teuthology.orchestra.run.smithi070.stdout:Restarting the monitor... 2023-10-18T12:16:06.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:06 smithi070 ceph-mon[98134]: from='client.? 172.21.15.70:0/1942767625' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:16:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:07 smithi070 systemd[1]: Stopping Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:16:07.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a[98110]: 2023-10-18T12:16:07.425+0000 7f63a8ed3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:16:07.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a[98110]: 2023-10-18T12:16:07.425+0000 7f63a8ed3700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-18T12:16:08.096 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:07 smithi070 bash[98470]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a 2023-10-18T12:16:08.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 systemd[1]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a.service: Succeeded. 2023-10-18T12:16:08.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 systemd[1]: Stopped Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:16:08.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 systemd[1]: Starting Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:16:08.974 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 podman[98599]: 2023-10-18T12:16:08.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 bash[98599]: 7c1966836a81b06c2528602126039815212af42e64267cfc18474ebb97591765 2023-10-18T12:16:08.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: set uid:gid to 167:167 (ceph:ceph) 2023-10-18T12:16:09.033 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev), process ceph-mon, pid 7 2023-10-18T12:16:09.034 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: pidfile_write: ignore empty --pid-file 2023-10-18T12:16:09.034 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: load: jerasure load: lrc 2023-10-18T12:16:09.034 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: RocksDB version: 7.9.2 2023-10-18T12:16:09.034 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Git sha 0 2023-10-18T12:16:09.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Compile date 2023-10-16 22:08:19 2023-10-18T12:16:09.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: DB SUMMARY 2023-10-18T12:16:09.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: DB Session ID: IDCYO6ATYSQIY1L9U3HC 2023-10-18T12:16:09.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: CURRENT file: CURRENT 2023-10-18T12:16:09.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: IDENTITY file: IDENTITY 2023-10-18T12:16:09.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-18T12:16:09.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-18T12:16:09.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88216 ; 2023-10-18T12:16:09.037 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.error_if_exists: 0 2023-10-18T12:16:09.037 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.create_if_missing: 0 2023-10-18T12:16:09.037 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.paranoid_checks: 1 2023-10-18T12:16:09.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-18T12:16:09.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-18T12:16:09.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-18T12:16:09.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.env: 0x55e6ff04ada0 2023-10-18T12:16:09.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.fs: PosixFileSystem 2023-10-18T12:16:09.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.info_log: 0x55e7003e16a0 2023-10-18T12:16:09.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_file_opening_threads: 16 2023-10-18T12:16:09.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.statistics: (nil) 2023-10-18T12:16:09.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.use_fsync: 0 2023-10-18T12:16:09.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_log_file_size: 0 2023-10-18T12:16:09.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-18T12:16:09.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-18T12:16:09.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.keep_log_file_num: 1000 2023-10-18T12:16:09.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.recycle_log_file_num: 0 2023-10-18T12:16:09.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.allow_fallocate: 1 2023-10-18T12:16:09.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.allow_mmap_reads: 0 2023-10-18T12:16:09.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.allow_mmap_writes: 0 2023-10-18T12:16:09.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.use_direct_reads: 0 2023-10-18T12:16:09.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-18T12:16:09.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.create_missing_column_families: 0 2023-10-18T12:16:09.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.db_log_dir: 2023-10-18T12:16:09.042 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.wal_dir: 2023-10-18T12:16:09.042 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-18T12:16:09.042 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-18T12:16:09.042 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-18T12:16:09.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-18T12:16:09.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-18T12:16:09.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-18T12:16:09.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.advise_random_on_open: 1 2023-10-18T12:16:09.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.db_write_buffer_size: 0 2023-10-18T12:16:09.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.write_buffer_manager: 0x55e700399360 2023-10-18T12:16:09.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-18T12:16:09.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-18T12:16:09.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-18T12:16:09.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.rate_limiter: (nil) 2023-10-18T12:16:09.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-18T12:16:09.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.wal_recovery_mode: 2 2023-10-18T12:16:09.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.enable_thread_tracking: 0 2023-10-18T12:16:09.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.enable_pipelined_write: 0 2023-10-18T12:16:09.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.unordered_write: 0 2023-10-18T12:16:09.045 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-18T12:16:09.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-18T12:16:09.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-18T12:16:09.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-18T12:16:09.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.row_cache: None 2023-10-18T12:16:09.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.wal_filter: None 2023-10-18T12:16:09.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-18T12:16:09.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.allow_ingest_behind: 0 2023-10-18T12:16:09.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.two_write_queues: 0 2023-10-18T12:16:09.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.manual_wal_flush: 0 2023-10-18T12:16:09.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.wal_compression: 0 2023-10-18T12:16:09.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.atomic_flush: 0 2023-10-18T12:16:09.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-18T12:16:09.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-18T12:16:09.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-18T12:16:09.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.log_readahead_size: 0 2023-10-18T12:16:09.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-18T12:16:09.049 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.best_efforts_recovery: 0 2023-10-18T12:16:09.049 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-18T12:16:09.049 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-18T12:16:09.049 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.allow_data_in_errors: 0 2023-10-18T12:16:09.049 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.db_host_id: __hostname__ 2023-10-18T12:16:09.050 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-18T12:16:09.051 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_background_jobs: 2 2023-10-18T12:16:09.051 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_background_compactions: -1 2023-10-18T12:16:09.051 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_subcompactions: 1 2023-10-18T12:16:09.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-18T12:16:09.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-18T12:16:09.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-18T12:16:09.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_total_wal_size: 0 2023-10-18T12:16:09.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-18T12:16:09.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-18T12:16:09.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-18T12:16:09.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-18T12:16:09.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_open_files: -1 2023-10-18T12:16:09.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bytes_per_sync: 0 2023-10-18T12:16:09.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-18T12:16:09.054 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-18T12:16:09.054 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_readahead_size: 0 2023-10-18T12:16:09.054 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_background_flushes: -1 2023-10-18T12:16:09.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Compression algorithms supported: 2023-10-18T12:16:09.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-18T12:16:09.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kZSTD supported: 0 2023-10-18T12:16:09.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kXpressCompression supported: 0 2023-10-18T12:16:09.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kLZ4HCCompression supported: 1 2023-10-18T12:16:09.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kZlibCompression supported: 1 2023-10-18T12:16:09.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kSnappyCompression supported: 1 2023-10-18T12:16:09.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kLZ4Compression supported: 1 2023-10-18T12:16:09.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: kBZip2Compression supported: 0 2023-10-18T12:16:09.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-18T12:16:09.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-18T12:16:09.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-18T12:16:09.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-18T12:16:09.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-18T12:16:09.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.merge_operator: 2023-10-18T12:16:09.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_filter: None 2023-10-18T12:16:09.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_filter_factory: None 2023-10-18T12:16:09.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.sst_partitioner_factory: None 2023-10-18T12:16:09.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-18T12:16:09.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-18T12:16:09.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e7003e17a0) 2023-10-18T12:16:09.058 INFO:journalctl@ceph.mon.a.smithi070.stdout: cache_index_and_filter_blocks: 1 2023-10-18T12:16:09.059 INFO:journalctl@ceph.mon.a.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-18T12:16:09.059 INFO:journalctl@ceph.mon.a.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-18T12:16:09.059 INFO:journalctl@ceph.mon.a.smithi070.stdout: pin_top_level_index_and_filter: 1 2023-10-18T12:16:09.059 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_type: 0 2023-10-18T12:16:09.059 INFO:journalctl@ceph.mon.a.smithi070.stdout: data_block_index_type: 0 2023-10-18T12:16:09.060 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_shortening: 1 2023-10-18T12:16:09.060 INFO:journalctl@ceph.mon.a.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-18T12:16:09.060 INFO:journalctl@ceph.mon.a.smithi070.stdout: checksum: 4 2023-10-18T12:16:09.060 INFO:journalctl@ceph.mon.a.smithi070.stdout: no_block_cache: 0 2023-10-18T12:16:09.060 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache: 0x55e7004371f0 2023-10-18T12:16:09.061 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_name: BinnedLRUCache 2023-10-18T12:16:09.061 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_options: 2023-10-18T12:16:09.061 INFO:journalctl@ceph.mon.a.smithi070.stdout: capacity : 536870912 2023-10-18T12:16:09.061 INFO:journalctl@ceph.mon.a.smithi070.stdout: num_shard_bits : 4 2023-10-18T12:16:09.061 INFO:journalctl@ceph.mon.a.smithi070.stdout: strict_capacity_limit : 0 2023-10-18T12:16:09.061 INFO:journalctl@ceph.mon.a.smithi070.stdout: high_pri_pool_ratio: 0.000 2023-10-18T12:16:09.062 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_compressed: (nil) 2023-10-18T12:16:09.062 INFO:journalctl@ceph.mon.a.smithi070.stdout: persistent_cache: (nil) 2023-10-18T12:16:09.062 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_size: 4096 2023-10-18T12:16:09.062 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_size_deviation: 10 2023-10-18T12:16:09.062 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_restart_interval: 16 2023-10-18T12:16:09.063 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_block_restart_interval: 1 2023-10-18T12:16:09.063 INFO:journalctl@ceph.mon.a.smithi070.stdout: metadata_block_size: 4096 2023-10-18T12:16:09.063 INFO:journalctl@ceph.mon.a.smithi070.stdout: partition_filters: 0 2023-10-18T12:16:09.063 INFO:journalctl@ceph.mon.a.smithi070.stdout: use_delta_encoding: 1 2023-10-18T12:16:09.063 INFO:journalctl@ceph.mon.a.smithi070.stdout: filter_policy: bloomfilter 2023-10-18T12:16:09.064 INFO:journalctl@ceph.mon.a.smithi070.stdout: whole_key_filtering: 1 2023-10-18T12:16:09.064 INFO:journalctl@ceph.mon.a.smithi070.stdout: verify_compression: 0 2023-10-18T12:16:09.064 INFO:journalctl@ceph.mon.a.smithi070.stdout: read_amp_bytes_per_bit: 0 2023-10-18T12:16:09.064 INFO:journalctl@ceph.mon.a.smithi070.stdout: format_version: 5 2023-10-18T12:16:09.064 INFO:journalctl@ceph.mon.a.smithi070.stdout: enable_index_compression: 1 2023-10-18T12:16:09.065 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_align: 0 2023-10-18T12:16:09.065 INFO:journalctl@ceph.mon.a.smithi070.stdout: max_auto_readahead_size: 262144 2023-10-18T12:16:09.065 INFO:journalctl@ceph.mon.a.smithi070.stdout: prepopulate_block_cache: 0 2023-10-18T12:16:09.065 INFO:journalctl@ceph.mon.a.smithi070.stdout: initial_auto_readahead_size: 8192 2023-10-18T12:16:09.065 INFO:journalctl@ceph.mon.a.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2023-10-18T12:16:09.065 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.write_buffer_size: 33554432 2023-10-18T12:16:09.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_write_buffer_number: 2 2023-10-18T12:16:09.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression: NoCompression 2023-10-18T12:16:09.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression: Disabled 2023-10-18T12:16:09.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.prefix_extractor: nullptr 2023-10-18T12:16:09.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-18T12:16:09.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.num_levels: 7 2023-10-18T12:16:09.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-18T12:16:09.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-18T12:16:09.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-18T12:16:09.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-18T12:16:09.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-18T12:16:09.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-18T12:16:09.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-18T12:16:09.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-18T12:16:09.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-18T12:16:09.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-18T12:16:09.069 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-18T12:16:09.069 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-18T12:16:09.069 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-18T12:16:09.069 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.level: 32767 2023-10-18T12:16:09.069 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.strategy: 0 2023-10-18T12:16:09.070 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-18T12:16:09.070 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-18T12:16:09.070 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-18T12:16:09.070 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-18T12:16:09.070 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.enabled: false 2023-10-18T12:16:09.071 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-18T12:16:09.071 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-18T12:16:09.071 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-18T12:16:09.071 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-18T12:16:09.071 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.target_file_size_base: 67108864 2023-10-18T12:16:09.072 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-18T12:16:09.072 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-18T12:16:09.072 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-18T12:16:09.072 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-18T12:16:09.072 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-18T12:16:09.072 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-18T12:16:09.073 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-18T12:16:09.073 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-18T12:16:09.073 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-18T12:16:09.073 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-18T12:16:09.073 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-18T12:16:09.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-18T12:16:09.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-18T12:16:09.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-18T12:16:09.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.arena_block_size: 1048576 2023-10-18T12:16:09.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-18T12:16:09.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-18T12:16:09.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.disable_auto_compactions: 0 2023-10-18T12:16:09.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-18T12:16:09.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-18T12:16:09.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-18T12:16:09.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-18T12:16:09.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-18T12:16:09.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-18T12:16:09.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-18T12:16:09.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-18T12:16:09.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-18T12:16:09.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-18T12:16:09.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.table_properties_collectors: 2023-10-18T12:16:09.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.inplace_update_support: 0 2023-10-18T12:16:09.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-18T12:16:09.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-18T12:16:09.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-18T12:16:09.078 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-18T12:16:09.078 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.bloom_locality: 0 2023-10-18T12:16:09.078 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.max_successive_merges: 0 2023-10-18T12:16:09.078 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-18T12:16:09.078 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.paranoid_file_checks: 0 2023-10-18T12:16:09.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.force_consistency_checks: 1 2023-10-18T12:16:09.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.report_bg_io_stats: 0 2023-10-18T12:16:09.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.ttl: 2592000 2023-10-18T12:16:09.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-18T12:16:09.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-18T12:16:09.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-18T12:16:09.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.enable_blob_files: false 2023-10-18T12:16:09.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.min_blob_size: 0 2023-10-18T12:16:09.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.blob_file_size: 268435456 2023-10-18T12:16:09.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-18T12:16:09.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-18T12:16:09.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-18T12:16:09.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-18T12:16:09.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-18T12:16:09.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.blob_file_starting_level: 0 2023-10-18T12:16:09.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-18T12:16:09.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-18T12:16:09.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-18T12:16:09.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0ce7c55d-e7cc-4939-a92a-d303d5c424f4 2023-10-18T12:16:09.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631368982162, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-18T12:16:09.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-18T12:16:09.083 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631368983560, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 84476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 268, "table_properties": {"data_size": 82644, "index_size": 242, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10967, "raw_average_key_size": 48, "raw_value_size": 76477, "raw_average_value_size": 336, "num_data_blocks": 10, "num_entries": 227, "num_filter_entries": 227, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697631368, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0ce7c55d-e7cc-4939-a92a-d303d5c424f4", "db_session_id": "IDCYO6ATYSQIY1L9U3HC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-18T12:16:09.083 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631368983690, "job": 1, "event": "recovery_finished"} 2023-10-18T12:16:09.083 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-18T12:16:09.083 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-18T12:16:09.083 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e7004d4000 2023-10-18T12:16:09.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: DB pointer 0x55e7004c0000 2023-10-18T12:16:09.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-18T12:16:09.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-18T12:16:09.084 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** DB Stats ** 2023-10-18T12:16:09.084 INFO:journalctl@ceph.mon.a.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T12:16:09.085 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-18T12:16:09.085 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T12:16:09.085 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T12:16:09.085 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-18T12:16:09.085 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T12:16:09.085 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T12:16:09.086 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2023-10-18T12:16:09.086 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** Compaction Stats [default] ** 2023-10-18T12:16:09.086 INFO:journalctl@ceph.mon.a.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T12:16:09.086 INFO:journalctl@ceph.mon.a.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-18T12:16:09.086 INFO:journalctl@ceph.mon.a.smithi070.stdout: L0 2/0 84.33 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 92.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:16:09.087 INFO:journalctl@ceph.mon.a.smithi070.stdout: Sum 2/0 84.33 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 92.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:16:09.087 INFO:journalctl@ceph.mon.a.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 92.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:16:09.087 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2023-10-18T12:16:09.087 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** Compaction Stats [default] ** 2023-10-18T12:16:09.087 INFO:journalctl@ceph.mon.a.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T12:16:09.088 INFO:journalctl@ceph.mon.a.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-18T12:16:09.088 INFO:journalctl@ceph.mon.a.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 92.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:16:09.088 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2023-10-18T12:16:09.088 INFO:journalctl@ceph.mon.a.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-18T12:16:09.088 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2023-10-18T12:16:09.089 INFO:journalctl@ceph.mon.a.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T12:16:09.089 INFO:journalctl@ceph.mon.a.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-18T12:16:09.089 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-18T12:16:09.089 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-18T12:16:09.089 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-18T12:16:09.089 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-18T12:16:09.090 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative compaction: 0.00 GB write, 13.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T12:16:09.090 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval compaction: 0.00 GB write, 13.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T12:16:09.090 INFO:journalctl@ceph.mon.a.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-18T12:16:09.090 INFO:journalctl@ceph.mon.a.smithi070.stdout: Block cache BinnedLRUCache@0x55e7004371f0#7 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-10-18T12:16:09.090 INFO:journalctl@ceph.mon.a.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2023-10-18T12:16:09.091 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2023-10-18T12:16:09.091 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-18T12:16:09.091 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: starting mon.a rank 0 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:09.091 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???) e1 preinit fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:09.091 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).mds e1 new map 2023-10-18T12:16:09.092 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).mds e1 print_map 2023-10-18T12:16:09.092 INFO:journalctl@ceph.mon.a.smithi070.stdout: e1 2023-10-18T12:16:09.092 INFO:journalctl@ceph.mon.a.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T12:16:09.092 INFO:journalctl@ceph.mon.a.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T12:16:09.092 INFO:journalctl@ceph.mon.a.smithi070.stdout: legacy client fscid: -1 2023-10-18T12:16:09.093 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2023-10-18T12:16:09.093 INFO:journalctl@ceph.mon.a.smithi070.stdout: No filesystems configured 2023-10-18T12:16:09.093 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-18T12:16:09.093 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:16:09.093 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:16:09.094 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:16:09.094 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:08 smithi070 ceph-mon[98639]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-18T12:16:09.094 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:09 smithi070 systemd[1]: Started Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:16:09.094 INFO:teuthology.orchestra.run.smithi070.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-18T12:16:09.296 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:09 smithi070 ceph-mon[98639]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-18T12:16:09.296 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:09 smithi070 ceph-mon[98639]: monmap e1: 1 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:16:09.297 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:09 smithi070 ceph-mon[98639]: fsmap 2023-10-18T12:16:09.297 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:09 smithi070 ceph-mon[98639]: osdmap e1: 0 total, 0 up, 0 in 2023-10-18T12:16:09.297 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:09 smithi070 ceph-mon[98639]: mgrmap e1: no daemons active 2023-10-18T12:16:10.017 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-18T12:16:10.039 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:16:10.039 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mgr... 2023-10-18T12:16:10.040 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-18T12:16:10.040 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-18T12:16:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x 2023-10-18T12:16:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x.service: Unit ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x.service not loaded. 2023-10-18T12:16:10.590 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7.target.wants/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@.service. 2023-10-18T12:16:10.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:10 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3724077915' entity='client.admin' 2023-10-18T12:16:10.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:10 smithi070 systemd[1]: Starting Ceph mgr.x for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:16:11.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 podman[98870]: 2023-10-18T12:16:11.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 bash[98870]: 5879f69518d52a1e73c7987b1b951b9288ba254c837f31ef105327e9e6ae6530 2023-10-18T12:16:11.389 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T12:16:11.390 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2023-10-18T12:16:11.405 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T12:16:11.406 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2023-10-18T12:16:11.406 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2023-10-18T12:16:11.406 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2023-10-18T12:16:11.426 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T12:16:11.426 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2023-10-18T12:16:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T12:16:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2023-10-18T12:16:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2023-10-18T12:16:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-18T12:16:11.446 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr to start... 2023-10-18T12:16:11.446 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr... 2023-10-18T12:16:11.628 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 systemd[1]: Started Ceph mgr.x for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:16:11.628 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:11.511+0000 7f1cb0f74200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:16:11.629 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:11.624+0000 7f1cb0f74200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:16:12.103 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:11.742+0000 7f1cb0f74200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:16:12.103 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:11.873+0000 7f1cb0f74200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:16:12.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1562026479' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-18T12:16:12.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:12.100+0000 7f1cb0f74200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:16:12.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:12.553 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:16:12.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "f94fde2a-6daf-11ee-8db6-212e2dc638e7", 2023-10-18T12:16:12.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2023-10-18T12:16:12.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-18T12:16:12.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-18T12:16:12.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-18T12:16:12.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-18T12:16:12.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-18T12:16:12.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2023-10-18T12:16:12.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-18T12:16:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2023-10-18T12:16:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-18T12:16:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-18T12:16:12.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:12.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-18T12:16:12.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-18T12:16:12.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-18T12:16:12.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:12.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-18T12:16:12.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-18T12:16:12.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-18T12:16:12.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-18T12:16:12.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-18T12:16:12.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-18T12:16:12.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-18T12:16:12.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-18T12:16:12.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-18T12:16:12.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-18T12:16:12.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-18T12:16:12.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-18T12:16:12.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-18T12:16:12.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-18T12:16:12.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-18T12:16:12.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-18T12:16:12.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:12.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-18T12:16:12.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-18T12:16:12.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-18T12:16:12.566 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-18T12:16:12.566 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-18T12:16:12.566 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-18T12:16:12.566 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-18T12:16:12.566 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-18T12:16:12.567 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2023-10-18T12:16:12.567 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:12.567 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T12:16:12.567 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.567 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-18T12:16:12.567 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:12.568 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2023-10-18T12:16:03.805485+0000", 2023-10-18T12:16:12.568 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T12:16:12.568 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:12.568 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-18T12:16:12.568 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:16:12.569 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (1/15)... 2023-10-18T12:16:12.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:12.593+0000 7f1cb0f74200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:16:12.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:12.758+0000 7f1cb0f74200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:16:13.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:12.873+0000 7f1cb0f74200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:16:13.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:12.996+0000 7f1cb0f74200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:16:13.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:13.534+0000 7f1cb0f74200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:16:14.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:14.411+0000 7f1cb0f74200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:16:14.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:14.516+0000 7f1cb0f74200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:16:15.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:15 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1516908756' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-18T12:16:15.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:15.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:16:15.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "f94fde2a-6daf-11ee-8db6-212e2dc638e7", 2023-10-18T12:16:15.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2023-10-18T12:16:15.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-18T12:16:15.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-18T12:16:15.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-18T12:16:15.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-18T12:16:15.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-18T12:16:15.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2023-10-18T12:16:15.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:15.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-18T12:16:15.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2023-10-18T12:16:15.552 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:15.552 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-18T12:16:15.552 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-18T12:16:15.552 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:15.553 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-18T12:16:15.553 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-18T12:16:15.553 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.553 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-18T12:16:15.553 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:15.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-18T12:16:15.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-18T12:16:15.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-18T12:16:15.554 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-18T12:16:15.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-18T12:16:15.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-18T12:16:15.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-18T12:16:15.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-18T12:16:15.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-18T12:16:15.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-18T12:16:15.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-18T12:16:15.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-18T12:16:15.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-18T12:16:15.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-18T12:16:15.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-18T12:16:15.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-18T12:16:15.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:15.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-18T12:16:15.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-18T12:16:15.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-18T12:16:15.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-18T12:16:15.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-18T12:16:15.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-18T12:16:15.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-18T12:16:15.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-18T12:16:15.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2023-10-18T12:16:15.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:15.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T12:16:15.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-18T12:16:15.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:15.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2023-10-18T12:16:03.805485+0000", 2023-10-18T12:16:15.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T12:16:15.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:15.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-18T12:16:15.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:16:15.562 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (2/15)... 2023-10-18T12:16:15.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:15.546+0000 7f1cb0f74200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:16:15.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:15.666+0000 7f1cb0f74200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:16:16.262 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:15.893+0000 7f1cb0f74200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:16:16.544 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:16.259+0000 7f1cb0f74200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:16:16.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:16.541+0000 7f1cb0f74200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:16:16.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:16.733+0000 7f1cb0f74200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:16:17.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:17.082+0000 7f1cb0f74200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:16:18.544 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:18.114+0000 7f1cb0f74200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:16:18.545 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:18.287+0000 7f1cb0f74200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:16:18.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:18.541+0000 7f1cb0f74200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:16:18.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:18.688+0000 7f1cb0f74200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:16:19.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:18.967+0000 7f1cb0f74200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:16:20.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:20.045+0000 7f1cb0f74200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:16:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: Activating manager daemon x 2023-10-18T12:16:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: mgrmap e2: x(active, starting, since 0.00430013s) 2023-10-18T12:16:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T12:16:20.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T12:16:20.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T12:16:20.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:16:20.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T12:16:20.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: Manager daemon x is now available 2023-10-18T12:16:20.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T12:16:20.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T12:16:20.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' 2023-10-18T12:16:20.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' 2023-10-18T12:16:20.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:20 smithi070 ceph-mon[98639]: from='mgr.14100 172.21.15.70:0/3678507222' entity='mgr.x' 2023-10-18T12:16:22.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:22 smithi070 ceph-mon[98639]: mgrmap e3: x(active, since 1.00964s) 2023-10-18T12:16:23.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:23 smithi070 ceph-mon[98639]: mgrmap e4: x(active, since 2s) 2023-10-18T12:16:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:25 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/44053818' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-18T12:16:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:16:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "f94fde2a-6daf-11ee-8db6-212e2dc638e7", 2023-10-18T12:16:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2023-10-18T12:16:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-18T12:16:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-18T12:16:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-18T12:16:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-18T12:16:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-18T12:16:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2023-10-18T12:16:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-18T12:16:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2023-10-18T12:16:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-10-18T12:16:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-18T12:16:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-18T12:16:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-18T12:16:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-18T12:16:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-18T12:16:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-18T12:16:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-18T12:16:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-18T12:16:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-18T12:16:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-18T12:16:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-18T12:16:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-18T12:16:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-18T12:16:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-18T12:16:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-18T12:16:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-18T12:16:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-18T12:16:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-18T12:16:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-18T12:16:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-18T12:16:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-18T12:16:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-18T12:16:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-18T12:16:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-18T12:16:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-18T12:16:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-18T12:16:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-18T12:16:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-18T12:16:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2023-10-18T12:16:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2023-10-18T12:16:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T12:16:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-18T12:16:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T12:16:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2023-10-18T12:16:03.805485+0000", 2023-10-18T12:16:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T12:16:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2023-10-18T12:16:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-18T12:16:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:16:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:mgr is available 2023-10-18T12:16:26.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:26 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1457311214' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-18T12:16:26.625 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:26.626 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2023-10-18T12:16:26.626 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:26.627 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-18T12:16:26.627 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-18T12:16:26.627 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-18T12:16:26.627 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-18T12:16:26.627 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-18T12:16:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-18T12:16:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2023-10-18T12:16:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2023-10-18T12:16:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-18T12:16:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-18T12:16:26.629 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-18T12:16:26.629 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-18T12:16:26.629 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-18T12:16:26.629 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling cephadm module... 2023-10-18T12:16:27.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:27 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2170951298' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-18T12:16:27.657 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:27 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: ignoring --setuser ceph since I am not root 2023-10-18T12:16:27.657 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:27 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: ignoring --setgroup ceph since I am not root 2023-10-18T12:16:28.024 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:27 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:27.654+0000 7fbb665eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:16:28.025 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:27 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:27.758+0000 7fbb665eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:16:28.025 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:27 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:27.900+0000 7fbb665eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:16:28.025 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:28 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:28.022+0000 7fbb665eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:16:28.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:28 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:28.258+0000 7fbb665eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:16:28.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:28 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2170951298' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-18T12:16:28.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:28 smithi070 ceph-mon[98639]: mgrmap e5: x(active, since 7s) 2023-10-18T12:16:29.018 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:28 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:28.727+0000 7fbb665eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:16:29.019 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:28 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:28.899+0000 7fbb665eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:16:29.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:16:29.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-18T12:16:29.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-18T12:16:29.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-18T12:16:29.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-18T12:16:29.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:16:29.106 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2023-10-18T12:16:29.106 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 5... 2023-10-18T12:16:29.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:29 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:29.017+0000 7fbb665eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:16:29.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:29 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:29.149+0000 7fbb665eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:16:29.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:29 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/833336070' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-18T12:16:29.720 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:29 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:29.716+0000 7fbb665eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:16:30.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:30 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:30.561+0000 7fbb665eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:16:30.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:30 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:30.666+0000 7fbb665eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:16:31.958 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:31 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:31.613+0000 7fbb665eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:16:31.958 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:31 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:31.731+0000 7fbb665eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:16:32.318 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:31 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:31.955+0000 7fbb665eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:16:32.595 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:32 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:32.315+0000 7fbb665eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:16:32.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:32 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:32.593+0000 7fbb665eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:16:32.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:32 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:32.783+0000 7fbb665eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:16:33.604 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:33 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:33.128+0000 7fbb665eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:16:34.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:34 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:34.050+0000 7fbb665eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:16:34.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:34 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:34.221+0000 7fbb665eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:16:34.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:34 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:34.469+0000 7fbb665eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:16:34.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:34 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:34.614+0000 7fbb665eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:16:35.357 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:34 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:34.890+0000 7fbb665eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:16:36.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:35.955+0000 7fbb665eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:16:36.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: Active manager daemon x restarted 2023-10-18T12:16:36.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: Activating manager daemon x 2023-10-18T12:16:36.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: osdmap e2: 0 total, 0 up, 0 in 2023-10-18T12:16:36.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: mgrmap e6: x(active, starting, since 0.00372416s) 2023-10-18T12:16:36.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:16:36.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T12:16:36.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T12:16:36.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T12:16:36.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T12:16:36.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: Manager daemon x is now available 2023-10-18T12:16:36.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:36.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:36.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:16:36.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:16:36.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:36 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:16:37.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:37 smithi070 ceph-mon[98639]: Found migration_current of "None". Setting to last migration. 2023-10-18T12:16:37.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:37 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T12:16:37.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:37 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T12:16:37.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:37 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:37.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:37 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:37.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:37 smithi070 ceph-mon[98639]: mgrmap e7: x(active, since 1.00692s) 2023-10-18T12:16:37.500 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:16:37.501 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-18T12:16:37.501 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-18T12:16:37.501 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:16:37.501 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 5 is available 2023-10-18T12:16:37.502 INFO:teuthology.orchestra.run.smithi070.stdout:Setting orchestrator backend to cephadm... 2023-10-18T12:16:38.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:38 smithi070 ceph-mon[98639]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-18T12:16:38.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:38 smithi070 ceph-mon[98639]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-18T12:16:38.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:38 smithi070 ceph-mon[98639]: [18/Oct/2023:12:16:37] ENGINE Bus STARTING 2023-10-18T12:16:38.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:38 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:16:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: [18/Oct/2023:12:16:37] ENGINE Serving on https://172.21.15.70:7150 2023-10-18T12:16:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: [18/Oct/2023:12:16:37] ENGINE Serving on http://172.21.15.70:8765 2023-10-18T12:16:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: [18/Oct/2023:12:16:37] ENGINE Bus STARTED 2023-10-18T12:16:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: mgrmap e8: x(active, since 2s) 2023-10-18T12:16:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:39.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:39 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:16:39.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-18T12:16:39.856 INFO:teuthology.orchestra.run.smithi070.stdout:Generating ssh key... 2023-10-18T12:16:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-mon[98639]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:41.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: Generating public/private rsa key pair. 2023-10-18T12:16:41.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: Your identification has been saved in /tmp/tmpafitcs37/key. 2023-10-18T12:16:41.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: Your public key has been saved in /tmp/tmpafitcs37/key.pub. 2023-10-18T12:16:41.106 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: The key fingerprint is: 2023-10-18T12:16:41.106 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: SHA256:tQuGZ5F84Ge86MsnWk+bbPdl1ODy37jkyQFf+p5Rm1o ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:41.107 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: The key's randomart image is: 2023-10-18T12:16:41.107 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: +---[RSA 3072]----+ 2023-10-18T12:16:41.107 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | . | 2023-10-18T12:16:41.108 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | o + | 2023-10-18T12:16:41.108 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | = * . | 2023-10-18T12:16:41.108 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | . B o . ..| 2023-10-18T12:16:41.109 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | . S o o . =| 2023-10-18T12:16:41.109 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | = . . = ++| 2023-10-18T12:16:41.110 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | o o *Eo| 2023-10-18T12:16:41.110 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | o.=oo.+oO+| 2023-10-18T12:16:41.110 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: | ..o+=. oB+=| 2023-10-18T12:16:41.111 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: +----[SHA256]-----+ 2023-10-18T12:16:42.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:41 smithi070 ceph-mon[98639]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:42.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:41 smithi070 ceph-mon[98639]: Generating ssh key... 2023-10-18T12:16:42.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:41 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:42.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:41 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:42.418 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:16:42.419 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-18T12:16:42.419 INFO:teuthology.orchestra.run.smithi070.stdout:Adding key to root@localhost authorized_keys... 2023-10-18T12:16:42.421 INFO:teuthology.orchestra.run.smithi070.stdout:Adding host smithi070... 2023-10-18T12:16:43.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:42 smithi070 ceph-mon[98639]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:44.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:43 smithi070 ceph-mon[98639]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:44 smithi070 ceph-mon[98639]: Deploying cephadm binary to smithi070 2023-10-18T12:16:46.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Added host 'smithi070' with addr '172.21.15.70' 2023-10-18T12:16:46.803 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying unmanaged mon service... 2023-10-18T12:16:47.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:47 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:47.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:47 smithi070 ceph-mon[98639]: Added host smithi070 2023-10-18T12:16:47.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:47 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:16:48.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-18T12:16:48.282 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying unmanaged mgr service... 2023-10-18T12:16:48.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:48 smithi070 ceph-mon[98639]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:48.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:48 smithi070 ceph-mon[98639]: Saving service mon spec with placement count:5 2023-10-18T12:16:48.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:48 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:49.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-18T12:16:50.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:50 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:50 smithi070 ceph-mon[98639]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:16:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:50 smithi070 ceph-mon[98639]: Saving service mgr spec with placement count:2 2023-10-18T12:16:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:50 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:50 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:51.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:51 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2354400380' entity='client.admin' 2023-10-18T12:16:52.237 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling the dashboard module... 2023-10-18T12:16:52.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:52 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1052377663' entity='client.admin' 2023-10-18T12:16:52.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:52 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:53.980 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:53 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1759871896' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-18T12:16:53.980 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:53 smithi070 ceph-mon[98639]: from='mgr.14118 172.21.15.70:0/816180445' entity='mgr.x' 2023-10-18T12:16:53.981 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: ignoring --setuser ceph since I am not root 2023-10-18T12:16:53.981 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: ignoring --setgroup ceph since I am not root 2023-10-18T12:16:54.237 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:53.977+0000 7fb247b39200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:16:54.237 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:54.084+0000 7fb247b39200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:16:54.495 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:54.234+0000 7fb247b39200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:16:54.496 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:54.357+0000 7fb247b39200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:16:54.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:54 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1759871896' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-18T12:16:54.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:54 smithi070 ceph-mon[98639]: mgrmap e9: x(active, since 17s) 2023-10-18T12:16:54.751 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:54.609+0000 7fb247b39200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:16:55.284 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:55.104+0000 7fb247b39200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:16:55.546 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:55.281+0000 7fb247b39200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:16:55.546 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:55.415+0000 7fb247b39200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:16:55.547 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:55.543+0000 7fb247b39200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:16:55.610 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:16:55.610 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-18T12:16:55.611 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-18T12:16:55.611 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-18T12:16:55.611 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-18T12:16:55.611 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:16:55.611 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2023-10-18T12:16:55.612 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 9... 2023-10-18T12:16:55.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:16:55 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1947494449' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-18T12:16:56.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:56.116+0000 7fb247b39200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:16:57.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:56.956+0000 7fb247b39200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:16:57.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:57.060+0000 7fb247b39200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:16:58.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:58.008+0000 7fb247b39200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:16:58.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:58.128+0000 7fb247b39200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:16:58.721 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:58.353+0000 7fb247b39200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:16:58.999 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:58.719+0000 7fb247b39200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:16:59.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:58.997+0000 7fb247b39200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:16:59.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:59.188+0000 7fb247b39200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:16:59.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:16:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:16:59.535+0000 7fb247b39200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:17:00.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:00.460+0000 7fb247b39200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:17:00.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:00.633+0000 7fb247b39200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:17:01.308 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:00.884+0000 7fb247b39200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:17:01.308 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:01 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:01.029+0000 7fb247b39200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:17:01.604 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:01 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:01.305+0000 7fb247b39200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:17:02.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:02.380+0000 7fb247b39200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:17:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: Active manager daemon x restarted 2023-10-18T12:17:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: Activating manager daemon x 2023-10-18T12:17:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: osdmap e3: 0 total, 0 up, 0 in 2023-10-18T12:17:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: mgrmap e10: x(active, starting, since 0.00405531s) 2023-10-18T12:17:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T12:17:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T12:17:02.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T12:17:02.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T12:17:02.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: Manager daemon x is now available 2023-10-18T12:17:02.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:02.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:02 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:03.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2023-10-18T12:17:03.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-18T12:17:03.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-18T12:17:03.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2023-10-18T12:17:03.809 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 9 is available 2023-10-18T12:17:03.809 INFO:teuthology.orchestra.run.smithi070.stdout:Generating a dashboard self-signed certificate... 2023-10-18T12:17:03.824 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T12:17:03.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T12:17:03.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:03 smithi070 ceph-mon[98639]: mgrmap e11: x(active, since 1.00728s) 2023-10-18T12:17:04.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:04 smithi070 ceph-mon[98639]: [18/Oct/2023:12:17:03] ENGINE Bus STARTING 2023-10-18T12:17:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:04 smithi070 ceph-mon[98639]: [18/Oct/2023:12:17:03] ENGINE Serving on https://172.21.15.70:7150 2023-10-18T12:17:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:04 smithi070 ceph-mon[98639]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-18T12:17:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:04 smithi070 ceph-mon[98639]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-18T12:17:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:04 smithi070 ceph-mon[98639]: [18/Oct/2023:12:17:03] ENGINE Serving on http://172.21.15.70:8765 2023-10-18T12:17:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:04 smithi070 ceph-mon[98639]: [18/Oct/2023:12:17:03] ENGINE Bus STARTED 2023-10-18T12:17:05.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-18T12:17:05.300 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial admin user... 2023-10-18T12:17:05.582 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:05 smithi070 ceph-mon[98639]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:05.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:05.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:05.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:05.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:05 smithi070 ceph-mon[98639]: mgrmap e12: x(active, since 2s) 2023-10-18T12:17:06.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$M67kyUqWse1SphJc9cKER.2VA8bl5ri0bp1YtPWVcedOK3oIJpSxy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1697631426, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-18T12:17:06.777 INFO:teuthology.orchestra.run.smithi070.stdout:Fetching dashboard port number... 2023-10-18T12:17:07.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:07 smithi070 ceph-mon[98639]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:07.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:07 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:07.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:07 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:08.050 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 8443 2023-10-18T12:17:08.088 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T12:17:08.089 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2023-10-18T12:17:08.106 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T12:17:08.106 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2023-10-18T12:17:08.107 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2023-10-18T12:17:08.107 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-18T12:17:08.109 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph Dashboard is now available at: 2023-10-18T12:17:08.109 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:08.110 INFO:teuthology.orchestra.run.smithi070.stdout: URL: https://smithi070.front.sepia.ceph.com:8443/ 2023-10-18T12:17:08.110 INFO:teuthology.orchestra.run.smithi070.stdout: User: admin 2023-10-18T12:17:08.111 INFO:teuthology.orchestra.run.smithi070.stdout: Password: vysd40pq5h 2023-10-18T12:17:08.111 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:08.112 INFO:teuthology.orchestra.run.smithi070.stdout:Saving cluster configuration to /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config directory 2023-10-18T12:17:08.112 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling autotune for osd_memory_target 2023-10-18T12:17:08.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:08 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/938933934' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-18T12:17:10.839 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:10 smithi070 ceph-mon[98639]: mgrmap e13: x(active, since 6s) 2023-10-18T12:17:10.868 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-18T12:17:10.869 INFO:teuthology.orchestra.run.smithi070.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-18T12:17:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.870 INFO:teuthology.orchestra.run.smithi070.stdout:Or, if you are only running a single cluster on this host: 2023-10-18T12:17:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-18T12:17:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.871 INFO:teuthology.orchestra.run.smithi070.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-18T12:17:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: ceph telemetry on 2023-10-18T12:17:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.871 INFO:teuthology.orchestra.run.smithi070.stdout:For more information see: 2023-10-18T12:17:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-18T12:17:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:10.872 INFO:teuthology.orchestra.run.smithi070.stdout:Bootstrap complete. 2023-10-18T12:17:10.928 INFO:tasks.cephadm:Fetching config... 2023-10-18T12:17:10.929 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:17:10.929 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-18T12:17:10.973 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-18T12:17:10.973 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:17:10.974 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-18T12:17:11.060 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-18T12:17:11.060 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:17:11.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-18T12:17:11.169 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-18T12:17:11.170 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:17:11.170 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-18T12:17:11.252 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-18T12:17:11.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-18T12:17:11.396 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:11.427 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-18T12:17:11.474 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:11.487 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-18T12:17:11.532 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7M4Paofce7qdeWfkX5UBdm9PTpTP/WJOEeiDgQTtQ6YJ40CAhoA+aT9mE4uQ5dJZK+cxFuir6QmAXvwTzuNDSkezlWGcpRoOq23sax91/Xy5XwWViHRf/wDc+ORL+B294yye2izp0e5DXJ8K5McRniaOxnVVa2HE0lf6ohRw705cdMMpyJSG8gXXvWrWzwQpREw1dHelXzNDb16YYyhbWqTV93gPCSQ0k3demuWTZgfNZ97F17WmvjMJSJvDHbUEapOsQyr5YXrJRgIbXAwrpvOfAJNJrYs4kKTmsHp15zbfSrNuKCM3bIFLpuRHKk2WhE+iEQQ4o71OOz9ZkXPTXakTOtnKxnSh2L142qgG4VdGLjZpjL9yIThnxrunnLVGLOb1IkV1tBzmNEbUb6fmIdwNTDToq52vvPJrFGhR6XsE1/SMQzvT/uUbjHsaEIYZ6ANg0eesuPfzWPM4E2ew7i/Z6D+Hd2V9/yaCdc400Zi9R2y/um+Ooy9DZ+VItdDc= ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:11.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-18T12:17:11.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:11 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1787478134' entity='client.admin' 2023-10-18T12:17:12.285 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:14.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:14.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/261868392' entity='client.admin' 2023-10-18T12:17:14.829 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-18T12:17:14.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-18T12:17:15.443 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:15 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:15.443 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:15 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:18.075 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi152 2023-10-18T12:17:18.076 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:17:18.076 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.conf 2023-10-18T12:17:18.096 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:17:18.097 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:18.156 INFO:tasks.cephadm:Adding host smithi152 to orchestrator... 2023-10-18T12:17:18.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi152 2023-10-18T12:17:18.916 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:19.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:18 smithi070 ceph-mon[98639]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:19.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:18 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:19.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:18 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:19.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:18 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:19.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:18 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:19.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:18 smithi070 ceph-mon[98639]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:17:19.777 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:19 smithi070 ceph-mon[98639]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:19.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:19 smithi070 ceph-mon[98639]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:21.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:20 smithi070 ceph-mon[98639]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:20 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:20 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:20 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:22.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:21 smithi070 ceph-mon[98639]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:23.042 INFO:teuthology.orchestra.run.smithi070.stdout:Added host 'smithi152' with addr '172.21.15.152' 2023-10-18T12:17:23.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:22 smithi070 ceph-mon[98639]: Deploying cephadm binary to smithi152 2023-10-18T12:17:23.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-18T12:17:23.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:23 smithi070 ceph-mon[98639]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:23.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:23 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:23.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:23 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:23.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:23 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:24.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:24 smithi070 ceph-mon[98639]: Added host smithi152 2023-10-18T12:17:24.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:24 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:25.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:25 smithi070 ceph-mon[98639]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:25.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:25 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:26.032 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:26.033 INFO:teuthology.orchestra.run.smithi070.stdout:[{"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}] 2023-10-18T12:17:26.512 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2023-10-18T12:17:26.512 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:17:26.513 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2023-10-18T12:17:26.533 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:17:26.533 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:26.593 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2023-10-18T12:17:26.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi186 2023-10-18T12:17:27.322 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:27.848 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:27 smithi070 ceph-mon[98639]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:17:27.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:27 smithi070 ceph-mon[98639]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:29.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:29.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:29.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:29.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:17:29.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:29.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:29.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:29 smithi070 ceph-mon[98639]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:30.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:30 smithi070 ceph-mon[98639]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:30 smithi070 ceph-mon[98639]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:30 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:30 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:30 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:30.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:30 smithi070 ceph-mon[98639]: Deploying cephadm binary to smithi186 2023-10-18T12:17:31.406 INFO:teuthology.orchestra.run.smithi070.stdout:Added host 'smithi186' with addr '172.21.15.186' 2023-10-18T12:17:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:31 smithi070 ceph-mon[98639]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:31.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-18T12:17:32.634 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:32.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:32 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:32.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:32 smithi070 ceph-mon[98639]: Added host smithi186 2023-10-18T12:17:32.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:32 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:32.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:32 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:33.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:33 smithi070 ceph-mon[98639]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:33.779 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:33 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:34.225 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:17:34.226 INFO:teuthology.orchestra.run.smithi070.stdout:[{"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2023-10-18T12:17:34.713 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-18T12:17:34.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd crush tunables default 2023-10-18T12:17:35.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:34 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:36.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:35 smithi070 ceph-mon[98639]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:17:36.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:35 smithi070 ceph-mon[98639]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:37.839 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2023-10-18T12:17:38.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:38.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:38.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:38.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:38.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/985804014' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-18T12:17:38.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:37 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:38.428 INFO:tasks.cephadm:Adding mon.a on smithi070 2023-10-18T12:17:38.429 INFO:tasks.cephadm:Adding mon.b on smithi152 2023-10-18T12:17:38.429 INFO:tasks.cephadm:Adding mon.c on smithi186 2023-10-18T12:17:38.429 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch apply mon '3;smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c' 2023-10-18T12:17:39.016 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:39.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:38 smithi070 ceph-mon[98639]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:17:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:38 smithi070 ceph-mon[98639]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:38 smithi070 ceph-mon[98639]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:38 smithi070 ceph-mon[98639]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/985804014' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-18T12:17:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:38 smithi070 ceph-mon[98639]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:40.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:39 smithi070 ceph-mon[98639]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:40.294 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2023-10-18T12:17:40.800 DEBUG:teuthology.orchestra.run.smithi152:mon.b> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.b.service 2023-10-18T12:17:40.804 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.c.service 2023-10-18T12:17:40.806 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-18T12:17:40.807 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-18T12:17:40.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:17:40.838 INFO:journalctl@ceph.mon.b.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:17:41.375 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: Saving service mon spec with placement smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c;count:3 2023-10-18T12:17:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:17:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:41.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: Deploying daemon mon.c on smithi186 2023-10-18T12:17:41.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:41 smithi070 ceph-mon[98639]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:43.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:43 smithi186 systemd[1]: Starting Ceph mon.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:17:43.558 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:17:43.559 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","modified":"2023-10-18T12:15:59.082484Z","created":"2023-10-18T12:15:59.082484Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-18T12:17:43.560 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2023-10-18T12:17:43.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:43 smithi070 ceph-mon[98639]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:43.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:43 smithi186 podman[135182]: 2023-10-18T12:17:44.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: set uid:gid to 167:167 (ceph:ceph) 2023-10-18T12:17:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev), process ceph-mon, pid 7 2023-10-18T12:17:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pidfile_write: ignore empty --pid-file 2023-10-18T12:17:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 bash[135182]: 7d107bdaa63c1844f9b3394e19776ee181f29e21f2624ae9ab5efb2632ccffd4 2023-10-18T12:17:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: load: jerasure load: lrc 2023-10-18T12:17:44.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: RocksDB version: 7.9.2 2023-10-18T12:17:44.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Git sha 0 2023-10-18T12:17:44.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Compile date 2023-10-16 22:08:19 2023-10-18T12:17:44.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: DB SUMMARY 2023-10-18T12:17:44.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: DB Session ID: 5DOX09R1NKMU8Q5VK8L3 2023-10-18T12:17:44.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: CURRENT file: CURRENT 2023-10-18T12:17:44.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: IDENTITY file: IDENTITY 2023-10-18T12:17:44.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-18T12:17:44.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-18T12:17:44.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-18T12:17:44.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.error_if_exists: 0 2023-10-18T12:17:44.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.create_if_missing: 0 2023-10-18T12:17:44.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.paranoid_checks: 1 2023-10-18T12:17:44.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-18T12:17:44.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-18T12:17:44.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-18T12:17:44.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.env: 0x55abd287ada0 2023-10-18T12:17:44.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.fs: PosixFileSystem 2023-10-18T12:17:44.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.info_log: 0x55abd46b7660 2023-10-18T12:17:44.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_file_opening_threads: 16 2023-10-18T12:17:44.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.statistics: (nil) 2023-10-18T12:17:44.474 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.use_fsync: 0 2023-10-18T12:17:44.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_log_file_size: 0 2023-10-18T12:17:44.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-18T12:17:44.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-18T12:17:44.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.keep_log_file_num: 1000 2023-10-18T12:17:44.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.recycle_log_file_num: 0 2023-10-18T12:17:44.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.allow_fallocate: 1 2023-10-18T12:17:44.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.allow_mmap_reads: 0 2023-10-18T12:17:44.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.allow_mmap_writes: 0 2023-10-18T12:17:44.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.use_direct_reads: 0 2023-10-18T12:17:44.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-18T12:17:44.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.create_missing_column_families: 0 2023-10-18T12:17:44.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.db_log_dir: 2023-10-18T12:17:44.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.wal_dir: 2023-10-18T12:17:44.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-18T12:17:44.477 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-18T12:17:44.478 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-18T12:17:44.478 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-18T12:17:44.478 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-18T12:17:44.478 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-18T12:17:44.478 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.advise_random_on_open: 1 2023-10-18T12:17:44.478 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.db_write_buffer_size: 0 2023-10-18T12:17:44.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.write_buffer_manager: 0x55abd466f360 2023-10-18T12:17:44.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-18T12:17:44.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-18T12:17:44.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-18T12:17:44.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.rate_limiter: (nil) 2023-10-18T12:17:44.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-18T12:17:44.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.wal_recovery_mode: 2 2023-10-18T12:17:44.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.enable_thread_tracking: 0 2023-10-18T12:17:44.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.enable_pipelined_write: 0 2023-10-18T12:17:44.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.unordered_write: 0 2023-10-18T12:17:44.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-18T12:17:44.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-18T12:17:44.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-18T12:17:44.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-18T12:17:44.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.row_cache: None 2023-10-18T12:17:44.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.wal_filter: None 2023-10-18T12:17:44.482 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-18T12:17:44.482 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.allow_ingest_behind: 0 2023-10-18T12:17:44.482 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.two_write_queues: 0 2023-10-18T12:17:44.482 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.manual_wal_flush: 0 2023-10-18T12:17:44.482 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.wal_compression: 0 2023-10-18T12:17:44.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.atomic_flush: 0 2023-10-18T12:17:44.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-18T12:17:44.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-18T12:17:44.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-18T12:17:44.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.log_readahead_size: 0 2023-10-18T12:17:44.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-18T12:17:44.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.best_efforts_recovery: 0 2023-10-18T12:17:44.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-18T12:17:44.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-18T12:17:44.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.allow_data_in_errors: 0 2023-10-18T12:17:44.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.db_host_id: __hostname__ 2023-10-18T12:17:44.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-18T12:17:44.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_background_jobs: 2 2023-10-18T12:17:44.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_background_compactions: -1 2023-10-18T12:17:44.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_subcompactions: 1 2023-10-18T12:17:44.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-18T12:17:44.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-18T12:17:44.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-18T12:17:44.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_total_wal_size: 0 2023-10-18T12:17:44.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-18T12:17:44.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-18T12:17:44.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-18T12:17:44.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-18T12:17:44.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_open_files: -1 2023-10-18T12:17:44.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bytes_per_sync: 0 2023-10-18T12:17:44.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-18T12:17:44.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-18T12:17:44.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_readahead_size: 0 2023-10-18T12:17:44.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_background_flushes: -1 2023-10-18T12:17:44.490 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Compression algorithms supported: 2023-10-18T12:17:44.490 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-18T12:17:44.490 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kZSTD supported: 0 2023-10-18T12:17:44.490 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kXpressCompression supported: 0 2023-10-18T12:17:44.490 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kLZ4HCCompression supported: 1 2023-10-18T12:17:44.491 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kZlibCompression supported: 1 2023-10-18T12:17:44.491 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kSnappyCompression supported: 1 2023-10-18T12:17:44.491 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kLZ4Compression supported: 1 2023-10-18T12:17:44.491 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: kBZip2Compression supported: 0 2023-10-18T12:17:44.491 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-18T12:17:44.492 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-18T12:17:44.492 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-10-18T12:17:44.492 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-18T12:17:44.492 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-18T12:17:44.492 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.merge_operator: 2023-10-18T12:17:44.492 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_filter: None 2023-10-18T12:17:44.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_filter_factory: None 2023-10-18T12:17:44.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.sst_partitioner_factory: None 2023-10-18T12:17:44.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-18T12:17:44.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-18T12:17:44.493 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55abd46b7760) 2023-10-18T12:17:44.494 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks: 1 2023-10-18T12:17:44.494 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-18T12:17:44.494 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-18T12:17:44.494 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_top_level_index_and_filter: 1 2023-10-18T12:17:44.494 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_type: 0 2023-10-18T12:17:44.495 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_index_type: 0 2023-10-18T12:17:44.495 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_shortening: 1 2023-10-18T12:17:44.495 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-18T12:17:44.495 INFO:journalctl@ceph.mon.c.smithi186.stdout: checksum: 4 2023-10-18T12:17:44.495 INFO:journalctl@ceph.mon.c.smithi186.stdout: no_block_cache: 0 2023-10-18T12:17:44.495 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache: 0x55abd470d1f0 2023-10-18T12:17:44.496 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_name: BinnedLRUCache 2023-10-18T12:17:44.496 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_options: 2023-10-18T12:17:44.496 INFO:journalctl@ceph.mon.c.smithi186.stdout: capacity : 536870912 2023-10-18T12:17:44.496 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_shard_bits : 4 2023-10-18T12:17:44.496 INFO:journalctl@ceph.mon.c.smithi186.stdout: strict_capacity_limit : 0 2023-10-18T12:17:44.497 INFO:journalctl@ceph.mon.c.smithi186.stdout: high_pri_pool_ratio: 0.000 2023-10-18T12:17:44.497 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_compressed: (nil) 2023-10-18T12:17:44.497 INFO:journalctl@ceph.mon.c.smithi186.stdout: persistent_cache: (nil) 2023-10-18T12:17:44.497 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size: 4096 2023-10-18T12:17:44.497 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size_deviation: 10 2023-10-18T12:17:44.498 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_restart_interval: 16 2023-10-18T12:17:44.498 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_block_restart_interval: 1 2023-10-18T12:17:44.498 INFO:journalctl@ceph.mon.c.smithi186.stdout: metadata_block_size: 4096 2023-10-18T12:17:44.498 INFO:journalctl@ceph.mon.c.smithi186.stdout: partition_filters: 0 2023-10-18T12:17:44.498 INFO:journalctl@ceph.mon.c.smithi186.stdout: use_delta_encoding: 1 2023-10-18T12:17:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout: filter_policy: bloomfilter 2023-10-18T12:17:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout: whole_key_filtering: 1 2023-10-18T12:17:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout: verify_compression: 0 2023-10-18T12:17:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout: read_amp_bytes_per_bit: 0 2023-10-18T12:17:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout: format_version: 5 2023-10-18T12:17:44.499 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_index_compression: 1 2023-10-18T12:17:44.500 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_align: 0 2023-10-18T12:17:44.500 INFO:journalctl@ceph.mon.c.smithi186.stdout: max_auto_readahead_size: 262144 2023-10-18T12:17:44.500 INFO:journalctl@ceph.mon.c.smithi186.stdout: prepopulate_block_cache: 0 2023-10-18T12:17:44.500 INFO:journalctl@ceph.mon.c.smithi186.stdout: initial_auto_readahead_size: 8192 2023-10-18T12:17:44.500 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_file_reads_for_auto_readahead: 2 2023-10-18T12:17:44.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.write_buffer_size: 33554432 2023-10-18T12:17:44.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_write_buffer_number: 2 2023-10-18T12:17:44.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression: NoCompression 2023-10-18T12:17:44.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression: Disabled 2023-10-18T12:17:44.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.prefix_extractor: nullptr 2023-10-18T12:17:44.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-18T12:17:44.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.num_levels: 7 2023-10-18T12:17:44.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-18T12:17:44.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-18T12:17:44.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-18T12:17:44.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-18T12:17:44.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-18T12:17:44.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-18T12:17:44.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-18T12:17:44.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-18T12:17:44.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-18T12:17:44.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-18T12:17:44.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-18T12:17:44.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-18T12:17:44.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-18T12:17:44.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.level: 32767 2023-10-18T12:17:44.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.strategy: 0 2023-10-18T12:17:44.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-18T12:17:44.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-18T12:17:44.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-18T12:17:44.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-18T12:17:44.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.enabled: false 2023-10-18T12:17:44.506 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-18T12:17:44.506 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-18T12:17:44.506 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-18T12:17:44.506 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-18T12:17:44.506 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.target_file_size_base: 67108864 2023-10-18T12:17:44.507 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-18T12:17:44.507 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-18T12:17:44.507 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-18T12:17:44.507 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-18T12:17:44.507 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-18T12:17:44.507 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-18T12:17:44.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-18T12:17:44.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-18T12:17:44.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-18T12:17:44.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-18T12:17:44.508 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-18T12:17:44.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-18T12:17:44.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-18T12:17:44.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-18T12:17:44.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.arena_block_size: 1048576 2023-10-18T12:17:44.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-18T12:17:44.510 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-18T12:17:44.510 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.disable_auto_compactions: 0 2023-10-18T12:17:44.510 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-18T12:17:44.510 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-18T12:17:44.510 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-18T12:17:44.511 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-18T12:17:44.511 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-18T12:17:44.511 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-18T12:17:44.511 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-18T12:17:44.511 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-18T12:17:44.511 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-18T12:17:44.512 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-18T12:17:44.512 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.table_properties_collectors: 2023-10-18T12:17:44.512 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.inplace_update_support: 0 2023-10-18T12:17:44.512 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-18T12:17:44.512 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-18T12:17:44.513 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-18T12:17:44.513 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-18T12:17:44.513 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.bloom_locality: 0 2023-10-18T12:17:44.513 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.max_successive_merges: 0 2023-10-18T12:17:44.513 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-18T12:17:44.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.paranoid_file_checks: 0 2023-10-18T12:17:44.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.force_consistency_checks: 1 2023-10-18T12:17:44.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.report_bg_io_stats: 0 2023-10-18T12:17:44.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.ttl: 2592000 2023-10-18T12:17:44.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-18T12:17:44.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-18T12:17:44.515 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-18T12:17:44.515 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.enable_blob_files: false 2023-10-18T12:17:44.515 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.min_blob_size: 0 2023-10-18T12:17:44.515 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.blob_file_size: 268435456 2023-10-18T12:17:44.515 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-18T12:17:44.516 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-18T12:17:44.516 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-18T12:17:44.516 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-18T12:17:44.516 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-18T12:17:44.516 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.blob_file_starting_level: 0 2023-10-18T12:17:44.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-18T12:17:44.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-18T12:17:44.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-18T12:17:44.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e6deb316-b5d4-4a61-a6d4-2054de7962b5 2023-10-18T12:17:44.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631464115492, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-18T12:17:44.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-18T12:17:44.518 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631464115914, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697631464, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e6deb316-b5d4-4a61-a6d4-2054de7962b5", "db_session_id": "5DOX09R1NKMU8Q5VK8L3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-18T12:17:44.518 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631464115981, "job": 1, "event": "recovery_finished"} 2023-10-18T12:17:44.518 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-18T12:17:44.518 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-18T12:17:44.518 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55abd47aa000 2023-10-18T12:17:44.519 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: DB pointer 0x55abd4796000 2023-10-18T12:17:44.519 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-18T12:17:44.519 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-18T12:17:44.519 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** DB Stats ** 2023-10-18T12:17:44.519 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T12:17:44.520 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-18T12:17:44.520 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T12:17:44.520 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T12:17:44.520 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-18T12:17:44.520 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T12:17:44.521 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T12:17:44.521 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-10-18T12:17:44.521 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2023-10-18T12:17:44.521 INFO:journalctl@ceph.mon.c.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T12:17:44.521 INFO:journalctl@ceph.mon.c.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-18T12:17:44.522 INFO:journalctl@ceph.mon.c.smithi186.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T12:17:44.522 INFO:journalctl@ceph.mon.c.smithi186.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T12:17:44.522 INFO:journalctl@ceph.mon.c.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T12:17:44.522 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-10-18T12:17:44.522 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2023-10-18T12:17:44.522 INFO:journalctl@ceph.mon.c.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T12:17:44.523 INFO:journalctl@ceph.mon.c.smithi186.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-18T12:17:44.523 INFO:journalctl@ceph.mon.c.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T12:17:44.523 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-10-18T12:17:44.523 INFO:journalctl@ceph.mon.c.smithi186.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-18T12:17:44.523 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-10-18T12:17:44.524 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T12:17:44.524 INFO:journalctl@ceph.mon.c.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-18T12:17:44.524 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-18T12:17:44.524 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-18T12:17:44.524 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-18T12:17:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-18T12:17:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T12:17:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T12:17:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-18T12:17:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache BinnedLRUCache@0x55abd470d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-18T12:17:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-18T12:17:44.526 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-10-18T12:17:44.526 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-18T12:17:44.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-18T12:17:44.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: using public_addr v2:172.21.15.186:0/0 -> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2023-10-18T12:17:44.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: starting mon.c rank -1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:44.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(???) e0 preinit fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:44.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).mds e1 new map 2023-10-18T12:17:44.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-18T12:17:44.527 INFO:journalctl@ceph.mon.c.smithi186.stdout: e1 2023-10-18T12:17:44.527 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T12:17:44.528 INFO:journalctl@ceph.mon.c.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T12:17:44.528 INFO:journalctl@ceph.mon.c.smithi186.stdout: legacy client fscid: -1 2023-10-18T12:17:44.528 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-10-18T12:17:44.528 INFO:journalctl@ceph.mon.c.smithi186.stdout: No filesystems configured 2023-10-18T12:17:44.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-18T12:17:44.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-18T12:17:44.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-18T12:17:44.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-18T12:17:44.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-18T12:17:44.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-18T12:17:44.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-18T12:17:44.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:17:44.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:17:44.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:17:44.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:17:44.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:44.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:44.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:44.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:17:44.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.533 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:44.533 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:44.533 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:44.533 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:44.533 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.534 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.534 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.534 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Deploying cephadm binary to smithi186 2023-10-18T12:17:44.534 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.534 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Added host smithi186 2023-10-18T12:17:44.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:44.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:17:44.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:44.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:44.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:44.537 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/985804014' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-18T12:17:44.538 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.538 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.538 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.538 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:17:44.538 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:44.539 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:44.539 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:44.539 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/985804014' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-18T12:17:44.539 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:44.539 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.540 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:44.540 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Saving service mon spec with placement smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c;count:3 2023-10-18T12:17:44.540 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.540 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:44.540 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:44.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:44.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:44.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:17:44.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:44.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: Deploying daemon mon.c on smithi186 2023-10-18T12:17:44.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:44.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 ceph-mon[135253]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-18T12:17:44.542 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:44 smithi186 systemd[1]: Started Ceph mon.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:17:45.056 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-18T12:17:45.057 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-18T12:17:45.630 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:17:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: mon.a calling monitor election 2023-10-18T12:17:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: mon.c calling monitor election 2023-10-18T12:17:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T12:17:49.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: monmap e2: 2 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:17:49.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: fsmap 2023-10-18T12:17:49.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:49.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: mgrmap e13: x(active, since 46s) 2023-10-18T12:17:49.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: overall HEALTH_OK 2023-10-18T12:17:49.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:49.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:49.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:49.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:17:49.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:49 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:49.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: mon.a calling monitor election 2023-10-18T12:17:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: mon.c calling monitor election 2023-10-18T12:17:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:49.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:49.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T12:17:49.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: monmap e2: 2 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:17:49.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: fsmap 2023-10-18T12:17:49.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:49.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: mgrmap e13: x(active, since 46s) 2023-10-18T12:17:49.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: overall HEALTH_OK 2023-10-18T12:17:49.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:49.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:49.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:49.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:17:49.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:49 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:50.087 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:17:50.087 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","modified":"2023-10-18T12:17:44.147552Z","created":"2023-10-18T12:15:59.082484Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-18T12:17:50.089 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2023-10-18T12:17:50.422 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:50 smithi186 ceph-mon[135253]: Deploying daemon mon.b on smithi152 2023-10-18T12:17:50.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:50 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/3662346108' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T12:17:50.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:50 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:50.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:50 smithi070 ceph-mon[98639]: Deploying daemon mon.b on smithi152 2023-10-18T12:17:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:50 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/3662346108' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T12:17:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:50 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:51.479 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-18T12:17:51.479 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-18T12:17:51.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:51 smithi186 ceph-mon[135253]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:51 smithi070 ceph-mon[98639]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:51.605 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:17:51 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:17:51.147+0000 7fb2011c1700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-18T12:17:52.043 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:17:52.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:51 smithi152 systemd[1]: Starting Ceph mon.b for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:17:52.492 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 podman[134726]: 2023-10-18T12:17:52.493 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: set uid:gid to 167:167 (ceph:ceph) 2023-10-18T12:17:52.493 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev), process ceph-mon, pid 8 2023-10-18T12:17:52.493 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pidfile_write: ignore empty --pid-file 2023-10-18T12:17:52.493 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: load: jerasure load: lrc 2023-10-18T12:17:52.493 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: RocksDB version: 7.9.2 2023-10-18T12:17:52.494 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Git sha 0 2023-10-18T12:17:52.494 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Compile date 2023-10-16 22:08:19 2023-10-18T12:17:52.494 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: DB SUMMARY 2023-10-18T12:17:52.494 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: DB Session ID: JFM3WZZA92QYEACEKAZC 2023-10-18T12:17:52.494 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: CURRENT file: CURRENT 2023-10-18T12:17:52.495 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: IDENTITY file: IDENTITY 2023-10-18T12:17:52.495 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-18T12:17:52.495 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-18T12:17:52.495 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-18T12:17:52.495 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.error_if_exists: 0 2023-10-18T12:17:52.496 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.create_if_missing: 0 2023-10-18T12:17:52.496 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.paranoid_checks: 1 2023-10-18T12:17:52.496 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-18T12:17:52.496 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-18T12:17:52.496 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-18T12:17:52.497 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.env: 0x55d617982da0 2023-10-18T12:17:52.497 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.fs: PosixFileSystem 2023-10-18T12:17:52.497 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.info_log: 0x55d61960f660 2023-10-18T12:17:52.497 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_file_opening_threads: 16 2023-10-18T12:17:52.497 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.statistics: (nil) 2023-10-18T12:17:52.498 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.use_fsync: 0 2023-10-18T12:17:52.498 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_log_file_size: 0 2023-10-18T12:17:52.498 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-18T12:17:52.498 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-18T12:17:52.498 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.keep_log_file_num: 1000 2023-10-18T12:17:52.498 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.recycle_log_file_num: 0 2023-10-18T12:17:52.499 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.allow_fallocate: 1 2023-10-18T12:17:52.499 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.allow_mmap_reads: 0 2023-10-18T12:17:52.499 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.allow_mmap_writes: 0 2023-10-18T12:17:52.499 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.use_direct_reads: 0 2023-10-18T12:17:52.499 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-18T12:17:52.500 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.create_missing_column_families: 0 2023-10-18T12:17:52.500 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.db_log_dir: 2023-10-18T12:17:52.500 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.wal_dir: 2023-10-18T12:17:52.500 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-18T12:17:52.501 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-18T12:17:52.501 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-18T12:17:52.501 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-18T12:17:52.501 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-18T12:17:52.501 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-18T12:17:52.502 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.advise_random_on_open: 1 2023-10-18T12:17:52.502 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.db_write_buffer_size: 0 2023-10-18T12:17:52.502 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.write_buffer_manager: 0x55d6195c7360 2023-10-18T12:17:52.502 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-18T12:17:52.502 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-18T12:17:52.503 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-18T12:17:52.503 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.rate_limiter: (nil) 2023-10-18T12:17:52.503 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-18T12:17:52.503 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.wal_recovery_mode: 2 2023-10-18T12:17:52.503 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.enable_thread_tracking: 0 2023-10-18T12:17:52.503 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.enable_pipelined_write: 0 2023-10-18T12:17:52.504 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.unordered_write: 0 2023-10-18T12:17:52.504 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-18T12:17:52.504 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-18T12:17:52.504 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-18T12:17:52.504 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-18T12:17:52.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.row_cache: None 2023-10-18T12:17:52.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.wal_filter: None 2023-10-18T12:17:52.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-18T12:17:52.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.allow_ingest_behind: 0 2023-10-18T12:17:52.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.two_write_queues: 0 2023-10-18T12:17:52.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.manual_wal_flush: 0 2023-10-18T12:17:52.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.wal_compression: 0 2023-10-18T12:17:52.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.atomic_flush: 0 2023-10-18T12:17:52.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-18T12:17:52.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-18T12:17:52.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-18T12:17:52.507 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.log_readahead_size: 0 2023-10-18T12:17:52.507 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-18T12:17:52.507 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.best_efforts_recovery: 0 2023-10-18T12:17:52.507 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-18T12:17:52.507 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-18T12:17:52.508 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.allow_data_in_errors: 0 2023-10-18T12:17:52.508 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.db_host_id: __hostname__ 2023-10-18T12:17:52.509 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-18T12:17:52.510 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_background_jobs: 2 2023-10-18T12:17:52.510 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_background_compactions: -1 2023-10-18T12:17:52.510 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_subcompactions: 1 2023-10-18T12:17:52.510 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-18T12:17:52.510 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-18T12:17:52.511 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-18T12:17:52.511 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_total_wal_size: 0 2023-10-18T12:17:52.511 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-18T12:17:52.511 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-18T12:17:52.511 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-18T12:17:52.512 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-18T12:17:52.512 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_open_files: -1 2023-10-18T12:17:52.512 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bytes_per_sync: 0 2023-10-18T12:17:52.512 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-18T12:17:52.512 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-18T12:17:52.512 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_readahead_size: 0 2023-10-18T12:17:52.513 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_background_flushes: -1 2023-10-18T12:17:52.513 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Compression algorithms supported: 2023-10-18T12:17:52.513 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-18T12:17:52.513 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kZSTD supported: 0 2023-10-18T12:17:52.513 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kXpressCompression supported: 0 2023-10-18T12:17:52.513 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kLZ4HCCompression supported: 1 2023-10-18T12:17:52.514 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kZlibCompression supported: 1 2023-10-18T12:17:52.514 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kSnappyCompression supported: 1 2023-10-18T12:17:52.514 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kLZ4Compression supported: 1 2023-10-18T12:17:52.514 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: kBZip2Compression supported: 0 2023-10-18T12:17:52.514 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-18T12:17:52.515 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-18T12:17:52.515 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-18T12:17:52.515 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-18T12:17:52.515 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-18T12:17:52.515 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.merge_operator: 2023-10-18T12:17:52.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_filter: None 2023-10-18T12:17:52.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_filter_factory: None 2023-10-18T12:17:52.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.sst_partitioner_factory: None 2023-10-18T12:17:52.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-18T12:17:52.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-18T12:17:52.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d61960f760) 2023-10-18T12:17:52.517 INFO:journalctl@ceph.mon.b.smithi152.stdout: cache_index_and_filter_blocks: 1 2023-10-18T12:17:52.517 INFO:journalctl@ceph.mon.b.smithi152.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-18T12:17:52.517 INFO:journalctl@ceph.mon.b.smithi152.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-18T12:17:52.517 INFO:journalctl@ceph.mon.b.smithi152.stdout: pin_top_level_index_and_filter: 1 2023-10-18T12:17:52.517 INFO:journalctl@ceph.mon.b.smithi152.stdout: index_type: 0 2023-10-18T12:17:52.518 INFO:journalctl@ceph.mon.b.smithi152.stdout: data_block_index_type: 0 2023-10-18T12:17:52.518 INFO:journalctl@ceph.mon.b.smithi152.stdout: index_shortening: 1 2023-10-18T12:17:52.518 INFO:journalctl@ceph.mon.b.smithi152.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-18T12:17:52.518 INFO:journalctl@ceph.mon.b.smithi152.stdout: checksum: 4 2023-10-18T12:17:52.518 INFO:journalctl@ceph.mon.b.smithi152.stdout: no_block_cache: 0 2023-10-18T12:17:52.519 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_cache: 0x55d6196651f0 2023-10-18T12:17:52.519 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_cache_name: BinnedLRUCache 2023-10-18T12:17:52.519 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_cache_options: 2023-10-18T12:17:52.519 INFO:journalctl@ceph.mon.b.smithi152.stdout: capacity : 536870912 2023-10-18T12:17:52.519 INFO:journalctl@ceph.mon.b.smithi152.stdout: num_shard_bits : 4 2023-10-18T12:17:52.520 INFO:journalctl@ceph.mon.b.smithi152.stdout: strict_capacity_limit : 0 2023-10-18T12:17:52.520 INFO:journalctl@ceph.mon.b.smithi152.stdout: high_pri_pool_ratio: 0.000 2023-10-18T12:17:52.520 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_cache_compressed: (nil) 2023-10-18T12:17:52.520 INFO:journalctl@ceph.mon.b.smithi152.stdout: persistent_cache: (nil) 2023-10-18T12:17:52.520 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_size: 4096 2023-10-18T12:17:52.520 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_size_deviation: 10 2023-10-18T12:17:52.521 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_restart_interval: 16 2023-10-18T12:17:52.521 INFO:journalctl@ceph.mon.b.smithi152.stdout: index_block_restart_interval: 1 2023-10-18T12:17:52.521 INFO:journalctl@ceph.mon.b.smithi152.stdout: metadata_block_size: 4096 2023-10-18T12:17:52.521 INFO:journalctl@ceph.mon.b.smithi152.stdout: partition_filters: 0 2023-10-18T12:17:52.521 INFO:journalctl@ceph.mon.b.smithi152.stdout: use_delta_encoding: 1 2023-10-18T12:17:52.522 INFO:journalctl@ceph.mon.b.smithi152.stdout: filter_policy: bloomfilter 2023-10-18T12:17:52.522 INFO:journalctl@ceph.mon.b.smithi152.stdout: whole_key_filtering: 1 2023-10-18T12:17:52.522 INFO:journalctl@ceph.mon.b.smithi152.stdout: verify_compression: 0 2023-10-18T12:17:52.522 INFO:journalctl@ceph.mon.b.smithi152.stdout: read_amp_bytes_per_bit: 0 2023-10-18T12:17:52.522 INFO:journalctl@ceph.mon.b.smithi152.stdout: format_version: 5 2023-10-18T12:17:52.522 INFO:journalctl@ceph.mon.b.smithi152.stdout: enable_index_compression: 1 2023-10-18T12:17:52.523 INFO:journalctl@ceph.mon.b.smithi152.stdout: block_align: 0 2023-10-18T12:17:52.523 INFO:journalctl@ceph.mon.b.smithi152.stdout: max_auto_readahead_size: 262144 2023-10-18T12:17:52.523 INFO:journalctl@ceph.mon.b.smithi152.stdout: prepopulate_block_cache: 0 2023-10-18T12:17:52.523 INFO:journalctl@ceph.mon.b.smithi152.stdout: initial_auto_readahead_size: 8192 2023-10-18T12:17:52.523 INFO:journalctl@ceph.mon.b.smithi152.stdout: num_file_reads_for_auto_readahead: 2 2023-10-18T12:17:52.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.write_buffer_size: 33554432 2023-10-18T12:17:52.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_write_buffer_number: 2 2023-10-18T12:17:52.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression: NoCompression 2023-10-18T12:17:52.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression: Disabled 2023-10-18T12:17:52.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.prefix_extractor: nullptr 2023-10-18T12:17:52.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-18T12:17:52.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.num_levels: 7 2023-10-18T12:17:52.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-18T12:17:52.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-18T12:17:52.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-18T12:17:52.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-18T12:17:52.526 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-18T12:17:52.526 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-18T12:17:52.526 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-18T12:17:52.526 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-18T12:17:52.526 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-18T12:17:52.527 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-18T12:17:52.527 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-18T12:17:52.527 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-18T12:17:52.527 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-18T12:17:52.527 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.level: 32767 2023-10-18T12:17:52.528 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.strategy: 0 2023-10-18T12:17:52.528 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-18T12:17:52.528 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-18T12:17:52.528 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-18T12:17:52.528 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-18T12:17:52.529 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.enabled: false 2023-10-18T12:17:52.529 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-18T12:17:52.529 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-18T12:17:52.529 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-18T12:17:52.529 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-18T12:17:52.529 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.target_file_size_base: 67108864 2023-10-18T12:17:52.530 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-18T12:17:52.530 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-18T12:17:52.530 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-18T12:17:52.530 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-18T12:17:52.530 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-18T12:17:52.531 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-18T12:17:52.531 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-18T12:17:52.531 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-18T12:17:52.531 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-18T12:17:52.531 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-18T12:17:52.531 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-18T12:17:52.532 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-18T12:17:52.532 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-18T12:17:52.532 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-18T12:17:52.532 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.arena_block_size: 1048576 2023-10-18T12:17:52.532 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-18T12:17:52.533 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-18T12:17:52.533 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.disable_auto_compactions: 0 2023-10-18T12:17:52.533 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-18T12:17:52.533 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-18T12:17:52.533 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-18T12:17:52.533 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-18T12:17:52.534 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-18T12:17:52.534 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-18T12:17:52.534 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-18T12:17:52.534 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-18T12:17:52.534 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-18T12:17:52.535 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-18T12:17:52.535 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.table_properties_collectors: 2023-10-18T12:17:52.535 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.inplace_update_support: 0 2023-10-18T12:17:52.535 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-18T12:17:52.535 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-18T12:17:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-18T12:17:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-18T12:17:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.bloom_locality: 0 2023-10-18T12:17:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.max_successive_merges: 0 2023-10-18T12:17:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-18T12:17:52.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.paranoid_file_checks: 0 2023-10-18T12:17:52.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.force_consistency_checks: 1 2023-10-18T12:17:52.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.report_bg_io_stats: 0 2023-10-18T12:17:52.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.ttl: 2592000 2023-10-18T12:17:52.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-18T12:17:52.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-18T12:17:52.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-18T12:17:52.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.enable_blob_files: false 2023-10-18T12:17:52.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.min_blob_size: 0 2023-10-18T12:17:52.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.blob_file_size: 268435456 2023-10-18T12:17:52.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-18T12:17:52.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-18T12:17:52.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-18T12:17:52.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-18T12:17:52.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-18T12:17:52.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.blob_file_starting_level: 0 2023-10-18T12:17:52.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-18T12:17:52.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-18T12:17:52.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-18T12:17:52.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 706d43c1-3d64-4c3e-8ce6-7488273fddfc 2023-10-18T12:17:52.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631472381857, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-18T12:17:52.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-18T12:17:52.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631472382442, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697631472, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "706d43c1-3d64-4c3e-8ce6-7488273fddfc", "db_session_id": "JFM3WZZA92QYEACEKAZC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-18T12:17:52.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697631472382531, "job": 1, "event": "recovery_finished"} 2023-10-18T12:17:52.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-18T12:17:52.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-18T12:17:52.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d619702000 2023-10-18T12:17:52.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: DB pointer 0x55d6196ee000 2023-10-18T12:17:52.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-18T12:17:52.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-18T12:17:52.542 INFO:journalctl@ceph.mon.b.smithi152.stdout: ** DB Stats ** 2023-10-18T12:17:52.542 INFO:journalctl@ceph.mon.b.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T12:17:52.543 INFO:journalctl@ceph.mon.b.smithi152.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-18T12:17:52.543 INFO:journalctl@ceph.mon.b.smithi152.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T12:17:52.543 INFO:journalctl@ceph.mon.b.smithi152.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T12:17:52.543 INFO:journalctl@ceph.mon.b.smithi152.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-18T12:17:52.543 INFO:journalctl@ceph.mon.b.smithi152.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T12:17:52.544 INFO:journalctl@ceph.mon.b.smithi152.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T12:17:52.544 INFO:journalctl@ceph.mon.b.smithi152.stdout: 2023-10-18T12:17:52.544 INFO:journalctl@ceph.mon.b.smithi152.stdout: ** Compaction Stats [default] ** 2023-10-18T12:17:52.544 INFO:journalctl@ceph.mon.b.smithi152.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T12:17:52.544 INFO:journalctl@ceph.mon.b.smithi152.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-18T12:17:52.545 INFO:journalctl@ceph.mon.b.smithi152.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:17:52.545 INFO:journalctl@ceph.mon.b.smithi152.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:17:52.545 INFO:journalctl@ceph.mon.b.smithi152.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:17:52.545 INFO:journalctl@ceph.mon.b.smithi152.stdout: 2023-10-18T12:17:52.545 INFO:journalctl@ceph.mon.b.smithi152.stdout: ** Compaction Stats [default] ** 2023-10-18T12:17:52.545 INFO:journalctl@ceph.mon.b.smithi152.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T12:17:52.546 INFO:journalctl@ceph.mon.b.smithi152.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-18T12:17:52.546 INFO:journalctl@ceph.mon.b.smithi152.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T12:17:52.546 INFO:journalctl@ceph.mon.b.smithi152.stdout: 2023-10-18T12:17:52.546 INFO:journalctl@ceph.mon.b.smithi152.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-18T12:17:52.546 INFO:journalctl@ceph.mon.b.smithi152.stdout: 2023-10-18T12:17:52.547 INFO:journalctl@ceph.mon.b.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T12:17:52.547 INFO:journalctl@ceph.mon.b.smithi152.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-18T12:17:52.547 INFO:journalctl@ceph.mon.b.smithi152.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-18T12:17:52.547 INFO:journalctl@ceph.mon.b.smithi152.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-18T12:17:52.547 INFO:journalctl@ceph.mon.b.smithi152.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-18T12:17:52.548 INFO:journalctl@ceph.mon.b.smithi152.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-18T12:17:52.548 INFO:journalctl@ceph.mon.b.smithi152.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T12:17:52.548 INFO:journalctl@ceph.mon.b.smithi152.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T12:17:52.548 INFO:journalctl@ceph.mon.b.smithi152.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-18T12:17:52.548 INFO:journalctl@ceph.mon.b.smithi152.stdout: Block cache BinnedLRUCache@0x55d6196651f0#8 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-10-18T12:17:52.548 INFO:journalctl@ceph.mon.b.smithi152.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-18T12:17:52.549 INFO:journalctl@ceph.mon.b.smithi152.stdout: 2023-10-18T12:17:52.549 INFO:journalctl@ceph.mon.b.smithi152.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-18T12:17:52.549 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-18T12:17:52.549 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: using public_addr v2:172.21.15.152:0/0 -> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2023-10-18T12:17:52.549 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: starting mon.b rank -1 at public addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] at bind addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:52.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(???) e0 preinit fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:17:52.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 bash[134726]: 09965e2e00233b4e882ccc275f27450fe9968fd998481cc64cc10fc548e63585 2023-10-18T12:17:52.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).mds e1 new map 2023-10-18T12:17:52.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-18T12:17:52.550 INFO:journalctl@ceph.mon.b.smithi152.stdout: e1 2023-10-18T12:17:52.551 INFO:journalctl@ceph.mon.b.smithi152.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T12:17:52.551 INFO:journalctl@ceph.mon.b.smithi152.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T12:17:52.551 INFO:journalctl@ceph.mon.b.smithi152.stdout: legacy client fscid: -1 2023-10-18T12:17:52.551 INFO:journalctl@ceph.mon.b.smithi152.stdout: 2023-10-18T12:17:52.551 INFO:journalctl@ceph.mon.b.smithi152.stdout: No filesystems configured 2023-10-18T12:17:52.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-18T12:17:52.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-18T12:17:52.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-18T12:17:52.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-18T12:17:52.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-18T12:17:52.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-18T12:17:52.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-18T12:17:52.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:17:52.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:17:52.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T12:17:52.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:17:52.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:52.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:52.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:52.556 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:17:52.556 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.556 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:52.556 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:52.556 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:52.557 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:52.557 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.557 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.557 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.557 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Deploying cephadm binary to smithi186 2023-10-18T12:17:52.557 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.558 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.558 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Added host smithi186 2023-10-18T12:17:52.558 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:52.558 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.558 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.559 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.559 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.559 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:17:52.559 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.559 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.560 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.560 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.560 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.560 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.560 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:17:52.560 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:52.561 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:52.561 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/985804014' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-18T12:17:52.561 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.561 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.561 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.562 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:17:52.562 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:17:52.562 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:17:52.562 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:17:52.562 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/985804014' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-18T12:17:52.563 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:52.563 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.563 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:17:52.563 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Saving service mon spec with placement smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c;count:3 2023-10-18T12:17:52.563 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.563 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:52.564 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:52.564 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:17:52.564 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.564 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:17:52.564 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:52.565 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Deploying daemon mon.c on smithi186 2023-10-18T12:17:52.565 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.565 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.565 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:52.565 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.a calling monitor election 2023-10-18T12:17:52.566 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.566 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.566 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.566 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.566 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.c calling monitor election 2023-10-18T12:17:52.566 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.567 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.567 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.567 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.567 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.567 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T12:17:52.568 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: monmap e2: 2 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:17:52.568 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: fsmap 2023-10-18T12:17:52.568 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:52.568 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mgrmap e13: x(active, since 46s) 2023-10-18T12:17:52.568 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: overall HEALTH_OK 2023-10-18T12:17:52.569 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.569 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.569 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:52.569 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:17:52.569 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:17:52.569 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: Deploying daemon mon.b on smithi152 2023-10-18T12:17:52.570 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/3662346108' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T12:17:52.570 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:52.570 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:52.570 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 ceph-mon[134767]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-18T12:17:52.570 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:52 smithi152 systemd[1]: Started Ceph mon.b for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:17:57.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:57.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:57.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: mon.c calling monitor election 2023-10-18T12:17:57.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: mon.a calling monitor election 2023-10-18T12:17:57.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:57.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:57.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:57.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T12:17:57.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: monmap e3: 3 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],b=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:17:57.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: fsmap 2023-10-18T12:17:57.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:57.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: mgrmap e13: x(active, since 55s) 2023-10-18T12:17:57.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-18T12:17:57.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-18T12:17:57.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-18T12:17:57.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: mon.b (rank 2) addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] is down (out of quorum) 2023-10-18T12:17:57.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.860 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.860 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:57 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.954 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-18T12:17:57.954 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","modified":"2023-10-18T12:17:52.414628Z","created":"2023-10-18T12:15:59.082484Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-18T12:17:57.956 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2023-10-18T12:17:57.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: mon.c calling monitor election 2023-10-18T12:17:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: mon.a calling monitor election 2023-10-18T12:17:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:57.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:57.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T12:17:57.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: monmap e3: 3 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],b=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:17:57.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: fsmap 2023-10-18T12:17:57.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:57.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: mgrmap e13: x(active, since 55s) 2023-10-18T12:17:57.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-18T12:17:57.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-18T12:17:57.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-18T12:17:57.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: mon.b (rank 2) addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] is down (out of quorum) 2023-10-18T12:17:57.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:57.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:57 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:58.409 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-18T12:17:58.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-18T12:17:58.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:58 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/1176601182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T12:17:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:17:58 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:58.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:58 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:58.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/1176601182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T12:17:58.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:17:58 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.153 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:17:59.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:59.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:17:59.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: mon.c calling monitor election 2023-10-18T12:17:59.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: mon.a calling monitor election 2023-10-18T12:17:59.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:17:59.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:59.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:17:59.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:17:59.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T12:17:59.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: monmap e3: 3 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],b=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:17:59.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: fsmap 2023-10-18T12:17:59.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:17:59.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: mgrmap e13: x(active, since 55s) 2023-10-18T12:17:59.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-18T12:17:59.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-18T12:17:59.926 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-18T12:17:59.926 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: mon.b (rank 2) addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] is down (out of quorum) 2023-10-18T12:17:59.926 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:59.926 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:59.927 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:59.927 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:17:59.927 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:17:59.927 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/1176601182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T12:17:59.928 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:17:59 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:00.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: mon.b calling monitor election 2023-10-18T12:18:00.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:18:00.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:18:00.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:18:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: mon.b calling monitor election 2023-10-18T12:18:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: mon.c calling monitor election 2023-10-18T12:18:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: mon.a calling monitor election 2023-10-18T12:18:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-18T12:18:00.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: monmap e3: 3 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],b=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:18:00.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: fsmap 2023-10-18T12:18:00.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:18:00.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: mgrmap e13: x(active, since 57s) 2023-10-18T12:18:00.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-18T12:18:00.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Cluster is now healthy 2023-10-18T12:18:00.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: overall HEALTH_OK 2023-10-18T12:18:00.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:00 smithi186 ceph-mon[135253]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.542 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: mon.b calling monitor election 2023-10-18T12:18:00.543 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:18:00.543 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:18:00.543 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:18:00.543 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: mon.b calling monitor election 2023-10-18T12:18:00.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: mon.c calling monitor election 2023-10-18T12:18:00.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: mon.a calling monitor election 2023-10-18T12:18:00.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-18T12:18:00.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: monmap e3: 3 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],b=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:18:00.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: fsmap 2023-10-18T12:18:00.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:18:00.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: mgrmap e13: x(active, since 57s) 2023-10-18T12:18:00.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-18T12:18:00.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Cluster is now healthy 2023-10-18T12:18:00.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: overall HEALTH_OK 2023-10-18T12:18:00.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.547 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:00 smithi070 ceph-mon[98639]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: mon.b calling monitor election 2023-10-18T12:18:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:18:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:18:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:18:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: mon.b calling monitor election 2023-10-18T12:18:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: mon.c calling monitor election 2023-10-18T12:18:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: mon.a calling monitor election 2023-10-18T12:18:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-18T12:18:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: monmap e3: 3 mons at {a=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],b=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T12:18:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: fsmap 2023-10-18T12:18:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T12:18:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: mgrmap e13: x(active, since 57s) 2023-10-18T12:18:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-18T12:18:00.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Cluster is now healthy 2023-10-18T12:18:00.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: overall HEALTH_OK 2023-10-18T12:18:00.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:00.676 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:00 smithi152 ceph-mon[134767]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:01.008 INFO:teuthology.orchestra.run.smithi070.stdout:# minimal ceph.conf for f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:18:01.008 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2023-10-18T12:18:01.008 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:18:01.008 INFO:teuthology.orchestra.run.smithi070.stdout: mon_host = [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2023-10-18T12:18:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: Reconfiguring mon.a (unknown last config time)... 2023-10-18T12:18:01.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:01.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:01.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:01.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: Reconfiguring daemon mon.a on smithi070 2023-10-18T12:18:01.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:01 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1701820380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:01.445 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-18T12:18:01.445 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:18:01.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2023-10-18T12:18:01.517 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:18:01.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:01.630 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:18:01.630 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2023-10-18T12:18:01.662 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.662 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.662 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.663 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:01.663 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:01.663 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.663 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.663 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: Reconfiguring mon.a (unknown last config time)... 2023-10-18T12:18:01.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:01.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:01.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:01.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: Reconfiguring daemon mon.a on smithi070 2023-10-18T12:18:01.665 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:01 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1701820380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:01.667 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:18:01.668 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:01.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:01.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: Reconfiguring mon.a (unknown last config time)... 2023-10-18T12:18:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: Reconfiguring daemon mon.a on smithi070 2023-10-18T12:18:01.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:01 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1701820380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:01.738 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:18:01.738 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2023-10-18T12:18:01.774 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:18:01.774 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:01.845 INFO:tasks.cephadm:enabling module snap_schedule 2023-10-18T12:18:01.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T12:18:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:03.410 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.410 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.411 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: Reconfiguring mon.b (monmap changed)... 2023-10-18T12:18:03.411 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:03.411 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:03.412 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:03.412 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: Reconfiguring daemon mon.b on smithi152 2023-10-18T12:18:03.412 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:03.412 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:03 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: Reconfiguring mon.b (monmap changed)... 2023-10-18T12:18:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:03.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:03.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:03.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: Reconfiguring daemon mon.b on smithi152 2023-10-18T12:18:03.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:03.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:03 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: Reconfiguring mon.b (monmap changed)... 2023-10-18T12:18:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: Reconfiguring daemon mon.b on smithi152 2023-10-18T12:18:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:03.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:03 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: Reconfiguring mon.c (monmap changed)... 2023-10-18T12:18:04.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:04.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: Reconfiguring daemon mon.c on smithi186 2023-10-18T12:18:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/799686467' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-18T12:18:04.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:04 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-18T12:18:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: Reconfiguring mon.c (monmap changed)... 2023-10-18T12:18:04.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:04.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:04.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:04.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: Reconfiguring daemon mon.c on smithi186 2023-10-18T12:18:04.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/799686467' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-18T12:18:04.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:04 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-18T12:18:04.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:04.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: Reconfiguring mon.c (monmap changed)... 2023-10-18T12:18:04.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T12:18:04.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T12:18:04.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:04.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: Reconfiguring daemon mon.c on smithi186 2023-10-18T12:18:04.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/799686467' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-18T12:18:04.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:04 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-18T12:18:05.294 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: ignoring --setuser ceph since I am not root 2023-10-18T12:18:05.295 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: ignoring --setgroup ceph since I am not root 2023-10-18T12:18:05.295 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:05.292+0000 7f33a0d01200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:18:05.468 INFO:tasks.cephadm:Adding mgr.x on smithi070 2023-10-18T12:18:05.468 INFO:tasks.cephadm:Adding mgr.y on smithi152 2023-10-18T12:18:05.469 INFO:tasks.cephadm:Adding mgr.z on smithi186 2023-10-18T12:18:05.469 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch apply mgr '3;smithi070=x;smithi152=y;smithi186=z' 2023-10-18T12:18:05.603 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:05.397+0000 7f33a0d01200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:18:05.604 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:05.515+0000 7f33a0d01200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:18:05.857 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:05.635+0000 7f33a0d01200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:18:05.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:05.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:05.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:05.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:05.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-18T12:18:05.860 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-mon[98639]: mgrmap e14: x(active, since 62s) 2023-10-18T12:18:05.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:05.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:05.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:05.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:05.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-18T12:18:05.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:05 smithi152 ceph-mon[134767]: mgrmap e14: x(active, since 62s) 2023-10-18T12:18:05.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:05.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:05.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='mgr.14150 172.21.15.70:0/1128444344' entity='mgr.x' 2023-10-18T12:18:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-18T12:18:05.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:05 smithi186 ceph-mon[135253]: mgrmap e14: x(active, since 62s) 2023-10-18T12:18:06.032 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:18:06.303 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:05.857+0000 7f33a0d01200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:18:06.581 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:06.301+0000 7f33a0d01200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:18:06.581 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:06.464+0000 7f33a0d01200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:18:06.581 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:06.579+0000 7f33a0d01200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:18:06.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:06.702+0000 7f33a0d01200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:18:07.604 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:07.235+0000 7f33a0d01200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:18:08.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:08 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:08.072+0000 7f33a0d01200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:18:08.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:08 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:08.180+0000 7f33a0d01200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:18:09.482 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:09.134+0000 7f33a0d01200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:18:09.482 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:09.254+0000 7f33a0d01200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:18:09.482 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:09.480+0000 7f33a0d01200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:18:10.104 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:09.844+0000 7f33a0d01200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:18:10.604 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:10.123+0000 7f33a0d01200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:18:10.605 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:10.315+0000 7f33a0d01200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:18:11.104 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:10.660+0000 7f33a0d01200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:18:11.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:11.591+0000 7f33a0d01200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:18:11.855 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:11.763+0000 7f33a0d01200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:18:12.354 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:12.013+0000 7f33a0d01200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:18:12.355 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:12.159+0000 7f33a0d01200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:18:12.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:12.440+0000 7f33a0d01200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:18:13.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:18:13.512+0000 7f33a0d01200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:18:13.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: Active manager daemon x restarted 2023-10-18T12:18:13.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: Activating manager daemon x 2023-10-18T12:18:13.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: osdmap e5: 0 total, 0 up, 0 in 2023-10-18T12:18:13.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: mgrmap e15: x(active, starting, since 0.00502793s) 2023-10-18T12:18:13.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:18:13.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:13.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:18:13.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T12:18:13.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T12:18:13.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T12:18:13.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T12:18:13.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: Manager daemon x is now available 2023-10-18T12:18:13.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:13.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:13.874 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: Active manager daemon x restarted 2023-10-18T12:18:13.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: Activating manager daemon x 2023-10-18T12:18:13.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: osdmap e5: 0 total, 0 up, 0 in 2023-10-18T12:18:13.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: mgrmap e15: x(active, starting, since 0.00502793s) 2023-10-18T12:18:13.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:18:13.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:13.876 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:18:13.876 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T12:18:13.876 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T12:18:13.876 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T12:18:13.876 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T12:18:13.877 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: Manager daemon x is now available 2023-10-18T12:18:13.877 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:13.877 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:13.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: Active manager daemon x restarted 2023-10-18T12:18:13.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: Activating manager daemon x 2023-10-18T12:18:13.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: osdmap e5: 0 total, 0 up, 0 in 2023-10-18T12:18:13.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: mgrmap e15: x(active, starting, since 0.00502793s) 2023-10-18T12:18:13.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:18:13.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:18:13.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:18:13.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T12:18:13.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T12:18:13.884 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T12:18:13.884 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T12:18:13.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: Manager daemon x is now available 2023-10-18T12:18:13.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:13.885 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:14.534 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2023-10-18T12:18:14.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T12:18:14.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T12:18:14.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:14 smithi070 ceph-mon[98639]: mgrmap e16: x(active, since 1.00869s) 2023-10-18T12:18:14.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:14.903 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T12:18:14.903 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T12:18:14.903 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:14 smithi186 ceph-mon[135253]: mgrmap e16: x(active, since 1.00869s) 2023-10-18T12:18:14.904 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:14.905 DEBUG:teuthology.orchestra.run.smithi152:mgr.y> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.y.service 2023-10-18T12:18:14.909 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.z.service 2023-10-18T12:18:14.911 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2023-10-18T12:18:14.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set client client_mount_timeout 600 2023-10-18T12:18:14.917 INFO:tasks.cephadm:setting config [client] debug_client = 20 2023-10-18T12:18:14.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set client debug_client 20 2023-10-18T12:18:14.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T12:18:14.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T12:18:14.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:14 smithi152 ceph-mon[134767]: mgrmap e16: x(active, since 1.00869s) 2023-10-18T12:18:14.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:14.937 INFO:journalctl@ceph.mgr.y.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:18:14.941 INFO:journalctl@ceph.mgr.z.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:18:14.962 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2023-10-18T12:18:14.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set client debug_ms 1 2023-10-18T12:18:15.010 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2023-10-18T12:18:15.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set client rados_mon_op_timeout 900 2023-10-18T12:18:15.057 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2023-10-18T12:18:15.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set client rados_osd_op_timeout 900 2023-10-18T12:18:15.105 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2023-10-18T12:18:15.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds debug_mds 20 2023-10-18T12:18:15.155 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2023-10-18T12:18:15.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds debug_mds_balancer 20 2023-10-18T12:18:15.208 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2023-10-18T12:18:15.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds debug_ms 1 2023-10-18T12:18:15.263 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-18T12:18:15.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_fragment_size_max 10000 2023-10-18T12:18:15.317 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2023-10-18T12:18:15.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_merge_size 5 2023-10-18T12:18:15.367 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2023-10-18T12:18:15.368 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_split_bits 3 2023-10-18T12:18:15.421 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2023-10-18T12:18:15.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_split_size 100 2023-10-18T12:18:15.478 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2023-10-18T12:18:15.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_debug_frag True 2023-10-18T12:18:15.538 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2023-10-18T12:18:15.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_debug_scatterstat True 2023-10-18T12:18:15.579 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: [18/Oct/2023:12:18:14] ENGINE Bus STARTING 2023-10-18T12:18:15.579 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: [18/Oct/2023:12:18:14] ENGINE Serving on https://172.21.15.70:7150 2023-10-18T12:18:15.579 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi070=x;smithi152=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:18:15.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: Saving service mgr spec with placement smithi070=x;smithi152=y;smithi186=z;count:3 2023-10-18T12:18:15.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: [18/Oct/2023:12:18:14] ENGINE Serving on http://172.21.15.70:8765 2023-10-18T12:18:15.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: [18/Oct/2023:12:18:14] ENGINE Bus STARTED 2023-10-18T12:18:15.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.581 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.581 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.590 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2023-10-18T12:18:15.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_op_complaint_time 180 2023-10-18T12:18:15.643 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2023-10-18T12:18:15.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_verify_scatter True 2023-10-18T12:18:15.698 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2023-10-18T12:18:15.698 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds osd_op_complaint_time 180 2023-10-18T12:18:15.753 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2023-10-18T12:18:15.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds rados_mon_op_timeout 900 2023-10-18T12:18:15.806 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2023-10-18T12:18:15.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mds rados_osd_op_timeout 900 2023-10-18T12:18:15.864 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2023-10-18T12:18:15.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mgr debug_cephsqlite 20 2023-10-18T12:18:15.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: [18/Oct/2023:12:18:14] ENGINE Bus STARTING 2023-10-18T12:18:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: [18/Oct/2023:12:18:14] ENGINE Serving on https://172.21.15.70:7150 2023-10-18T12:18:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi070=x;smithi152=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:18:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: Saving service mgr spec with placement smithi070=x;smithi152=y;smithi186=z;count:3 2023-10-18T12:18:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: [18/Oct/2023:12:18:14] ENGINE Serving on http://172.21.15.70:8765 2023-10-18T12:18:15.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: [18/Oct/2023:12:18:14] ENGINE Bus STARTED 2023-10-18T12:18:15.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.936 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2023-10-18T12:18:15.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T12:18:15.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: [18/Oct/2023:12:18:14] ENGINE Bus STARTING 2023-10-18T12:18:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: [18/Oct/2023:12:18:14] ENGINE Serving on https://172.21.15.70:7150 2023-10-18T12:18:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi070=x;smithi152=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:18:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: Saving service mgr spec with placement smithi070=x;smithi152=y;smithi186=z;count:3 2023-10-18T12:18:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: [18/Oct/2023:12:18:14] ENGINE Serving on http://172.21.15.70:8765 2023-10-18T12:18:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: [18/Oct/2023:12:18:14] ENGINE Bus STARTED 2023-10-18T12:18:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:15.993 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2023-10-18T12:18:15.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T12:18:16.062 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2023-10-18T12:18:16.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set mon mon_op_complaint_time 120 2023-10-18T12:18:16.122 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2023-10-18T12:18:16.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config set osd osd_op_complaint_time 180 2023-10-18T12:18:16.185 DEBUG:tasks.cephadm:set 24 configs 2023-10-18T12:18:16.186 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi070') 2023-10-18T12:18:16.360 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:16.465 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:16.512 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:16.824 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:17.468 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: mgrmap e17: x(active, since 2s) 2023-10-18T12:18:17.469 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.469 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.469 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.469 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:17.469 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.469 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:17.470 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.470 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: mgrmap e17: x(active, since 2s) 2023-10-18T12:18:17.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:17.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:17.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: mgrmap e17: x(active, since 2s) 2023-10-18T12:18:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:17.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:17.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:17.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:18.302 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:18.468 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:18.847 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:20.591 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:20.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:20.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:20.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:20.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:20.593 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:18:20.593 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:18:20.593 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:18:20.593 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:20.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:20 smithi070 ceph-mon[98639]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:20.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:20.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:20.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:18:20.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:18:20.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:18:20.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:20.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:20 smithi152 ceph-mon[134767]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:18:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi070:/etc/ceph/ceph.conf 2023-10-18T12:18:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi152:/etc/ceph/ceph.conf 2023-10-18T12:18:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-18T12:18:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T12:18:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:20.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:20 smithi186 ceph-mon[135253]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:21.144 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:21.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:21.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:21.695 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.695 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.695 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.695 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.695 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:21.696 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1950362244' entity='client.admin' 2023-10-18T12:18:21.696 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:21 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:21.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:21.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:21.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:21.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1950362244' entity='client.admin' 2023-10-18T12:18:21.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:21 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:21.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: Updating smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:21.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: Updating smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:21.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-10-18T12:18:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1950362244' entity='client.admin' 2023-10-18T12:18:21.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:21 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:22.482 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi070') 2023-10-18T12:18:23.012 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:23.019 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:23.026 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:23.027 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.027 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.027 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.027 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:23.028 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T12:18:23.028 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:23.028 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:23.028 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:22 smithi070 ceph-mon[98639]: Deploying daemon mgr.z on smithi186 2023-10-18T12:18:23.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:23.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:23.123 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T12:18:23.123 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:23.123 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:23.123 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:22 smithi186 ceph-mon[135253]: Deploying daemon mgr.z on smithi186 2023-10-18T12:18:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: Updating smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T12:18:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T12:18:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:22 smithi152 ceph-mon[134767]: Deploying daemon mgr.z on smithi186 2023-10-18T12:18:23.709 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:23 smithi186 systemd[1]: Starting Ceph mgr.z for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:18:23.779 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:23.969 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:23 smithi186 podman[138279]: 2023-10-18T12:18:23.969 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:23 smithi186 bash[138279]: f9c9a71d42d0b6ce53247e8eb9268931d97590d9199fa1fca70b05be54488c7c 2023-10-18T12:18:23.970 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:23 smithi186 ceph-mgr[138319]: -- 172.21.15.186:0/686224440 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562531d6f1e0 con 0x562531d44c00 2023-10-18T12:18:24.338 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 systemd[1]: Started Ceph mgr.z for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:18:24.338 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:24.074+0000 7f226c48b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:18:24.339 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:24.154+0000 7f226c48b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:18:24.339 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:24.242+0000 7f226c48b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:18:24.339 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:24.335+0000 7f226c48b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:18:24.520 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:24.523 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:24.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:24.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:24.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T12:18:24.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:24.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:24.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:24 smithi152 ceph-mon[134767]: Deploying daemon mgr.y on smithi152 2023-10-18T12:18:24.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:24.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:24.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T12:18:24.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:24.699 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:24.699 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:24 smithi070 ceph-mon[98639]: Deploying daemon mgr.y on smithi152 2023-10-18T12:18:24.719 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:24.506+0000 7f226c48b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:18:24.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:24.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:24.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:24.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T12:18:24.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:24.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:24.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-mon[135253]: Deploying daemon mgr.y on smithi152 2023-10-18T12:18:25.187 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:24.871+0000 7f226c48b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:18:25.188 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:25.000+0000 7f226c48b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:18:25.188 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:25.088+0000 7f226c48b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:18:25.188 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:25.184+0000 7f226c48b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:18:25.248 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:25.448 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:25 smithi152 systemd[1]: Starting Ceph mgr.y for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:18:25.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:25.607+0000 7f226c48b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:18:26.036 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:25 smithi152 podman[137122]: 2023-10-18T12:18:26.036 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:25 smithi152 bash[137122]: b3646de8663cdeea8f0102ef1c22f9be10019d3764096498d27e7865da204226 2023-10-18T12:18:26.036 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:25 smithi152 ceph-mgr[137162]: -- 172.21.15.152:0/631138456 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e473b71e0 con 0x558e4738d000 2023-10-18T12:18:26.287 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 systemd[1]: Started Ceph mgr.y for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:18:26.287 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:26.111+0000 7fc66c8d3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:18:26.287 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:26.192+0000 7fc66c8d3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:18:26.582 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:26.284+0000 7fc66c8d3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:18:26.582 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:26.383+0000 7fc66c8d3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:18:26.583 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:26.583 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:26.583 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.584 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.584 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.584 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.584 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:26.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:26.718 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:26.277+0000 7f226c48b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:18:26.719 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:26.364+0000 7f226c48b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:18:26.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:26.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:26.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:26.921 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:26.579+0000 7fc66c8d3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:18:26.961 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:27.303 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:26.967+0000 7fc66c8d3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:18:27.304 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:27.104+0000 7fc66c8d3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:18:27.304 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:27.199+0000 7fc66c8d3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:18:27.304 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:27.300+0000 7fc66c8d3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:18:27.468 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:27.180+0000 7f226c48b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:18:27.468 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:27.277+0000 7f226c48b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:18:27.718 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:27.465+0000 7f226c48b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:18:28.144 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:27.759+0000 7f226c48b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:18:28.144 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:27.992+0000 7f226c48b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:18:28.144 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:28.141+0000 7f226c48b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:18:28.171 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:27.755+0000 7fc66c8d3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:18:28.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:28.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:28.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:28.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: Reconfiguring mgr.x (unknown last config time)... 2023-10-18T12:18:28.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:28.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:28.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:28.409 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:28 smithi070 ceph-mon[98639]: Reconfiguring daemon mgr.x on smithi070 2023-10-18T12:18:28.410 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:28.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:28.412 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:28.412 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.412 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: Reconfiguring mgr.x (unknown last config time)... 2023-10-18T12:18:28.412 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:28.413 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:28.413 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:28.413 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-mon[135253]: Reconfiguring daemon mgr.x on smithi070 2023-10-18T12:18:28.515 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:28.431+0000 7fc66c8d3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:18:28.515 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:28.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:28.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:28.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:28.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: Reconfiguring mgr.x (unknown last config time)... 2023-10-18T12:18:28.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T12:18:28.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T12:18:28.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:28.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-mon[134767]: Reconfiguring daemon mgr.x on smithi070 2023-10-18T12:18:28.718 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:28.408+0000 7f226c48b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:18:28.921 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:28.512+0000 7fc66c8d3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:18:29.181 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:29.218 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:29.221 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:29.287 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi070') 2023-10-18T12:18:29.441 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:29.441 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:29.441 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:29.442 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:29.113+0000 7f226c48b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:18:29.442 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:29.245+0000 7f226c48b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:18:29.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:29.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:29.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:29.551 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:29.283+0000 7fc66c8d3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:18:29.552 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:29.374+0000 7fc66c8d3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:18:29.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:29 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:29 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:29.718 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:29.439+0000 7f226c48b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:18:29.719 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:29.552+0000 7f226c48b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:18:29.831 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:29.548+0000 7fc66c8d3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:18:30.171 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:29.828+0000 7fc66c8d3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:18:30.172 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:30.056+0000 7fc66c8d3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:18:30.218 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:29.768+0000 7f226c48b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:18:30.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.643+0000 7fae91329700 1 -- 172.21.15.70:0/2432534406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae7403cf90 msgr2=0x7fae7403f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:30.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.643+0000 7fae91329700 1 --2- 172.21.15.70:0/2432534406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae7403cf90 0x7fae7403f450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fd0 tx=0x7fae7c00bf10 comp rx=0 tx=0).stop 2023-10-18T12:18:30.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.643+0000 7fae91329700 1 -- 172.21.15.70:0/2432534406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae8c101260 msgr2=0x7fae8c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:30.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.643+0000 7fae91329700 1 --2- 172.21.15.70:0/2432534406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae8c101260 0x7fae8c19ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ec10 tx=0x7fae8000ef20 comp rx=0 tx=0).stop 2023-10-18T12:18:30.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 -- 172.21.15.70:0/2432534406 shutdown_connections 2023-10-18T12:18:30.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 --2- 172.21.15.70:0/2432534406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c1057c0 0x7fae8c194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:30.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 --2- 172.21.15.70:0/2432534406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae8c1008a0 0x7fae8c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:30.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 --2- 172.21.15.70:0/2432534406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae7403cf90 0x7fae7403f450 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:30.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 --2- 172.21.15.70:0/2432534406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae8c101260 0x7fae8c19ac30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:30.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 -- 172.21.15.70:0/2432534406 >> 172.21.15.70:0/2432534406 conn(0x7fae8c0779e0 msgr2=0x7fae8c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:30.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 -- 172.21.15.70:0/2432534406 shutdown_connections 2023-10-18T12:18:30.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:28.644+0000 7fae91329700 1 -- 172.21.15.70:0/2432534406 wait complete. 2023-10-18T12:18:30.470 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:30.204+0000 7fc66c8d3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:18:30.793 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:30 smithi070 ceph-mon[98639]: Standby manager daemon z started 2023-10-18T12:18:30.793 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:30 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-18T12:18:30.794 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:30 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T12:18:30.794 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:30 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-18T12:18:30.794 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:30 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T12:18:30.921 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:30.468+0000 7fc66c8d3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:18:30.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-mon[134767]: Standby manager daemon z started 2023-10-18T12:18:30.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-18T12:18:30.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T12:18:30.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-18T12:18:30.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:30 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T12:18:30.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:18:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z[138295]: 2023-10-18T12:18:30.634+0000 7f226c48b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:18:30.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:30 smithi186 ceph-mon[135253]: Standby manager daemon z started 2023-10-18T12:18:30.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:30 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-18T12:18:30.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:30 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T12:18:30.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:30 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-18T12:18:30.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:30 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.186:0/3330047742' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T12:18:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 -- 172.21.15.70:0/3312315559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 msgr2=0x7fd6f80685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 --2- 172.21.15.70:0/3312315559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80685f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009990 tx=0x7fd6ec009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 -- 172.21.15.70:0/3312315559 shutdown_connections 2023-10-18T12:18:31.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 --2- 172.21.15.70:0/3312315559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6f8068b30 0x7fd6f810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 --2- 172.21.15.70:0/3312315559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f81067a0 0x7fd6f8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 --2- 172.21.15.70:0/3312315559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80685f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 -- 172.21.15.70:0/3312315559 >> 172.21.15.70:0/3312315559 conn(0x7fd6f8075790 msgr2=0x7fd6f8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:31.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 -- 172.21.15.70:0/3312315559 shutdown_connections 2023-10-18T12:18:31.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.023+0000 7fd6fe466700 1 -- 172.21.15.70:0/3312315559 wait complete. 2023-10-18T12:18:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.024+0000 7fd6fe466700 1 Processor -- start 2023-10-18T12:18:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 -- start start 2023-10-18T12:18:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 0x7fd6f80ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6f81067a0 0x7fd6f81002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81a5010 con 0x7fd6f8068190 2023-10-18T12:18:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81a5180 con 0x7fd6f8068b30 2023-10-18T12:18:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.025+0000 7fd6fe466700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81a5340 con 0x7fd6f81067a0 2023-10-18T12:18:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 0x7fd6f80ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 0x7fd6f80ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:55432/0 (socket says 172.21.15.70:55432) 2023-10-18T12:18:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36296/0 (socket says 172.21.15.70:36296) 2023-10-18T12:18:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 -- 172.21.15.70:0/2311560736 learned_addr learned my addr 172.21.15.70:0/2311560736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:31.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6fca03700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6f81067a0 0x7fd6f81002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 -- 172.21.15.70:0/2311560736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6f81067a0 msgr2=0x7fd6f81002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6f81067a0 0x7fd6f81002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 -- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 msgr2=0x7fd6f80ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.026+0000 7fd6f77fe700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.027+0000 7fd6f77fe700 1 -- 172.21.15.70:0/2311560736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0095f0 con 0x7fd6f8068b30 2023-10-18T12:18:31.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.027+0000 7fd6f7fff700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:31.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.027+0000 7fd6f77fe700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 0x7fd6f80ffd60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec005910 tx=0x7fd6ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:31.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.027+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec01d070 con 0x7fd6f8068b30 2023-10-18T12:18:31.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.027+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6ec022470 con 0x7fd6f8068b30 2023-10-18T12:18:31.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.027+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec017650 con 0x7fd6f8068b30 2023-10-18T12:18:31.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.028+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f81a55c0 con 0x7fd6f8068b30 2023-10-18T12:18:31.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.028+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f81a5980 con 0x7fd6f8068b30 2023-10-18T12:18:31.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.029+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f8108d50 con 0x7fd6f8068b30 2023-10-18T12:18:31.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.030+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7fd6ec0225e0 con 0x7fd6f8068b30 2023-10-18T12:18:31.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.031+0000 7fd6f57fa700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd6e006cfc0 0x7fd6e006f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.031+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd6ec08c720 con 0x7fd6f8068b30 2023-10-18T12:18:31.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.031+0000 7fd6f7fff700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd6e006cfc0 0x7fd6e006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.032+0000 7fd6f7fff700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd6e006cfc0 0x7fd6e006f480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009fd0 tx=0x7fd6e8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:31.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.037+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd6ec057ba0 con 0x7fd6f8068b30 2023-10-18T12:18:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.258+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7fd6f8100d00 con 0x7fd6f8068b30 2023-10-18T12:18:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.263+0000 7fd6f57fa700 1 -- 172.21.15.70:0/2311560736 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v18)=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd6ec027020 con 0x7fd6f8068b30 2023-10-18T12:18:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.265+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd6e006cfc0 msgr2=0x7fd6e006f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.265+0000 7fd6fe466700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd6e006cfc0 0x7fd6e006f480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009fd0 tx=0x7fd6e8009260 comp rx=0 tx=0).stop 2023-10-18T12:18:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.265+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 msgr2=0x7fd6f80ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.265+0000 7fd6fe466700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 0x7fd6f80ffd60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec005910 tx=0x7fd6ec004730 comp rx=0 tx=0).stop 2023-10-18T12:18:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 shutdown_connections 2023-10-18T12:18:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6f81067a0 0x7fd6f81002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6f8068190 0x7fd6f80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd6e006cfc0 0x7fd6e006f480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 --2- 172.21.15.70:0/2311560736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6f8068b30 0x7fd6f80ffd60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 >> 172.21.15.70:0/2311560736 conn(0x7fd6f8075790 msgr2=0x7fd6f80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 shutdown_connections 2023-10-18T12:18:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.266+0000 7fd6fe466700 1 -- 172.21.15.70:0/2311560736 wait complete. 2023-10-18T12:18:31.501 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:31.175+0000 7fc66c8d3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:18:31.501 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:31.306+0000 7fc66c8d3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:18:31.501 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:31.498+0000 7fc66c8d3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:18:31.830 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:31.611+0000 7fc66c8d3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:18:31.831 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-mon[134767]: mgrmap e18: x(active, since 17s), standbys: z 2023-10-18T12:18:31.831 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-18T12:18:31.832 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.904+0000 7fb206431700 1 -- 172.21.15.70:0/1582536084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 msgr2=0x7fb20010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.904+0000 7fb206431700 1 --2- 172.21.15.70:0/1582536084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb20010d650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800b0a0 tx=0x7fb1f800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.904+0000 7fb206431700 1 -- 172.21.15.70:0/1582536084 shutdown_connections 2023-10-18T12:18:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.904+0000 7fb206431700 1 --2- 172.21.15.70:0/1582536084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb20010d650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.904+0000 7fb206431700 1 --2- 172.21.15.70:0/1582536084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001057c0 0x7fb200105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.904+0000 7fb206431700 1 --2- 172.21.15.70:0/1582536084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2001008a0 0x7fb200100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.905+0000 7fb206431700 1 -- 172.21.15.70:0/1582536084 >> 172.21.15.70:0/1582536084 conn(0x7fb2000779e0 msgr2=0x7fb200077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.905+0000 7fb206431700 1 -- 172.21.15.70:0/1582536084 shutdown_connections 2023-10-18T12:18:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.905+0000 7fb206431700 1 -- 172.21.15.70:0/1582536084 wait complete. 2023-10-18T12:18:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.907+0000 7fb206431700 1 Processor -- start 2023-10-18T12:18:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.907+0000 7fb206431700 1 -- start start 2023-10-18T12:18:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb206431700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 0x7fb200198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb206431700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb200199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb206431700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2001057c0 0x7fb20019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb206431700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20019de40 con 0x7fb200101260 2023-10-18T12:18:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb206431700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20019dfb0 con 0x7fb2001008a0 2023-10-18T12:18:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb206431700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20019e170 con 0x7fb2001057c0 2023-10-18T12:18:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 0x7fb200198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 0x7fb200198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:55448/0 (socket says 172.21.15.70:55448) 2023-10-18T12:18:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 -- 172.21.15.70:0/171869447 learned_addr learned my addr 172.21.15.70:0/171869447 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:31.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb2049ce700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2001057c0 0x7fb20019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1ff7fe700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb200199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 -- 172.21.15.70:0/171869447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2001057c0 msgr2=0x7fb20019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2001057c0 0x7fb20019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 -- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 msgr2=0x7fb200199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb200199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1fffff700 1 -- 172.21.15.70:0/171869447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8009d00 con 0x7fb2001008a0 2023-10-18T12:18:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.908+0000 7fb1ff7fe700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb200199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:18:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.909+0000 7fb1fffff700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 0x7fb200198f40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000e970 tx=0x7fb1f000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.909+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0009790 con 0x7fb2001008a0 2023-10-18T12:18:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.909+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f000eec0 con 0x7fb2001008a0 2023-10-18T12:18:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.909+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20019e450 con 0x7fb2001008a0 2023-10-18T12:18:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.909+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0003cf0 con 0x7fb2001008a0 2023-10-18T12:18:31.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.909+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2001a5cb0 con 0x7fb2001008a0 2023-10-18T12:18:31.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.910+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb200101f30 con 0x7fb2001008a0 2023-10-18T12:18:31.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.911+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7fb1f0004020 con 0x7fb2001008a0 2023-10-18T12:18:31.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.912+0000 7fb1fd7fa700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e806cfc0 0x7fb1e806f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:31.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.912+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb1f0014070 con 0x7fb2001008a0 2023-10-18T12:18:31.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.913+0000 7fb1ff7fe700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e806cfc0 0x7fb1e806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:31.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.914+0000 7fb1ff7fe700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e806cfc0 0x7fb1e806f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb20019a4a0 tx=0x7fb1f4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:31.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:31.916+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb1f0056870 con 0x7fb2001008a0 2023-10-18T12:18:31.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:31 smithi186 ceph-mon[135253]: mgrmap e18: x(active, since 17s), standbys: z 2023-10-18T12:18:31.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-18T12:18:31.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:31 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:32.013 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:31 smithi070 ceph-mon[98639]: mgrmap e18: x(active, since 17s), standbys: z 2023-10-18T12:18:32.013 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-18T12:18:32.013 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:31 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.139+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7fb20004e4c0 con 0x7fb2001008a0 2023-10-18T12:18:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.145+0000 7fb1fd7fa700 1 -- 172.21.15.70:0/171869447 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v19)=0 v19) v1 ==== 117+0+0 (secure 0 0 0) 0x7fb20004e4c0 con 0x7fb2001008a0 2023-10-18T12:18:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e806cfc0 msgr2=0x7fb1e806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e806cfc0 0x7fb1e806f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb20019a4a0 tx=0x7fb1f4009260 comp rx=0 tx=0).stop 2023-10-18T12:18:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 msgr2=0x7fb200198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 0x7fb200198f40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000e970 tx=0x7fb1f000ec80 comp rx=0 tx=0).stop 2023-10-18T12:18:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 shutdown_connections 2023-10-18T12:18:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2001057c0 0x7fb20019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e806cfc0 0x7fb1e806f480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2001008a0 0x7fb200198f40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 --2- 172.21.15.70:0/171869447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb200101260 0x7fb200199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 >> 172.21.15.70:0/171869447 conn(0x7fb2000779e0 msgr2=0x7fb20010ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 shutdown_connections 2023-10-18T12:18:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.147+0000 7fb206431700 1 -- 172.21.15.70:0/171869447 wait complete. 2023-10-18T12:18:32.172 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:31 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:31.828+0000 7fc66c8d3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:18:32.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 -- 172.21.15.70:0/3877365941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c401067a0 msgr2=0x7f1c40106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3877365941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c401067a0 0x7f1c40106b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009990 tx=0x7f1c30009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 -- 172.21.15.70:0/3877365941 shutdown_connections 2023-10-18T12:18:32.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3877365941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 0x7f1c4010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3877365941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c401067a0 0x7f1c40106b80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3877365941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c40068190 0x7f1c400685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 -- 172.21.15.70:0/3877365941 >> 172.21.15.70:0/3877365941 conn(0x7f1c40075790 msgr2=0x7f1c40075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:32.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.060+0000 7f1c4672b700 1 -- 172.21.15.70:0/3877365941 shutdown_connections 2023-10-18T12:18:32.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.061+0000 7f1c4672b700 1 -- 172.21.15.70:0/3877365941 wait complete. 2023-10-18T12:18:32.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 Processor -- start 2023-10-18T12:18:32.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 -- start start 2023-10-18T12:18:32.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 0x7f1c40198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:32.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 0x7f1c401993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:32.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c401067a0 0x7f1c4019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:32.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019dd60 con 0x7f1c401067a0 2023-10-18T12:18:32.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019ded0 con 0x7f1c40068190 2023-10-18T12:18:32.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c4672b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019e090 con 0x7f1c40068b30 2023-10-18T12:18:32.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.062+0000 7f1c3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 0x7f1c40198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:32.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 0x7f1c40198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:55456/0 (socket says 172.21.15.70:55456) 2023-10-18T12:18:32.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 -- 172.21.15.70:0/3973998438 learned_addr learned my addr 172.21.15.70:0/3973998438 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:32.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c44cc8700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c401067a0 0x7f1c4019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:32.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3f7fe700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 0x7f1c401993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:32.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 -- 172.21.15.70:0/3973998438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 msgr2=0x7f1c401993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 0x7f1c401993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 -- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c401067a0 msgr2=0x7f1c4019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c401067a0 0x7f1c4019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 -- 172.21.15.70:0/3973998438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300095f0 con 0x7f1c40068190 2023-10-18T12:18:32.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3f7fe700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 0x7f1c401993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:18:32.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c44cc8700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c401067a0 0x7f1c4019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:18:32.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3ffff700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 0x7f1c40198eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c30000c00 tx=0x7f1c30005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:32.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.063+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3001d070 con 0x7f1c40068190 2023-10-18T12:18:32.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.064+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c30022470 con 0x7f1c40068190 2023-10-18T12:18:32.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.064+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30017650 con 0x7f1c40068190 2023-10-18T12:18:32.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.064+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4019e310 con 0x7f1c40068190 2023-10-18T12:18:32.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.064+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c401a5a00 con 0x7f1c40068190 2023-10-18T12:18:32.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.066+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c400fed60 con 0x7f1c40068190 2023-10-18T12:18:32.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.067+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7f1c300225e0 con 0x7f1c40068190 2023-10-18T12:18:32.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.068+0000 7f1c3d7fa700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c2806cfc0 0x7f1c2806f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:32.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.068+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1c3008c570 con 0x7f1c40068190 2023-10-18T12:18:32.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.068+0000 7f1c3f7fe700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c2806cfc0 0x7f1c2806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:32.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.070+0000 7f1c3f7fe700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c2806cfc0 0x7f1c2806f480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c4019a3c0 tx=0x7f1c34009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:32.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.072+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1c30057970 con 0x7f1c40068190 2023-10-18T12:18:32.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.312+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f1c40101ee0 con 0x7f1c40068190 2023-10-18T12:18:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.317+0000 7f1c3d7fa700 1 -- 172.21.15.70:0/3973998438 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v20)=0 v20) v1 ==== 119+0+0 (secure 0 0 0) 0x7f1c3005af90 con 0x7f1c40068190 2023-10-18T12:18:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.319+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c2806cfc0 msgr2=0x7f1c2806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.319+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c2806cfc0 0x7f1c2806f480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c4019a3c0 tx=0x7f1c34009260 comp rx=0 tx=0).stop 2023-10-18T12:18:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.319+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 msgr2=0x7f1c40198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.319+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 0x7f1c40198eb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c30000c00 tx=0x7f1c30005740 comp rx=0 tx=0).stop 2023-10-18T12:18:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 shutdown_connections 2023-10-18T12:18:32.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c401067a0 0x7f1c4019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c2806cfc0 0x7f1c2806f480 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c40068190 0x7f1c40198eb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 --2- 172.21.15.70:0/3973998438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c40068b30 0x7f1c401993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:32.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 >> 172.21.15.70:0/3973998438 conn(0x7f1c40075790 msgr2=0x7f1c4010bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:32.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 shutdown_connections 2023-10-18T12:18:32.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:32.320+0000 7f1c4672b700 1 -- 172.21.15.70:0/3973998438 wait complete. 2023-10-18T12:18:33.153 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:18:32 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y[137138]: 2023-10-18T12:18:32.689+0000 7fc66c8d3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:18:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: Standby manager daemon y started 2023-10-18T12:18:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-18T12:18:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T12:18:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-18T12:18:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:33 smithi070 ceph-mon[98639]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T12:18:33.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: Standby manager daemon y started 2023-10-18T12:18:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-18T12:18:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T12:18:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-18T12:18:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:33 smithi152 ceph-mon[134767]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T12:18:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: Standby manager daemon y started 2023-10-18T12:18:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-18T12:18:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T12:18:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-18T12:18:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:33 smithi186 ceph-mon[135253]: from='mgr.? 172.21.15.152:0/3425468143' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T12:18:34.516 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:34 smithi070 ceph-mon[98639]: mgrmap e19: x(active, since 19s), standbys: z, y 2023-10-18T12:18:34.517 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-18T12:18:34.517 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:34 smithi070 ceph-mon[98639]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:34.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:34 smithi152 ceph-mon[134767]: mgrmap e19: x(active, since 19s), standbys: z, y 2023-10-18T12:18:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-18T12:18:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:34 smithi152 ceph-mon[134767]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:34.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:34 smithi186 ceph-mon[135253]: mgrmap e19: x(active, since 19s), standbys: z, y 2023-10-18T12:18:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-18T12:18:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:34 smithi186 ceph-mon[135253]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:35.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/261597558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381057c0 msgr2=0x7fdd38105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/261597558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381057c0 0x7fdd38105ba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009990 tx=0x7fdd34009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/261597558 shutdown_connections 2023-10-18T12:18:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/261597558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd38101260 0x7fdd3810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/261597558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381057c0 0x7fdd38105ba0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/261597558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd381008a0 0x7fdd38100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/261597558 >> 172.21.15.70:0/261597558 conn(0x7fdd380779e0 msgr2=0x7fdd38077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/261597558 shutdown_connections 2023-10-18T12:18:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.950+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/261597558 wait complete. 2023-10-18T12:18:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 Processor -- start 2023-10-18T12:18:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 -- start start 2023-10-18T12:18:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 0x7fdd38072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd38101260 0x7fdd38073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd381057c0 0x7fdd38073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381a5030 con 0x7fdd381008a0 2023-10-18T12:18:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381a51a0 con 0x7fdd381057c0 2023-10-18T12:18:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.951+0000 7fdd3fa3e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381a5360 con 0x7fdd38101260 2023-10-18T12:18:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 0x7fdd38072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3cfd9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd38101260 0x7fdd38073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 0x7fdd38072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36366/0 (socket says 172.21.15.70:36366) 2023-10-18T12:18:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 -- 172.21.15.70:0/2346826578 learned_addr learned my addr 172.21.15.70:0/2346826578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3dfdb700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd381057c0 0x7fdd38073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 -- 172.21.15.70:0/2346826578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd38101260 msgr2=0x7fdd38073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd38101260 0x7fdd38073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 -- 172.21.15.70:0/2346826578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd381057c0 msgr2=0x7fdd38073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd381057c0 0x7fdd38073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 -- 172.21.15.70:0/2346826578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340095f0 con 0x7fdd381008a0 2023-10-18T12:18:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3d7da700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 0x7fdd38072af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd340095c0 tx=0x7fdd3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3401d070 con 0x7fdd381008a0 2023-10-18T12:18:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd34005050 con 0x7fdd381008a0 2023-10-18T12:18:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004a70 con 0x7fdd381008a0 2023-10-18T12:18:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.952+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd381a55e0 con 0x7fdd381008a0 2023-10-18T12:18:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.953+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd381a59a0 con 0x7fdd381008a0 2023-10-18T12:18:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.954+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd38108d80 con 0x7fdd381008a0 2023-10-18T12:18:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.956+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fdd340051c0 con 0x7fdd381008a0 2023-10-18T12:18:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.958+0000 7fdd2e7fc700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdd240a1440 0x7fdd240a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.958+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdd340cbf00 con 0x7fdd381008a0 2023-10-18T12:18:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.960+0000 7fdd3cfd9700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdd240a1440 0x7fdd240a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.960+0000 7fdd3cfd9700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdd240a1440 0x7fdd240a3900 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd380741d0 tx=0x7fdd280073d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:34.963+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd34097150 con 0x7fdd381008a0 2023-10-18T12:18:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.180+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fdd38101f30 con 0x7fdd381008a0 2023-10-18T12:18:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.185+0000 7fdd2e7fc700 1 -- 172.21.15.70:0/2346826578 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fdd3409a770 con 0x7fdd381008a0 2023-10-18T12:18:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdd240a1440 msgr2=0x7fdd240a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdd240a1440 0x7fdd240a3900 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd380741d0 tx=0x7fdd280073d0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 msgr2=0x7fdd38072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 0x7fdd38072af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd340095c0 tx=0x7fdd3400f690 comp rx=0 tx=0).stop 2023-10-18T12:18:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 shutdown_connections 2023-10-18T12:18:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdd240a1440 0x7fdd240a3900 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd381057c0 0x7fdd38073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd381008a0 0x7fdd38072af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 --2- 172.21.15.70:0/2346826578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd38101260 0x7fdd38073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 >> 172.21.15.70:0/2346826578 conn(0x7fdd380779e0 msgr2=0x7fdd380fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.187+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 shutdown_connections 2023-10-18T12:18:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.188+0000 7fdd3fa3e700 1 -- 172.21.15.70:0/2346826578 wait complete. 2023-10-18T12:18:35.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 -- 172.21.15.70:0/3221176143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8072f60 msgr2=0x7fe1e80733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 --2- 172.21.15.70:0/3221176143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8072f60 0x7fe1e80733e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0098f0 tx=0x7fe1dc009c00 comp rx=0 tx=0).stop 2023-10-18T12:18:35.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 -- 172.21.15.70:0/3221176143 shutdown_connections 2023-10-18T12:18:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 --2- 172.21.15.70:0/3221176143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 0x7fe1e81139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 --2- 172.21.15.70:0/3221176143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8074770 0x7fe1e8072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 --2- 172.21.15.70:0/3221176143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8072f60 0x7fe1e80733e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 -- 172.21.15.70:0/3221176143 >> 172.21.15.70:0/3221176143 conn(0x7fe1e80779e0 msgr2=0x7fe1e8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 -- 172.21.15.70:0/3221176143 shutdown_connections 2023-10-18T12:18:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.473+0000 7fe1ee827700 1 -- 172.21.15.70:0/3221176143 wait complete. 2023-10-18T12:18:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.474+0000 7fe1ee827700 1 Processor -- start 2023-10-18T12:18:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.475+0000 7fe1ee827700 1 -- start start 2023-10-18T12:18:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.475+0000 7fe1ee827700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 0x7fe1e819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.475+0000 7fe1ee827700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 0x7fe1e819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.475+0000 7fe1ee827700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 0x7fe1e81a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1e7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 0x7fe1e819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1e7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 0x7fe1e819d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36376/0 (socket says 172.21.15.70:36376) 2023-10-18T12:18:35.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1e77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 0x7fe1e819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1e77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 0x7fe1e819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41458/0 (socket says 172.21.15.70:41458) 2023-10-18T12:18:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1e7fff700 1 -- 172.21.15.70:0/2139734276 learned_addr learned my addr 172.21.15.70:0/2139734276 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81a2210 con 0x7fe1e8072f60 2023-10-18T12:18:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81a2380 con 0x7fe1e8111570 2023-10-18T12:18:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.476+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81a2540 con 0x7fe1e8074770 2023-10-18T12:18:35.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1ecdc4700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 0x7fe1e81a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e77fe700 1 -- 172.21.15.70:0/2139734276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 msgr2=0x7fe1e81a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e77fe700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 0x7fe1e81a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e77fe700 1 -- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 msgr2=0x7fe1e819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e77fe700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 0x7fe1e819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e77fe700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009520 con 0x7fe1e8074770 2023-10-18T12:18:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1ecdc4700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 0x7fe1e81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:18:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e7fff700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 0x7fe1e819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e77fe700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 0x7fe1e819d8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009fd0 tx=0x7fe1dc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc01c070 con 0x7fe1e8074770 2023-10-18T12:18:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc009520 con 0x7fe1e8074770 2023-10-18T12:18:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1dc00bb20 con 0x7fe1e8074770 2023-10-18T12:18:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.477+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0038e0 con 0x7fe1e8074770 2023-10-18T12:18:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.478+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e81a9eb0 con 0x7fe1e8074770 2023-10-18T12:18:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.479+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e80740b0 con 0x7fe1e8074770 2023-10-18T12:18:35.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.480+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fe1dc020030 con 0x7fe1e8074770 2023-10-18T12:18:35.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.481+0000 7fe1e57fa700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1d00a14c0 0x7fe1d00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:35.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.482+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe1dc0cbf30 con 0x7fe1e8074770 2023-10-18T12:18:35.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.482+0000 7fe1e7fff700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1d00a14c0 0x7fe1d00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:35.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.482+0000 7fe1e7fff700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1d00a14c0 0x7fe1d00a3980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009fd0 tx=0x7fe1d8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:35.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.486+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1dc097180 con 0x7fe1e8074770 2023-10-18T12:18:35.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.695+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fe1e819e620 con 0x7fe1e8074770 2023-10-18T12:18:35.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.701+0000 7fe1e57fa700 1 -- 172.21.15.70:0/2139734276 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe1dc09a7a0 con 0x7fe1e8074770 2023-10-18T12:18:35.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1d00a14c0 msgr2=0x7fe1d00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1d00a14c0 0x7fe1d00a3980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009fd0 tx=0x7fe1d8009260 comp rx=0 tx=0).stop 2023-10-18T12:18:35.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 msgr2=0x7fe1e819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:35.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 0x7fe1e819d8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009fd0 tx=0x7fe1dc00fa00 comp rx=0 tx=0).stop 2023-10-18T12:18:35.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 shutdown_connections 2023-10-18T12:18:35.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1d00a14c0 0x7fe1d00a3980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1e8111570 0x7fe1e81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8072f60 0x7fe1e819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 --2- 172.21.15.70:0/2139734276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1e8074770 0x7fe1e819d8a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:35.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 >> 172.21.15.70:0/2139734276 conn(0x7fe1e80779e0 msgr2=0x7fe1e81101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:35.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 shutdown_connections 2023-10-18T12:18:35.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:35.703+0000 7fe1ee827700 1 -- 172.21.15.70:0/2139734276 wait complete. 2023-10-18T12:18:36.292 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:36 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2346826578' entity='client.admin' 2023-10-18T12:18:36.292 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:36 smithi070 ceph-mon[98639]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:36.293 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:36 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:36.431 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi070') 2023-10-18T12:18:36.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:36 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2346826578' entity='client.admin' 2023-10-18T12:18:36.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:36 smithi186 ceph-mon[135253]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:36.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:36 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:36 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2346826578' entity='client.admin' 2023-10-18T12:18:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:36 smithi152 ceph-mon[134767]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:36 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:37.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 -- 172.21.15.70:0/1102158822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c0073340 msgr2=0x7f83c010cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:37.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 --2- 172.21.15.70:0/1102158822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c0073340 0x7f83c010cd10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009990 tx=0x7f83b0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 -- 172.21.15.70:0/1102158822 shutdown_connections 2023-10-18T12:18:37.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 --2- 172.21.15.70:0/1102158822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83c010d250 0x7f83c010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 --2- 172.21.15.70:0/1102158822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0072a20 0x7f83c0072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 --2- 172.21.15.70:0/1102158822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c0073340 0x7f83c010cd10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 -- 172.21.15.70:0/1102158822 >> 172.21.15.70:0/1102158822 conn(0x7f83c00779e0 msgr2=0x7f83c0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:37.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 -- 172.21.15.70:0/1102158822 shutdown_connections 2023-10-18T12:18:37.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.930+0000 7f83c7c12700 1 -- 172.21.15.70:0/1102158822 wait complete. 2023-10-18T12:18:37.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 Processor -- start 2023-10-18T12:18:37.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 -- start start 2023-10-18T12:18:37.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0072a20 0x7f83c0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:37.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83c0073340 0x7f83c0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:37.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 0x7f83c019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:37.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c019ddb0 con 0x7f83c010d250 2023-10-18T12:18:37.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c019df20 con 0x7f83c0073340 2023-10-18T12:18:37.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.932+0000 7f83c7c12700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c019e0e0 con 0x7f83c0072a20 2023-10-18T12:18:37.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 0x7f83c019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:37.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c51ad700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83c0073340 0x7f83c0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:37.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 0x7f83c019d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36380/0 (socket says 172.21.15.70:36380) 2023-10-18T12:18:37.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 -- 172.21.15.70:0/985090240 learned_addr learned my addr 172.21.15.70:0/985090240 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:37.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c59ae700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0072a20 0x7f83c0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:37.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 -- 172.21.15.70:0/985090240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0072a20 msgr2=0x7f83c0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:37.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0072a20 0x7f83c0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 -- 172.21.15.70:0/985090240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83c0073340 msgr2=0x7f83c0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:37.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83c0073340 0x7f83c0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 -- 172.21.15.70:0/985090240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00095f0 con 0x7f83c010d250 2023-10-18T12:18:37.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c61af700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 0x7f83c019d7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83b800efc0 tx=0x7f83b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:37.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8018070 con 0x7f83c010d250 2023-10-18T12:18:37.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b800f040 con 0x7f83c010d250 2023-10-18T12:18:37.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800cd00 con 0x7f83c010d250 2023-10-18T12:18:37.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.933+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c019e360 con 0x7f83c010d250 2023-10-18T12:18:37.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.934+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c01a5c20 con 0x7f83c010d250 2023-10-18T12:18:37.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.935+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c0108460 con 0x7f83c010d250 2023-10-18T12:18:37.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.936+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f83b8003a00 con 0x7f83c010d250 2023-10-18T12:18:37.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.938+0000 7f83b6ffd700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83ac0a14c0 0x7f83ac0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:37.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.938+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f83b80cab50 con 0x7f83c010d250 2023-10-18T12:18:37.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.938+0000 7f83c59ae700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83ac0a14c0 0x7f83ac0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:37.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.938+0000 7f83c59ae700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83ac0a14c0 0x7f83ac0a3980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009c60 tx=0x7f83bc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:37.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:36.942+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f83b80cce60 con 0x7f83c010d250 2023-10-18T12:18:37.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.165+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f83c00740b0 con 0x7f83c010d250 2023-10-18T12:18:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.169+0000 7f83b6ffd700 1 -- 172.21.15.70:0/985090240 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v23)=0 v23) v1 ==== 133+0+0 (secure 0 0 0) 0x7f83b8095da0 con 0x7f83c010d250 2023-10-18T12:18:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83ac0a14c0 msgr2=0x7f83ac0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83ac0a14c0 0x7f83ac0a3980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009c60 tx=0x7f83bc009260 comp rx=0 tx=0).stop 2023-10-18T12:18:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 msgr2=0x7f83c019d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 0x7f83c019d7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83b800efc0 tx=0x7f83b800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 shutdown_connections 2023-10-18T12:18:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83c010d250 0x7f83c019d7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0072a20 0x7f83c0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83ac0a14c0 0x7f83ac0a3980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 --2- 172.21.15.70:0/985090240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83c0073340 0x7f83c0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.172+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 >> 172.21.15.70:0/985090240 conn(0x7f83c00779e0 msgr2=0x7f83c00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.173+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 shutdown_connections 2023-10-18T12:18:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:37.173+0000 7f83c7c12700 1 -- 172.21.15.70:0/985090240 wait complete. 2023-10-18T12:18:38.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1905373718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81008a0 msgr2=0x7f5ae8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1905373718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81008a0 0x7f5ae8100d20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009990 tx=0x7f5ad8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1905373718 shutdown_connections 2023-10-18T12:18:38.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1905373718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ae8101260 0x7f5ae810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1905373718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ae81057c0 0x7f5ae8105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1905373718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81008a0 0x7f5ae8100d20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1905373718 >> 172.21.15.70:0/1905373718 conn(0x7f5ae80779e0 msgr2=0x7f5ae8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:38.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.027+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1905373718 shutdown_connections 2023-10-18T12:18:38.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.028+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1905373718 wait complete. 2023-10-18T12:18:38.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 Processor -- start 2023-10-18T12:18:38.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 -- start start 2023-10-18T12:18:38.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ae81008a0 0x7f5ae8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ae8101260 0x7f5ae8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 0x7f5ae819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819de10 con 0x7f5ae81057c0 2023-10-18T12:18:38.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819df80 con 0x7f5ae81008a0 2023-10-18T12:18:38.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aeff6d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819e140 con 0x7f5ae8101260 2023-10-18T12:18:38.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aee50a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 0x7f5ae819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aee50a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 0x7f5ae819d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35054/0 (socket says 172.21.15.70:35054) 2023-10-18T12:18:38.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aee50a700 1 -- 172.21.15.70:0/1393590600 learned_addr learned my addr 172.21.15.70:0/1393590600 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:38.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aed508700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ae8101260 0x7f5ae8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.029+0000 7f5aedd09700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ae81008a0 0x7f5ae8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aee50a700 1 -- 172.21.15.70:0/1393590600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ae8101260 msgr2=0x7f5ae8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aee50a700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ae8101260 0x7f5ae8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aee50a700 1 -- 172.21.15.70:0/1393590600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ae81008a0 msgr2=0x7f5ae8198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aee50a700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ae81008a0 0x7f5ae8198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aee50a700 1 -- 172.21.15.70:0/1393590600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80095f0 con 0x7f5ae81057c0 2023-10-18T12:18:38.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aee50a700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 0x7f5ae819d800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000efc0 tx=0x7f5ae000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:38.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0018070 con 0x7f5ae81057c0 2023-10-18T12:18:38.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae000f040 con 0x7f5ae81057c0 2023-10-18T12:18:38.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae819e3c0 con 0x7f5ae81057c0 2023-10-18T12:18:38.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000cc10 con 0x7f5ae81057c0 2023-10-18T12:18:38.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.030+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81a5c80 con 0x7f5ae81057c0 2023-10-18T12:18:38.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.031+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8101f30 con 0x7f5ae81057c0 2023-10-18T12:18:38.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.033+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f5ae0003a00 con 0x7f5ae81057c0 2023-10-18T12:18:38.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.035+0000 7f5adeffd700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5ad40a14c0 0x7f5ad40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.036+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5ae00cab00 con 0x7f5ae81057c0 2023-10-18T12:18:38.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.036+0000 7f5aedd09700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5ad40a14c0 0x7f5ad40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.036+0000 7f5aedd09700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5ad40a14c0 0x7f5ad40a3980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8101930 tx=0x7f5ae4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:38.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.041+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5ae0095d50 con 0x7f5ae81057c0 2023-10-18T12:18:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.322+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f5ae819a220 con 0x7f5ae81057c0 2023-10-18T12:18:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.326+0000 7f5adeffd700 1 -- 172.21.15.70:0/1393590600 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5ae0099370 con 0x7f5ae81057c0 2023-10-18T12:18:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5ad40a14c0 msgr2=0x7f5ad40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5ad40a14c0 0x7f5ad40a3980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8101930 tx=0x7f5ae4009260 comp rx=0 tx=0).stop 2023-10-18T12:18:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 msgr2=0x7f5ae819d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 0x7f5ae819d800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000efc0 tx=0x7f5ae000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 shutdown_connections 2023-10-18T12:18:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ae81057c0 0x7f5ae819d800 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5ad40a14c0 0x7f5ad40a3980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ae81008a0 0x7f5ae8198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 --2- 172.21.15.70:0/1393590600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ae8101260 0x7f5ae8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 >> 172.21.15.70:0/1393590600 conn(0x7f5ae80779e0 msgr2=0x7f5ae810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 shutdown_connections 2023-10-18T12:18:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.330+0000 7f5aeff6d700 1 -- 172.21.15.70:0/1393590600 wait complete. 2023-10-18T12:18:38.336 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/985090240' entity='client.admin' 2023-10-18T12:18:38.336 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:38 smithi070 ceph-mon[98639]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:38.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.341+0000 7f154b288700 1 -- 172.21.15.70:0/1415293833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544072f60 msgr2=0x7f15440733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.341+0000 7f154b288700 1 --2- 172.21.15.70:0/1415293833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544072f60 0x7f15440733e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1534009990 tx=0x7f1534009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 -- 172.21.15.70:0/1415293833 shutdown_connections 2023-10-18T12:18:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 --2- 172.21.15.70:0/1415293833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1544111570 0x7f15441139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 --2- 172.21.15.70:0/1415293833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544074770 0x7f1544072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 --2- 172.21.15.70:0/1415293833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544072f60 0x7f15440733e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 -- 172.21.15.70:0/1415293833 >> 172.21.15.70:0/1415293833 conn(0x7f15440779e0 msgr2=0x7f1544077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 -- 172.21.15.70:0/1415293833 shutdown_connections 2023-10-18T12:18:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.342+0000 7f154b288700 1 -- 172.21.15.70:0/1415293833 wait complete. 2023-10-18T12:18:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 Processor -- start 2023-10-18T12:18:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 -- start start 2023-10-18T12:18:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 0x7f154419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544074770 0x7f154419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1544111570 0x7f15441a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15441a2180 con 0x7f1544074770 2023-10-18T12:18:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.344+0000 7f154b288700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15441a22f0 con 0x7f1544072f60 2023-10-18T12:18:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f154b288700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15441a24b0 con 0x7f1544111570 2023-10-18T12:18:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 0x7f154419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 0x7f154419d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43650/0 (socket says 172.21.15.70:43650) 2023-10-18T12:18:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 -- 172.21.15.70:0/2423554126 learned_addr learned my addr 172.21.15.70:0/2423554126 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549825700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1544111570 0x7f15441a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1548823700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544074770 0x7f154419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 -- 172.21.15.70:0/2423554126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1544111570 msgr2=0x7f15441a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1544111570 0x7f15441a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 -- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544074770 msgr2=0x7f154419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544074770 0x7f154419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.345+0000 7f1549024700 1 -- 172.21.15.70:0/2423554126 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540009520 con 0x7f1544072f60 2023-10-18T12:18:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.346+0000 7f1549024700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 0x7f154419d280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f154000eb30 tx=0x7f154000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.346+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154000cd40 con 0x7f1544072f60 2023-10-18T12:18:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.346+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154000cea0 con 0x7f1544072f60 2023-10-18T12:18:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.346+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540010430 con 0x7f1544072f60 2023-10-18T12:18:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.347+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15340095f0 con 0x7f1544072f60 2023-10-18T12:18:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.347+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154406a210 con 0x7f1544072f60 2023-10-18T12:18:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.349+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15440740b0 con 0x7f1544072f60 2023-10-18T12:18:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.352+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f1540003f40 con 0x7f1544072f60 2023-10-18T12:18:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.353+0000 7f153a7fc700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15300a14c0 0x7f15300a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.353+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1540014070 con 0x7f1544072f60 2023-10-18T12:18:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.353+0000 7f1548823700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15300a14c0 0x7f15300a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.354+0000 7f1548823700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15300a14c0 0x7f15300a3980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f153400b2e0 tx=0x7f1534005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.357+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f15400968b0 con 0x7f1544072f60 2023-10-18T12:18:38.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/985090240' entity='client.admin' 2023-10-18T12:18:38.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:38 smithi186 ceph-mon[135253]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:38.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.578+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f154404e4c0 con 0x7f1544072f60 2023-10-18T12:18:38.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.585+0000 7f153a7fc700 1 -- 172.21.15.70:0/2423554126 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v26)=0 v26) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1540099ed0 con 0x7f1544072f60 2023-10-18T12:18:38.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15300a14c0 msgr2=0x7f15300a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15300a14c0 0x7f15300a3980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f153400b2e0 tx=0x7f1534005fd0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 msgr2=0x7f154419d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:38.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 0x7f154419d280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f154000eb30 tx=0x7f154000eef0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 shutdown_connections 2023-10-18T12:18:38.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1544111570 0x7f15441a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15300a14c0 0x7f15300a3980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1544072f60 0x7f154419d280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 --2- 172.21.15.70:0/2423554126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1544074770 0x7f154419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:38.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.587+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 >> 172.21.15.70:0/2423554126 conn(0x7f15440779e0 msgr2=0x7f15441101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:38.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.588+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 shutdown_connections 2023-10-18T12:18:38.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.588+0000 7f154b288700 1 -- 172.21.15.70:0/2423554126 wait complete. 2023-10-18T12:18:38.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/985090240' entity='client.admin' 2023-10-18T12:18:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:38 smithi152 ceph-mon[134767]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:39.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 -- 172.21.15.70:0/2241894086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 msgr2=0x7fd36010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 --2- 172.21.15.70:0/2241894086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd36010d650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd358009f90 tx=0x7fd35800b330 comp rx=0 tx=0).stop 2023-10-18T12:18:39.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 -- 172.21.15.70:0/2241894086 shutdown_connections 2023-10-18T12:18:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 --2- 172.21.15.70:0/2241894086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd36010d650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 --2- 172.21.15.70:0/2241894086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601057c0 0x7fd360105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 --2- 172.21.15.70:0/2241894086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601008a0 0x7fd360100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.260+0000 7fd36678c700 1 -- 172.21.15.70:0/2241894086 >> 172.21.15.70:0/2241894086 conn(0x7fd3600779e0 msgr2=0x7fd360077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.261+0000 7fd36678c700 1 -- 172.21.15.70:0/2241894086 shutdown_connections 2023-10-18T12:18:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.261+0000 7fd36678c700 1 -- 172.21.15.70:0/2241894086 wait complete. 2023-10-18T12:18:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.262+0000 7fd36678c700 1 Processor -- start 2023-10-18T12:18:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 -- start start 2023-10-18T12:18:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601008a0 0x7fd360198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd360199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601057c0 0x7fd36019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36019de10 con 0x7fd360101260 2023-10-18T12:18:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36019df80 con 0x7fd3601057c0 2023-10-18T12:18:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.263+0000 7fd36678c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36019e140 con 0x7fd3601008a0 2023-10-18T12:18:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd35f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd360199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601008a0 0x7fd360198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd364d29700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601057c0 0x7fd36019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd35f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd360199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35064/0 (socket says 172.21.15.70:35064) 2023-10-18T12:18:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd35f7fe700 1 -- 172.21.15.70:0/2513840942 learned_addr learned my addr 172.21.15.70:0/2513840942 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd364d29700 1 -- 172.21.15.70:0/2513840942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601008a0 msgr2=0x7fd360198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd364d29700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601008a0 0x7fd360198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd364d29700 1 -- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 msgr2=0x7fd360199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd364d29700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd360199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd364d29700 1 -- 172.21.15.70:0/2513840942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358009c40 con 0x7fd3601057c0 2023-10-18T12:18:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.264+0000 7fd35ffff700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601008a0 0x7fd360198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:18:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd35f7fe700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd360199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd364d29700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601057c0 0x7fd36019d800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd35800b6c0 tx=0x7fd3580094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800e040 con 0x7fd3601057c0 2023-10-18T12:18:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36019e3c0 con 0x7fd3601057c0 2023-10-18T12:18:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd358007e10 con 0x7fd3601057c0 2023-10-18T12:18:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358005040 con 0x7fd3601057c0 2023-10-18T12:18:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.265+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3601a5c80 con 0x7fd3601057c0 2023-10-18T12:18:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.267+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd360101f30 con 0x7fd3601057c0 2023-10-18T12:18:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.268+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fd35801b450 con 0x7fd3601057c0 2023-10-18T12:18:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.271+0000 7fd35d7fa700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3480a1440 0x7fd3480a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.271+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd3580cc420 con 0x7fd3601057c0 2023-10-18T12:18:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.271+0000 7fd35ffff700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3480a1440 0x7fd3480a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.272+0000 7fd35ffff700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3480a1440 0x7fd3480a3900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd350005e90 tx=0x7fd350005d40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.274+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd358097670 con 0x7fd3601057c0 2023-10-18T12:18:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.525+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) v1 -- 0x7fd36019a220 con 0x7fd3601057c0 2023-10-18T12:18:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.530+0000 7fd35d7fa700 1 -- 172.21.15.70:0/2513840942 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v25)=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd35809ac90 con 0x7fd3601057c0 2023-10-18T12:18:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3480a1440 msgr2=0x7fd3480a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3480a1440 0x7fd3480a3900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd350005e90 tx=0x7fd350005d40 comp rx=0 tx=0).stop 2023-10-18T12:18:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601057c0 msgr2=0x7fd36019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601057c0 0x7fd36019d800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd35800b6c0 tx=0x7fd3580094b0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 shutdown_connections 2023-10-18T12:18:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3480a1440 0x7fd3480a3900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3601057c0 0x7fd36019d800 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3601008a0 0x7fd360198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 --2- 172.21.15.70:0/2513840942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd360101260 0x7fd360199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 >> 172.21.15.70:0/2513840942 conn(0x7fd3600779e0 msgr2=0x7fd36010ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 shutdown_connections 2023-10-18T12:18:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.533+0000 7fd36678c700 1 -- 172.21.15.70:0/2513840942 wait complete. 2023-10-18T12:18:39.563 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:39 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1393590600' entity='client.admin' 2023-10-18T12:18:39.563 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:39 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:39.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:39 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:39.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:39 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/408736202' entity='client.admin' 2023-10-18T12:18:39.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 -- 172.21.15.70:0/1494806184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc1067a0 msgr2=0x7fd5bc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 --2- 172.21.15.70:0/1494806184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc106b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009990 tx=0x7fd5ac009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 -- 172.21.15.70:0/1494806184 shutdown_connections 2023-10-18T12:18:39.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 --2- 172.21.15.70:0/1494806184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc068b30 0x7fd5bc10d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 --2- 172.21.15.70:0/1494806184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc106b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 --2- 172.21.15.70:0/1494806184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068190 0x7fd5bc0685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 -- 172.21.15.70:0/1494806184 >> 172.21.15.70:0/1494806184 conn(0x7fd5bc075790 msgr2=0x7fd5bc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 -- 172.21.15.70:0/1494806184 shutdown_connections 2023-10-18T12:18:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.928+0000 7fd5c251f700 1 -- 172.21.15.70:0/1494806184 wait complete. 2023-10-18T12:18:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.929+0000 7fd5c251f700 1 Processor -- start 2023-10-18T12:18:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 -- start start 2023-10-18T12:18:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc068190 0x7fd5bc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068b30 0x7fd5bc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc19dd10 con 0x7fd5bc1067a0 2023-10-18T12:18:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc19de80 con 0x7fd5bc068190 2023-10-18T12:18:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5c251f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc19e040 con 0x7fd5bc068b30 2023-10-18T12:18:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.930+0000 7fd5bb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068b30 0x7fd5bc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5bb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068b30 0x7fd5bc199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:51284/0 (socket says 172.21.15.70:51284) 2023-10-18T12:18:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5bb7fe700 1 -- 172.21.15.70:0/408736202 learned_addr learned my addr 172.21.15.70:0/408736202 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5bbfff700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc068190 0x7fd5bc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 -- 172.21.15.70:0/408736202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068b30 msgr2=0x7fd5bc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068b30 0x7fd5bc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 -- 172.21.15.70:0/408736202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc068190 msgr2=0x7fd5bc198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc068190 0x7fd5bc198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 -- 172.21.15.70:0/408736202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac0095f0 con 0x7fd5bc1067a0 2023-10-18T12:18:39.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5c0abc700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc19d700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4007f10 tx=0x7fd5b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:39.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.931+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4009ea0 con 0x7fd5bc1067a0 2023-10-18T12:18:39.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.932+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b400f040 con 0x7fd5bc1067a0 2023-10-18T12:18:39.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.932+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc19e320 con 0x7fd5bc1067a0 2023-10-18T12:18:39.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.932+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4004350 con 0x7fd5bc1067a0 2023-10-18T12:18:39.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.932+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc1a5b80 con 0x7fd5bc1067a0 2023-10-18T12:18:39.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.933+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc0fed60 con 0x7fd5bc1067a0 2023-10-18T12:18:39.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.935+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fd5b400db20 con 0x7fd5bc1067a0 2023-10-18T12:18:39.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.936+0000 7fd5b97fa700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5a40a14c0 0x7fd5a40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.937+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd5b40cb2f0 con 0x7fd5bc1067a0 2023-10-18T12:18:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.937+0000 7fd5bbfff700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5a40a14c0 0x7fd5a40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.937+0000 7fd5bbfff700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5a40a14c0 0x7fd5a40a3980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098e0 tx=0x7fd5ac005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:38.939+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5b4096550 con 0x7fd5bc1067a0 2023-10-18T12:18:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.155+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fd5bc04e530 con 0x7fd5bc1067a0 2023-10-18T12:18:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.159+0000 7fd5b97fa700 1 -- 172.21.15.70:0/408736202 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v27)=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd5bc04e530 con 0x7fd5bc1067a0 2023-10-18T12:18:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5a40a14c0 msgr2=0x7fd5a40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5a40a14c0 0x7fd5a40a3980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098e0 tx=0x7fd5ac005960 comp rx=0 tx=0).stop 2023-10-18T12:18:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc1067a0 msgr2=0x7fd5bc19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc19d700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4007f10 tx=0x7fd5b400d3b0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 shutdown_connections 2023-10-18T12:18:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5bc1067a0 0x7fd5bc19d700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5a40a14c0 0x7fd5a40a3980 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5bc068190 0x7fd5bc198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 --2- 172.21.15.70:0/408736202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5bc068b30 0x7fd5bc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:39.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 >> 172.21.15.70:0/408736202 conn(0x7fd5bc075790 msgr2=0x7fd5bc10bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:39.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 shutdown_connections 2023-10-18T12:18:39.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:39.162+0000 7fd5c251f700 1 -- 172.21.15.70:0/408736202 wait complete. 2023-10-18T12:18:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:39 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1393590600' entity='client.admin' 2023-10-18T12:18:39.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:39 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:39.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:39 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:39.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:39 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/408736202' entity='client.admin' 2023-10-18T12:18:39.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:39 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1393590600' entity='client.admin' 2023-10-18T12:18:39.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:39 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:39.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:39 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:39.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:39 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/408736202' entity='client.admin' 2023-10-18T12:18:40.556 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:40.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:40.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:40.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:40.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 -- 172.21.15.70:0/1048004590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41008a0 msgr2=0x7fcfa4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/1048004590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41008a0 0x7fcfa4100d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcf980098f0 tx=0x7fcf98009c00 comp rx=0 tx=0).stop 2023-10-18T12:18:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 -- 172.21.15.70:0/1048004590 shutdown_connections 2023-10-18T12:18:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/1048004590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 0x7fcfa410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/1048004590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa41057c0 0x7fcfa4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/1048004590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41008a0 0x7fcfa4100d20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 -- 172.21.15.70:0/1048004590 >> 172.21.15.70:0/1048004590 conn(0x7fcfa40779e0 msgr2=0x7fcfa4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 -- 172.21.15.70:0/1048004590 shutdown_connections 2023-10-18T12:18:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.644+0000 7fcfa96d8700 1 -- 172.21.15.70:0/1048004590 wait complete. 2023-10-18T12:18:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.645+0000 7fcfa96d8700 1 Processor -- start 2023-10-18T12:18:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.645+0000 7fcfa96d8700 1 -- start start 2023-10-18T12:18:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.646+0000 7fcfa96d8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa41008a0 0x7fcfa4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.646+0000 7fcfa96d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 0x7fcfa4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.646+0000 7fcfa96d8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41057c0 0x7fcfa4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.646+0000 7fcfa96d8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa41a4fc0 con 0x7fcfa4101260 2023-10-18T12:18:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.646+0000 7fcfa96d8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa41a5130 con 0x7fcfa41008a0 2023-10-18T12:18:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.646+0000 7fcfa96d8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa41a52f0 con 0x7fcfa41057c0 2023-10-18T12:18:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41057c0 0x7fcfa4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41057c0 0x7fcfa4073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:51318/0 (socket says 172.21.15.70:51318) 2023-10-18T12:18:40.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa37fe700 1 -- 172.21.15.70:0/3483235092 learned_addr learned my addr 172.21.15.70:0/3483235092 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:40.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa27fc700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 0x7fcfa4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa27fc700 1 -- 172.21.15.70:0/3483235092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41057c0 msgr2=0x7fcfa4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa27fc700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41057c0 0x7fcfa4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa27fc700 1 -- 172.21.15.70:0/3483235092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa41008a0 msgr2=0x7fcfa4072af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T12:18:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa27fc700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa41008a0 0x7fcfa4072af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.647+0000 7fcfa27fc700 1 -- 172.21.15.70:0/3483235092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c009d00 con 0x7fcfa4101260 2023-10-18T12:18:40.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.648+0000 7fcfa27fc700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 0x7fcfa4073030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009230 tx=0x7fcf98010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.648+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9801c070 con 0x7fcfa4101260 2023-10-18T12:18:40.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.648+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf9800bbe0 con 0x7fcfa4101260 2023-10-18T12:18:40.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.648+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980038c0 con 0x7fcfa4101260 2023-10-18T12:18:40.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.648+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf98009520 con 0x7fcfa4101260 2023-10-18T12:18:40.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.648+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa41a57f0 con 0x7fcfa4101260 2023-10-18T12:18:40.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.650+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa4108da0 con 0x7fcfa4101260 2023-10-18T12:18:40.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.654+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fcf98020030 con 0x7fcfa4101260 2023-10-18T12:18:40.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.657+0000 7fcf8bfff700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcf8c0a1590 0x7fcf8c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.658+0000 7fcfa2ffd700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcf8c0a1590 0x7fcf8c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.658+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fcf980cc350 con 0x7fcfa4101260 2023-10-18T12:18:40.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.659+0000 7fcfa2ffd700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcf8c0a1590 0x7fcf8c0a3a50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4101930 tx=0x7fcf94009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.660+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcf980976c0 con 0x7fcfa4101260 2023-10-18T12:18:40.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:40.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:40.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:40.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:40.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:40.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 -- 172.21.15.70:0/859881068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19800ff750 msgr2=0x7f19800ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 --2- 172.21.15.70:0/859881068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19800ff750 0x7f19800ffbd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1974009990 tx=0x7f1974009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 -- 172.21.15.70:0/859881068 shutdown_connections 2023-10-18T12:18:40.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 --2- 172.21.15.70:0/859881068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1980100110 0x7f198010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 --2- 172.21.15.70:0/859881068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19801056b0 0x7f1980105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 --2- 172.21.15.70:0/859881068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19800ff750 0x7f19800ffbd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 -- 172.21.15.70:0/859881068 >> 172.21.15.70:0/859881068 conn(0x7f1980075790 msgr2=0x7f1980075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 -- 172.21.15.70:0/859881068 shutdown_connections 2023-10-18T12:18:40.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.836+0000 7f19854dd700 1 -- 172.21.15.70:0/859881068 wait complete. 2023-10-18T12:18:40.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.837+0000 7f19854dd700 1 Processor -- start 2023-10-18T12:18:40.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.837+0000 7f19854dd700 1 -- start start 2023-10-18T12:18:40.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f19854dd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19800ff750 0x7f198010c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f19854dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 0x7f1980109320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f19854dd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19801056b0 0x7f1980109860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f19854dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980109ec0 con 0x7f1980100110 2023-10-18T12:18:40.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f19854dd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198010a030 con 0x7f19801056b0 2023-10-18T12:18:40.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f19854dd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198010a1f0 con 0x7f19800ff750 2023-10-18T12:18:40.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.838+0000 7f197effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19800ff750 0x7f198010c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 0x7f1980109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 0x7f1980109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35198/0 (socket says 172.21.15.70:35198) 2023-10-18T12:18:40.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 -- 172.21.15.70:0/2365387627 learned_addr learned my addr 172.21.15.70:0/2365387627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:40.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197f7fe700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19801056b0 0x7f1980109860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 -- 172.21.15.70:0/2365387627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19800ff750 msgr2=0x7f198010c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19800ff750 0x7f198010c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 -- 172.21.15.70:0/2365387627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19801056b0 msgr2=0x7f1980109860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19801056b0 0x7f1980109860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197e7fc700 1 -- 172.21.15.70:0/2365387627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740095f0 con 0x7f1980100110 2023-10-18T12:18:40.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.839+0000 7f197f7fe700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19801056b0 0x7f1980109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:40.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.840+0000 7f197e7fc700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 0x7f1980109320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1974005910 tx=0x7f1974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.840+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197401d070 con 0x7f1980100110 2023-10-18T12:18:40.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.840+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19801134a0 con 0x7f1980100110 2023-10-18T12:18:40.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.840+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1980113990 con 0x7f1980100110 2023-10-18T12:18:40.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.841+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1974022470 con 0x7f1980100110 2023-10-18T12:18:40.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.841+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974017650 con 0x7f1980100110 2023-10-18T12:18:40.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.842+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c0052f0 con 0x7f1980100110 2023-10-18T12:18:40.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.844+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f197400fac0 con 0x7f1980100110 2023-10-18T12:18:40.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.845+0000 7f1967fff700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19680a14c0 0x7f19680a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.846+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f19740cc150 con 0x7f1980100110 2023-10-18T12:18:40.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.846+0000 7f197effd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19680a14c0 0x7f19680a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.846+0000 7f197effd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19680a14c0 0x7f19680a3980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1970009a10 tx=0x7f1970009190 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.854+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f197405d100 con 0x7f1980100110 2023-10-18T12:18:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.912+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7fcfa4073fd0 con 0x7fcfa4101260 2023-10-18T12:18:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.915+0000 7fcf8bfff700 1 -- 172.21.15.70:0/3483235092 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v28)=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcf9809ace0 con 0x7fcfa4101260 2023-10-18T12:18:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.918+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcf8c0a1590 msgr2=0x7fcf8c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcf8c0a1590 0x7fcf8c0a3a50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4101930 tx=0x7fcf94009260 comp rx=0 tx=0).stop 2023-10-18T12:18:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 msgr2=0x7fcfa4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 0x7fcfa4073030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009230 tx=0x7fcf98010fd0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 shutdown_connections 2023-10-18T12:18:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa41057c0 0x7fcfa4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcf8c0a1590 0x7fcf8c0a3a50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa41008a0 0x7fcfa4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 --2- 172.21.15.70:0/3483235092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfa4101260 0x7fcfa4073030 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 >> 172.21.15.70:0/3483235092 conn(0x7fcfa40779e0 msgr2=0x7fcfa40fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 shutdown_connections 2023-10-18T12:18:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.919+0000 7fcfa96d8700 1 -- 172.21.15.70:0/3483235092 wait complete. 2023-10-18T12:18:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 -- 172.21.15.70:0/314949922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 msgr2=0x7fe54410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 --2- 172.21.15.70:0/314949922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 0x7fe54410f640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00b0a0 tx=0x7fe53c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 -- 172.21.15.70:0/314949922 shutdown_connections 2023-10-18T12:18:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 --2- 172.21.15.70:0/314949922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 0x7fe54410f640 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 --2- 172.21.15.70:0/314949922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544072a20 0x7fe544072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 --2- 172.21.15.70:0/314949922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544073340 0x7fe54410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 -- 172.21.15.70:0/314949922 >> 172.21.15.70:0/314949922 conn(0x7fe5440779e0 msgr2=0x7fe544077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 -- 172.21.15.70:0/314949922 shutdown_connections 2023-10-18T12:18:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.594+0000 7fe54a648700 1 -- 172.21.15.70:0/314949922 wait complete. 2023-10-18T12:18:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.595+0000 7fe54a648700 1 Processor -- start 2023-10-18T12:18:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.595+0000 7fe54a648700 1 -- start start 2023-10-18T12:18:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe54a648700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 0x7fe544199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe54a648700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544073340 0x7fe544199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe54a648700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 0x7fe54419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe54a648700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54419df10 con 0x7fe544072a20 2023-10-18T12:18:40.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe54a648700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54419e080 con 0x7fe54410d250 2023-10-18T12:18:40.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe54a648700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54419e240 con 0x7fe544073340 2023-10-18T12:18:40.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe543fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 0x7fe544199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe543fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 0x7fe544199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35150/0 (socket says 172.21.15.70:35150) 2023-10-18T12:18:40.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe543fff700 1 -- 172.21.15.70:0/1143546355 learned_addr learned my addr 172.21.15.70:0/1143546355 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:40.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe5437fe700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544073340 0x7fe544199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.596+0000 7fe548be5700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 0x7fe54419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe543fff700 1 -- 172.21.15.70:0/1143546355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544073340 msgr2=0x7fe544199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe543fff700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544073340 0x7fe544199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe543fff700 1 -- 172.21.15.70:0/1143546355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 msgr2=0x7fe54419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe543fff700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 0x7fe54419d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe543fff700 1 -- 172.21.15.70:0/1143546355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c009d00 con 0x7fe544072a20 2023-10-18T12:18:40.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe5437fe700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544073340 0x7fe544199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:40.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe543fff700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 0x7fe544199010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9a0 tx=0x7fe53400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534009790 con 0x7fe544072a20 2023-10-18T12:18:40.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54419e520 con 0x7fe544072a20 2023-10-18T12:18:40.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe534010430 con 0x7fe544072a20 2023-10-18T12:18:40.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534018680 con 0x7fe544072a20 2023-10-18T12:18:40.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.597+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5441a5d80 con 0x7fe544072a20 2023-10-18T12:18:40.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.599+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe544108460 con 0x7fe544072a20 2023-10-18T12:18:40.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.600+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fe5340105a0 con 0x7fe544072a20 2023-10-18T12:18:40.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.601+0000 7fe5417fa700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe52c0a14c0 0x7fe52c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.602+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe534014070 con 0x7fe544072a20 2023-10-18T12:18:40.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.602+0000 7fe5437fe700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe52c0a14c0 0x7fe52c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.602+0000 7fe5437fe700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe52c0a14c0 0x7fe52c0a3980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe54419a570 tx=0x7fe538009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.606+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe5340964d0 con 0x7fe544072a20 2023-10-18T12:18:40.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.963+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fe5440740b0 con 0x7fe544072a20 2023-10-18T12:18:40.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.967+0000 7fe5417fa700 1 -- 172.21.15.70:0/1143546355 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe534099af0 con 0x7fe544072a20 2023-10-18T12:18:40.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.970+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe52c0a14c0 msgr2=0x7fe52c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.970+0000 7fe54a648700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe52c0a14c0 0x7fe52c0a3980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe54419a570 tx=0x7fe538009260 comp rx=0 tx=0).stop 2023-10-18T12:18:40.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.970+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 msgr2=0x7fe544199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.970+0000 7fe54a648700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 0x7fe544199010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9a0 tx=0x7fe53400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 -- 172.21.15.70:0/2329449242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 msgr2=0x7f3f0c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/2329449242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 0x7f3f0c100d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009990 tx=0x7f3f00009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 -- 172.21.15.70:0/2329449242 shutdown_connections 2023-10-18T12:18:40.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/2329449242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/2329449242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c1057c0 0x7f3f0c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/2329449242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 0x7f3f0c100d20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 -- 172.21.15.70:0/2329449242 >> 172.21.15.70:0/2329449242 conn(0x7f3f0c0779e0 msgr2=0x7f3f0c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 -- 172.21.15.70:0/2329449242 shutdown_connections 2023-10-18T12:18:40.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.971+0000 7f3f10e4f700 1 -- 172.21.15.70:0/2329449242 wait complete. 2023-10-18T12:18:40.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 shutdown_connections 2023-10-18T12:18:40.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe52c0a14c0 0x7fe52c0a3980 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe54410d250 0x7fe54419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544072a20 0x7fe544199010 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 --2- 172.21.15.70:0/1143546355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe544073340 0x7fe544199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 >> 172.21.15.70:0/1143546355 conn(0x7fe5440779e0 msgr2=0x7fe5440fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 shutdown_connections 2023-10-18T12:18:40.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.972+0000 7fe54a648700 1 -- 172.21.15.70:0/1143546355 wait complete. 2023-10-18T12:18:40.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.973+0000 7f3f10e4f700 1 Processor -- start 2023-10-18T12:18:40.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.973+0000 7f3f10e4f700 1 -- start start 2023-10-18T12:18:40.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f10e4f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 0x7f3f0c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f10e4f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f10e4f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c1057c0 0x7f3f0c1947e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f10e4f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c194e90 con 0x7f3f0c1008a0 2023-10-18T12:18:40.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f10e4f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c195000 con 0x7f3f0c101260 2023-10-18T12:18:40.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f10e4f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1951c0 con 0x7f3f0c1057c0 2023-10-18T12:18:40.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f09d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f09d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43720/0 (socket says 172.21.15.70:43720) 2023-10-18T12:18:40.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f09d9b700 1 -- 172.21.15.70:0/1337540800 learned_addr learned my addr 172.21.15.70:0/1337540800 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:40.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f09d9b700 1 -- 172.21.15.70:0/1337540800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c1057c0 msgr2=0x7f3f0c1947e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:18:40.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f09d9b700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c1057c0 0x7f3f0c1947e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.974+0000 7f3f09d9b700 1 -- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 msgr2=0x7f3f0c19a680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:18:40.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.975+0000 7f3f0a59c700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 0x7f3f0c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.975+0000 7f3f09d9b700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 0x7f3f0c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.975+0000 7f3f09d9b700 1 -- 172.21.15.70:0/1337540800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000095f0 con 0x7f3f0c101260 2023-10-18T12:18:40.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.975+0000 7f3f09d9b700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c19abc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f00000c00 tx=0x7f3f00003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.975+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00005700 con 0x7f3f0c101260 2023-10-18T12:18:40.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.975+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0000b9b0 con 0x7f3f0c101260 2023-10-18T12:18:40.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.976+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c195440 con 0x7f3f0c101260 2023-10-18T12:18:40.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.976+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c1a16b0 con 0x7f3f0c101260 2023-10-18T12:18:40.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.976+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000b6f0 con 0x7f3f0c101260 2023-10-18T12:18:40.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.977+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c108da0 con 0x7f3f0c101260 2023-10-18T12:18:40.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.979+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f3f0000bb20 con 0x7f3f0c101260 2023-10-18T12:18:40.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.982+0000 7f3efb7fe700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3ef40a14b0 0x7f3ef40a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.982+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3f00011e40 con 0x7f3f0c101260 2023-10-18T12:18:40.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.982+0000 7f3f0a59c700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3ef40a14b0 0x7f3ef40a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.983+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f0009bff0 con 0x7f3f0c101260 2023-10-18T12:18:40.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.983+0000 7f3f0a59c700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3ef40a14b0 0x7f3ef40a3970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c101930 tx=0x7f3efc005ee0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:40.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.991+0000 7faa2ef40700 1 -- 172.21.15.70:0/3251561621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa280686d0 msgr2=0x7faa28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:40.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.991+0000 7faa2ef40700 1 --2- 172.21.15.70:0/3251561621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa280686d0 0x7faa28068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa14009990 tx=0x7faa14009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 -- 172.21.15.70:0/3251561621 shutdown_connections 2023-10-18T12:18:40.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 --2- 172.21.15.70:0/3251561621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa28103250 0x7faa28103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 --2- 172.21.15.70:0/3251561621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa281056f0 0x7faa28068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 --2- 172.21.15.70:0/3251561621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa280686d0 0x7faa28068b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 -- 172.21.15.70:0/3251561621 >> 172.21.15.70:0/3251561621 conn(0x7faa28075790 msgr2=0x7faa28075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 -- 172.21.15.70:0/3251561621 shutdown_connections 2023-10-18T12:18:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.992+0000 7faa2ef40700 1 -- 172.21.15.70:0/3251561621 wait complete. 2023-10-18T12:18:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.993+0000 7faa2ef40700 1 Processor -- start 2023-10-18T12:18:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.994+0000 7faa2ef40700 1 -- start start 2023-10-18T12:18:40.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ef40700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa280686d0 0x7faa2819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ef40700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa28103250 0x7faa2819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ef40700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa281056f0 0x7faa281a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:40.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ef40700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281a20a0 con 0x7faa28103250 2023-10-18T12:18:40.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ef40700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281a2210 con 0x7faa281056f0 2023-10-18T12:18:40.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ef40700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281a23d0 con 0x7faa280686d0 2023-10-18T12:18:40.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ccdc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa280686d0 0x7faa2819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ccdc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa280686d0 0x7faa2819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:51372/0 (socket says 172.21.15.70:51372) 2023-10-18T12:18:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2ccdc700 1 -- 172.21.15.70:0/489340532 learned_addr learned my addr 172.21.15.70:0/489340532 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa27fff700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa28103250 0x7faa2819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.995+0000 7faa2d4dd700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa281056f0 0x7faa281a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa2d4dd700 1 -- 172.21.15.70:0/489340532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa280686d0 msgr2=0x7faa2819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa2d4dd700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa280686d0 0x7faa2819d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa2d4dd700 1 -- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa28103250 msgr2=0x7faa2819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa2d4dd700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa28103250 0x7faa2819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa2d4dd700 1 -- 172.21.15.70:0/489340532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140095f0 con 0x7faa281056f0 2023-10-18T12:18:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa27fff700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa28103250 0x7faa2819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.996+0000 7faa2d4dd700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa281056f0 0x7faa281a1a90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa2000efc0 tx=0x7faa2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.997+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20018070 con 0x7faa281056f0 2023-10-18T12:18:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.997+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa281a2650 con 0x7faa281056f0 2023-10-18T12:18:41.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.997+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa281a9f10 con 0x7faa281056f0 2023-10-18T12:18:41.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.997+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2000f040 con 0x7faa281056f0 2023-10-18T12:18:41.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.997+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000cce0 con 0x7faa281056f0 2023-10-18T12:18:41.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:40.999+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c005320 con 0x7faa281056f0 2023-10-18T12:18:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.001+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7faa200092e0 con 0x7faa281056f0 2023-10-18T12:18:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.003+0000 7faa25ffb700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faa180a14c0 0x7faa180a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.003+0000 7faa2ccdc700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faa180a14c0 0x7faa180a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.003+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7faa200caae0 con 0x7faa281056f0 2023-10-18T12:18:41.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.004+0000 7faa2ccdc700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faa180a14c0 0x7faa180a3980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa1c005950 tx=0x7faa1c00a400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:41.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.006+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faa20095d30 con 0x7faa281056f0 2023-10-18T12:18:41.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.103+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f196c005b20 con 0x7f1980100110 2023-10-18T12:18:41.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.111+0000 7f1967fff700 1 -- 172.21.15.70:0/2365387627 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v30)=0 v30) v1 ==== 116+0+0 (secure 0 0 0) 0x7f1974097440 con 0x7f1980100110 2023-10-18T12:18:41.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.113+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19680a14c0 msgr2=0x7f19680a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.113+0000 7f19854dd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19680a14c0 0x7f19680a3980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1970009a10 tx=0x7f1970009190 comp rx=0 tx=0).stop 2023-10-18T12:18:41.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.113+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 msgr2=0x7f1980109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.113+0000 7f19854dd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 0x7f1980109320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1974005910 tx=0x7f1974004730 comp rx=0 tx=0).stop 2023-10-18T12:18:41.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 shutdown_connections 2023-10-18T12:18:41.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19680a14c0 0x7f19680a3980 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19801056b0 0x7f1980109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19800ff750 0x7f198010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 --2- 172.21.15.70:0/2365387627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1980100110 0x7f1980109320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 >> 172.21.15.70:0/2365387627 conn(0x7f1980075790 msgr2=0x7f19800fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 shutdown_connections 2023-10-18T12:18:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.114+0000 7f19854dd700 1 -- 172.21.15.70:0/2365387627 wait complete. 2023-10-18T12:18:41.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.238+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3f0c195cc0 con 0x7f3f0c101260 2023-10-18T12:18:41.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.243+0000 7f3efb7fe700 1 -- 172.21.15.70:0/1337540800 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f0009f610 con 0x7f3f0c101260 2023-10-18T12:18:41.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.246+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3ef40a14b0 msgr2=0x7f3ef40a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.246+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3ef40a14b0 0x7f3ef40a3970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c101930 tx=0x7f3efc005ee0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.246+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 msgr2=0x7f3f0c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.246+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c19abc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f00000c00 tx=0x7f3f00003c60 comp rx=0 tx=0).stop 2023-10-18T12:18:41.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 shutdown_connections 2023-10-18T12:18:41.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f0c1057c0 0x7f3f0c1947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c1008a0 0x7f3f0c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3ef40a14b0 0x7f3ef40a3970 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 --2- 172.21.15.70:0/1337540800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f0c101260 0x7f3f0c19abc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 >> 172.21.15.70:0/1337540800 conn(0x7f3f0c0779e0 msgr2=0x7f3f0c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:41.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 shutdown_connections 2023-10-18T12:18:41.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.247+0000 7f3f10e4f700 1 -- 172.21.15.70:0/1337540800 wait complete. 2023-10-18T12:18:41.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.275+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7faa0c005b50 con 0x7faa281056f0 2023-10-18T12:18:41.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.281+0000 7faa25ffb700 1 -- 172.21.15.70:0/489340532 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v32)=0 v32) v1 ==== 116+0+0 (secure 0 0 0) 0x7faa200b7e60 con 0x7faa281056f0 2023-10-18T12:18:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faa180a14c0 msgr2=0x7faa180a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faa180a14c0 0x7faa180a3980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa1c005950 tx=0x7faa1c00a400 comp rx=0 tx=0).stop 2023-10-18T12:18:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa281056f0 msgr2=0x7faa281a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa281056f0 0x7faa281a1a90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa2000efc0 tx=0x7faa2000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 shutdown_connections 2023-10-18T12:18:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faa180a14c0 0x7faa180a3980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa281056f0 0x7faa281a1a90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa280686d0 0x7faa2819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 --2- 172.21.15.70:0/489340532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa28103250 0x7faa2819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:41.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.283+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 >> 172.21.15.70:0/489340532 conn(0x7faa28075790 msgr2=0x7faa28102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:41.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.284+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 shutdown_connections 2023-10-18T12:18:41.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:41.284+0000 7faa2ef40700 1 -- 172.21.15.70:0/489340532 wait complete. 2023-10-18T12:18:42.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3483235092' entity='client.admin' 2023-10-18T12:18:42.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1143546355' entity='client.admin' 2023-10-18T12:18:42.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2365387627' entity='client.admin' 2023-10-18T12:18:42.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:42.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:42.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:42.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:42.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:42.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:41 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' 2023-10-18T12:18:42.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3483235092' entity='client.admin' 2023-10-18T12:18:42.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1143546355' entity='client.admin' 2023-10-18T12:18:42.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2365387627' entity='client.admin' 2023-10-18T12:18:42.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:42.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:42.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:42.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:42.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:42.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:41 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' 2023-10-18T12:18:42.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3483235092' entity='client.admin' 2023-10-18T12:18:42.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1143546355' entity='client.admin' 2023-10-18T12:18:42.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2365387627' entity='client.admin' 2023-10-18T12:18:42.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:18:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:18:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:18:42.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:41 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' 2023-10-18T12:18:42.986 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:42 smithi070 ceph-mon[98639]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:43.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:42 smithi152 ceph-mon[134767]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:43.212 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:42 smithi186 ceph-mon[135253]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:43.235 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.236 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 -- 172.21.15.70:0/1903554509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441008a0 msgr2=0x7f8f44100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:43.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/1903554509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441008a0 0x7f8f44100d20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009990 tx=0x7f8f34009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 -- 172.21.15.70:0/1903554509 shutdown_connections 2023-10-18T12:18:43.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/1903554509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f44101260 0x7f8f4410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/1903554509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f441057c0 0x7f8f44105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/1903554509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441008a0 0x7f8f44100d20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 -- 172.21.15.70:0/1903554509 >> 172.21.15.70:0/1903554509 conn(0x7f8f440779e0 msgr2=0x7f8f44077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 -- 172.21.15.70:0/1903554509 shutdown_connections 2023-10-18T12:18:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.284+0000 7f8f4c41d700 1 -- 172.21.15.70:0/1903554509 wait complete. 2023-10-18T12:18:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 Processor -- start 2023-10-18T12:18:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 -- start start 2023-10-18T12:18:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f441008a0 0x7f8f44072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f44101260 0x7f8f44073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 0x7f8f44073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441a5070 con 0x7f8f441057c0 2023-10-18T12:18:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441a51e0 con 0x7f8f44101260 2023-10-18T12:18:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.286+0000 7f8f4c41d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441a53a0 con 0x7f8f441008a0 2023-10-18T12:18:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 0x7f8f44073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:43.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 0x7f8f44073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35262/0 (socket says 172.21.15.70:35262) 2023-10-18T12:18:43.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 -- 172.21.15.70:0/3387747017 learned_addr learned my addr 172.21.15.70:0/3387747017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:43.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f499b8700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f44101260 0x7f8f44073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:43.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a1b9700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f441008a0 0x7f8f44072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:43.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 -- 172.21.15.70:0/3387747017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f441008a0 msgr2=0x7f8f44072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:43.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f441008a0 0x7f8f44072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 -- 172.21.15.70:0/3387747017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f44101260 msgr2=0x7f8f44073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:43.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f44101260 0x7f8f44073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 -- 172.21.15.70:0/3387747017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40009520 con 0x7f8f441057c0 2023-10-18T12:18:43.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f4a9ba700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 0x7f8f44073570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00efc0 tx=0x7f8f3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.287+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c018070 con 0x7f8f441057c0 2023-10-18T12:18:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.288+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c00f040 con 0x7f8f441057c0 2023-10-18T12:18:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.288+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340095f0 con 0x7f8f441057c0 2023-10-18T12:18:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.288+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00cc10 con 0x7f8f441057c0 2023-10-18T12:18:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.288+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f441a5990 con 0x7f8f441057c0 2023-10-18T12:18:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.289+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f44108da0 con 0x7f8f441057c0 2023-10-18T12:18:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.290+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f8f3c003a00 con 0x7f8f441057c0 2023-10-18T12:18:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.292+0000 7f8f3b7fe700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f300a14c0 0x7f8f300a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.292+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f8f3c0cab00 con 0x7f8f441057c0 2023-10-18T12:18:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.292+0000 7f8f4a1b9700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f300a14c0 0x7f8f300a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.293+0000 7f8f4a1b9700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f300a14c0 0x7f8f300a3980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f44101930 tx=0x7f8f40009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:43.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.296+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f3c095d50 con 0x7f8f441057c0 2023-10-18T12:18:43.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.495+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f8f44101f30 con 0x7f8f441057c0 2023-10-18T12:18:43.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.499+0000 7f8f3b7fe700 1 -- 172.21.15.70:0/3387747017 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v33)=0 v33) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8f3c099370 con 0x7f8f441057c0 2023-10-18T12:18:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f300a14c0 msgr2=0x7f8f300a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f300a14c0 0x7f8f300a3980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f44101930 tx=0x7f8f40009260 comp rx=0 tx=0).stop 2023-10-18T12:18:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 msgr2=0x7f8f44073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 0x7f8f44073570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00efc0 tx=0x7f8f3c00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 shutdown_connections 2023-10-18T12:18:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f441057c0 0x7f8f44073570 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f441008a0 0x7f8f44072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f300a14c0 0x7f8f300a3980 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 --2- 172.21.15.70:0/3387747017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f44101260 0x7f8f44073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 >> 172.21.15.70:0/3387747017 conn(0x7f8f440779e0 msgr2=0x7f8f440fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 shutdown_connections 2023-10-18T12:18:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:43.501+0000 7f8f4c41d700 1 -- 172.21.15.70:0/3387747017 wait complete. 2023-10-18T12:18:43.946 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'f94fde2a-6daf-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi070') 2023-10-18T12:18:43.950 INFO:tasks.cephadm:Deploying OSDs... 2023-10-18T12:18:43.950 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:18:43.951 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T12:18:43.994 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T12:18:43.995 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2023-10-18T12:18:44.079 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T12:18:44.079 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:44.080 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-18T12:18:44.080 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:44.080 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:44.080 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-18 12:17:14.610666773 +0000 2023-10-18T12:18:44.080 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-18 12:10:53.960174259 +0000 2023-10-18T12:18:44.081 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-18 12:10:53.960174259 +0000 2023-10-18T12:18:44.081 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-18 12:10:53.936174964 +0000 2023-10-18T12:18:44.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T12:18:44.187 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-18T12:18:44.187 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-18T12:18:44.187 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000193798 s, 2.6 MB/s 2023-10-18T12:18:44.190 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T12:18:44.235 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2023-10-18T12:18:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T12:18:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:44.320 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-18T12:18:44.320 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:44.320 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:44.321 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-18 12:17:14.684664582 +0000 2023-10-18T12:18:44.321 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-18 12:10:54.808149387 +0000 2023-10-18T12:18:44.321 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-18 12:10:54.808149387 +0000 2023-10-18T12:18:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-18 12:10:54.786150032 +0000 2023-10-18T12:18:44.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T12:18:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-18T12:18:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-18T12:18:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000190948 s, 2.7 MB/s 2023-10-18T12:18:44.427 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T12:18:44.472 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2023-10-18T12:18:44.556 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T12:18:44.557 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:44.557 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-18T12:18:44.557 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:44.558 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:44.558 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-18 12:17:14.756662451 +0000 2023-10-18T12:18:44.558 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-18 12:10:55.623125512 +0000 2023-10-18T12:18:44.558 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-18 12:10:55.623125512 +0000 2023-10-18T12:18:44.558 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-18 12:10:55.601126157 +0000 2023-10-18T12:18:44.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T12:18:44.662 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-18T12:18:44.663 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-18T12:18:44.663 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000195215 s, 2.6 MB/s 2023-10-18T12:18:44.665 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T12:18:44.711 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2023-10-18T12:18:44.795 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T12:18:44.795 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:44.796 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-18T12:18:44.796 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:44.796 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:44.796 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-18 12:17:14.841659936 +0000 2023-10-18T12:18:44.796 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-18 12:10:56.447101373 +0000 2023-10-18T12:18:44.797 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-18 12:10:56.447101373 +0000 2023-10-18T12:18:44.797 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-18 12:10:56.425102018 +0000 2023-10-18T12:18:44.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T12:18:44.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:44 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3387747017' entity='client.admin' 2023-10-18T12:18:44.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:44 smithi070 ceph-mon[98639]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:44.900 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-18T12:18:44.900 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-18T12:18:44.901 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000194446 s, 2.6 MB/s 2023-10-18T12:18:44.903 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T12:18:44.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:44 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3387747017' entity='client.admin' 2023-10-18T12:18:44.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:44 smithi152 ceph-mon[134767]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:44.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:44 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3387747017' entity='client.admin' 2023-10-18T12:18:44.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:44 smithi186 ceph-mon[135253]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:44.989 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-10-18T12:18:44.989 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T12:18:45.010 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T12:18:45.011 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2023-10-18T12:18:45.071 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T12:18:45.072 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:45.072 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 387816 Links: 1 2023-10-18T12:18:45.072 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:45.072 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:45.073 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-10-18 12:18:25.399378992 +0000 2023-10-18T12:18:45.073 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-10-18 12:07:40.157298394 +0000 2023-10-18T12:18:45.073 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-10-18 12:07:40.157298394 +0000 2023-10-18T12:18:45.073 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-10-18T12:18:45.074 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T12:18:45.144 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-10-18T12:18:45.145 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-10-18T12:18:45.145 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.00024822 s, 2.1 MB/s 2023-10-18T12:18:45.146 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T12:18:45.207 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2023-10-18T12:18:45.265 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T12:18:45.266 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:45.266 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 387883 Links: 1 2023-10-18T12:18:45.266 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:45.266 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:45.267 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-10-18 12:18:25.399378992 +0000 2023-10-18T12:18:45.267 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-10-18 12:07:40.671292201 +0000 2023-10-18T12:18:45.267 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-10-18 12:07:40.671292201 +0000 2023-10-18T12:18:45.267 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-10-18T12:18:45.267 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T12:18:45.334 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-10-18T12:18:45.335 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-10-18T12:18:45.335 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000245224 s, 2.1 MB/s 2023-10-18T12:18:45.336 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T12:18:45.396 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2023-10-18T12:18:45.455 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T12:18:45.455 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:45.455 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 395501 Links: 1 2023-10-18T12:18:45.455 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:45.456 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:45.456 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-10-18 12:18:25.399378992 +0000 2023-10-18T12:18:45.456 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-10-18 12:07:41.169286201 +0000 2023-10-18T12:18:45.456 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-10-18 12:07:41.169286201 +0000 2023-10-18T12:18:45.456 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-10-18T12:18:45.457 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T12:18:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-10-18T12:18:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-10-18T12:18:45.524 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000245618 s, 2.1 MB/s 2023-10-18T12:18:45.525 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T12:18:45.584 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2023-10-18T12:18:45.643 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T12:18:45.643 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:45.643 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 395628 Links: 1 2023-10-18T12:18:45.644 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:45.644 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:45.644 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-10-18 12:18:25.399378992 +0000 2023-10-18T12:18:45.644 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-10-18 12:07:41.670280165 +0000 2023-10-18T12:18:45.644 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-10-18 12:07:41.670280165 +0000 2023-10-18T12:18:45.645 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-10-18T12:18:45.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T12:18:45.709 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-10-18T12:18:45.710 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-10-18T12:18:45.710 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000203697 s, 2.5 MB/s 2023-10-18T12:18:45.711 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T12:18:45.769 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-18T12:18:45.770 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T12:18:45.791 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T12:18:45.792 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-10-18T12:18:45.853 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T12:18:45.853 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:45.853 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 389443 Links: 1 2023-10-18T12:18:45.854 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:45.854 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:45.854 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-18 12:18:23.367115313 +0000 2023-10-18T12:18:45.854 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-18 12:07:30.784320862 +0000 2023-10-18T12:18:45.854 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-18 12:07:30.784320862 +0000 2023-10-18T12:18:45.855 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-18T12:18:45.855 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T12:18:45.927 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-18T12:18:45.927 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-18T12:18:45.928 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000269255 s, 1.9 MB/s 2023-10-18T12:18:45.929 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T12:18:45.989 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-10-18T12:18:46.047 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T12:18:46.048 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:46.048 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 389495 Links: 1 2023-10-18T12:18:46.048 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:46.048 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:46.049 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-18 12:18:23.367115313 +0000 2023-10-18T12:18:46.049 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-18 12:07:31.287311776 +0000 2023-10-18T12:18:46.049 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-18 12:07:31.287311776 +0000 2023-10-18T12:18:46.049 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-18T12:18:46.050 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T12:18:46.116 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-18T12:18:46.117 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-18T12:18:46.117 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000260757 s, 2.0 MB/s 2023-10-18T12:18:46.118 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T12:18:46.178 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-10-18T12:18:46.237 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T12:18:46.237 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:46.238 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 392619 Links: 1 2023-10-18T12:18:46.238 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:46.238 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:46.238 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-18 12:18:23.368115294 +0000 2023-10-18T12:18:46.238 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-18 12:07:31.803302455 +0000 2023-10-18T12:18:46.239 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-18 12:07:31.803302455 +0000 2023-10-18T12:18:46.239 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-18T12:18:46.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T12:18:46.304 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-18T12:18:46.305 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-18T12:18:46.305 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000322343 s, 1.6 MB/s 2023-10-18T12:18:46.306 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T12:18:46.365 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-10-18T12:18:46.422 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T12:18:46.423 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T12:18:46.423 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 394427 Links: 1 2023-10-18T12:18:46.423 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:18:46.423 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T12:18:46.423 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-18 12:18:23.368115294 +0000 2023-10-18T12:18:46.424 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-18 12:07:32.311293279 +0000 2023-10-18T12:18:46.424 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-18 12:07:32.311293279 +0000 2023-10-18T12:18:46.424 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-18T12:18:46.424 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T12:18:46.490 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-18T12:18:46.490 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-18T12:18:46.490 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000201626 s, 2.5 MB/s 2023-10-18T12:18:46.491 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T12:18:46.548 INFO:tasks.cephadm:Deploying osd.0 on smithi070 with /dev/vg_nvme/lv_4... 2023-10-18T12:18:46.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-18T12:18:46.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:46 smithi070 ceph-mon[98639]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:46.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:46 smithi152 ceph-mon[134767]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:46.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:46 smithi186 ceph-mon[135253]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:47.295 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:48.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:48 smithi070 ceph-mon[98639]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:48.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:48 smithi152 ceph-mon[134767]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:48.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:48 smithi186 ceph-mon[135253]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-18T12:18:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-18T12:18:49.251 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-10-18T12:18:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-10-18T12:18:49.252 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0304706 s, 344 MB/s 2023-10-18T12:18:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-10-18T12:18:49.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2023-10-18T12:18:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:18:51.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:51 smithi070 ceph-mon[98639]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:51.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:51 smithi152 ceph-mon[134767]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:51 smithi186 ceph-mon[135253]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:51.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 -- 172.21.15.70:0/4286760070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 msgr2=0x7fd328105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:51.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 --2- 172.21.15.70:0/4286760070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd328105ba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd324009990 tx=0x7fd324009ca0 comp rx=0 tx=0).stop 2023-10-18T12:18:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 -- 172.21.15.70:0/4286760070 shutdown_connections 2023-10-18T12:18:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 --2- 172.21.15.70:0/4286760070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd328101260 0x7fd32810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 --2- 172.21.15.70:0/4286760070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd328105ba0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:51.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 --2- 172.21.15.70:0/4286760070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3281008a0 0x7fd328100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:51.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 -- 172.21.15.70:0/4286760070 >> 172.21.15.70:0/4286760070 conn(0x7fd3280779e0 msgr2=0x7fd328077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:18:51.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 -- 172.21.15.70:0/4286760070 shutdown_connections 2023-10-18T12:18:51.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.948+0000 7fd32f561700 1 -- 172.21.15.70:0/4286760070 wait complete. 2023-10-18T12:18:51.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.949+0000 7fd32f561700 1 Processor -- start 2023-10-18T12:18:51.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.949+0000 7fd32f561700 1 -- start start 2023-10-18T12:18:51.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32f561700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3281008a0 0x7fd328198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:51.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32f561700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd328101260 0x7fd3281994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32f561700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd32819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32f561700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32819de20 con 0x7fd3281057c0 2023-10-18T12:18:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32f561700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32819df90 con 0x7fd328101260 2023-10-18T12:18:51.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32f561700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32819e150 con 0x7fd3281008a0 2023-10-18T12:18:51.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32dafe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd32819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:51.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32dafe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd32819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44398/0 (socket says 172.21.15.70:44398) 2023-10-18T12:18:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32d2fd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3281008a0 0x7fd328198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32dafe700 1 -- 172.21.15.70:0/1284919409 learned_addr learned my addr 172.21.15.70:0/1284919409 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:18:51.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.950+0000 7fd32cafc700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd328101260 0x7fd3281994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:51.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32d2fd700 1 -- 172.21.15.70:0/1284919409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd328101260 msgr2=0x7fd3281994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:51.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32d2fd700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd328101260 0x7fd3281994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32d2fd700 1 -- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 msgr2=0x7fd32819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:18:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32d2fd700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd32819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:18:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32d2fd700 1 -- 172.21.15.70:0/1284919409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240095f0 con 0x7fd3281008a0 2023-10-18T12:18:51.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32dafe700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd32819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:18:51.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32d2fd700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3281008a0 0x7fd328198f70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098e0 tx=0x7fd324005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:51.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32401d070 con 0x7fd3281008a0 2023-10-18T12:18:51.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32819e3d0 con 0x7fd3281008a0 2023-10-18T12:18:51.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd324022470 con 0x7fd3281008a0 2023-10-18T12:18:51.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324017650 con 0x7fd3281008a0 2023-10-18T12:18:51.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.951+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3281a5c90 con 0x7fd3281008a0 2023-10-18T12:18:51.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.952+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd328101f30 con 0x7fd3281008a0 2023-10-18T12:18:51.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.954+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fd3240225e0 con 0x7fd3281008a0 2023-10-18T12:18:51.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.955+0000 7fd31e7fc700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3140a1440 0x7fd3140a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:18:51.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.955+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd3240cc580 con 0x7fd3281008a0 2023-10-18T12:18:51.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.956+0000 7fd32cafc700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3140a1440 0x7fd3140a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:18:51.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.956+0000 7fd32cafc700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3140a1440 0x7fd3140a3900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd32819a480 tx=0x7fd318009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:18:51.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:51.958+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd3240977d0 con 0x7fd3281008a0 2023-10-18T12:18:52.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:18:52.166+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd32819a230 con 0x7fd3140a1440 2023-10-18T12:18:53.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:53 smithi070 ceph-mon[98639]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:53.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:18:53.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:18:53.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:53.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:53 smithi152 ceph-mon[134767]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:53.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:18:53.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:18:53.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:53 smithi186 ceph-mon[135253]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:53.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:18:53.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:18:53.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:18:54.226 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:54 smithi070 ceph-mon[98639]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:18:54.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:54 smithi152 ceph-mon[134767]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:18:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:54 smithi186 ceph-mon[135253]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:18:55.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:55 smithi070 ceph-mon[98639]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:55.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:55 smithi152 ceph-mon[134767]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:55.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:55 smithi186 ceph-mon[135253]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:56.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:56 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3172532065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb6d70-44d3-441f-9d82-d24f1167aac3"}]: dispatch 2023-10-18T12:18:56.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:56 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3172532065' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cb6d70-44d3-441f-9d82-d24f1167aac3"}]': finished 2023-10-18T12:18:56.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:56 smithi070 ceph-mon[98639]: osdmap e6: 1 total, 0 up, 1 in 2023-10-18T12:18:56.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:56 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:18:56.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:56 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1596656337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:18:56.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:56 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3172532065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb6d70-44d3-441f-9d82-d24f1167aac3"}]: dispatch 2023-10-18T12:18:56.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:56 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3172532065' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cb6d70-44d3-441f-9d82-d24f1167aac3"}]': finished 2023-10-18T12:18:56.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:56 smithi152 ceph-mon[134767]: osdmap e6: 1 total, 0 up, 1 in 2023-10-18T12:18:56.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:56 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:18:56.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:56 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1596656337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:18:56.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:56 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3172532065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb6d70-44d3-441f-9d82-d24f1167aac3"}]: dispatch 2023-10-18T12:18:56.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:56 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3172532065' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cb6d70-44d3-441f-9d82-d24f1167aac3"}]': finished 2023-10-18T12:18:56.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:56 smithi186 ceph-mon[135253]: osdmap e6: 1 total, 0 up, 1 in 2023-10-18T12:18:56.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:56 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:18:56.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:56 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1596656337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:18:57.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:57 smithi070 ceph-mon[98639]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:57.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:57 smithi152 ceph-mon[134767]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:57.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:57 smithi186 ceph-mon[135253]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:59.295 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:18:59 smithi070 ceph-mon[98639]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:59.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:18:59 smithi152 ceph-mon[134767]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:18:59.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:18:59 smithi186 ceph-mon[135253]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:01.180 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:01 smithi070 ceph-mon[98639]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:01 smithi152 ceph-mon[134767]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:01 smithi186 ceph-mon[135253]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:03.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:03 smithi070 ceph-mon[98639]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-18T12:19:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:03.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:03 smithi152 ceph-mon[134767]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-18T12:19:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:03 smithi186 ceph-mon[135253]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:03.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-18T12:19:03.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:04.216 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:04 smithi070 ceph-mon[98639]: Deploying daemon osd.0 on smithi070 2023-10-18T12:19:04.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:04 smithi152 ceph-mon[134767]: Deploying daemon osd.0 on smithi070 2023-10-18T12:19:04.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:04 smithi186 ceph-mon[135253]: Deploying daemon osd.0 on smithi070 2023-10-18T12:19:05.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:05 smithi070 ceph-mon[98639]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:05.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:05 smithi152 ceph-mon[134767]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:05 smithi186 ceph-mon[135253]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:07 smithi070 ceph-mon[98639]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:07.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:07 smithi152 ceph-mon[134767]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:07 smithi186 ceph-mon[135253]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:09 smithi070 ceph-mon[98639]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:09 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:09 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:09 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:09.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:09 smithi152 ceph-mon[134767]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:09.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:09 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:09 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:09 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:09 smithi186 ceph-mon[135253]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:09 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:09.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:09 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:09.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:09 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:11 smithi070 ceph-mon[98639]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:11 smithi152 ceph-mon[134767]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:11.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:11 smithi186 ceph-mon[135253]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:12.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.227+0000 7fd31e7fc700 1 -- 172.21.15.70:0/1284919409 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd32819a230 con 0x7fd3140a1440 2023-10-18T12:19:12.229 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 0 on host 'smithi070' 2023-10-18T12:19:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3140a1440 msgr2=0x7fd3140a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3140a1440 0x7fd3140a3900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd32819a480 tx=0x7fd318009260 comp rx=0 tx=0).stop 2023-10-18T12:19:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3281008a0 msgr2=0x7fd328198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3281008a0 0x7fd328198f70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098e0 tx=0x7fd324005740 comp rx=0 tx=0).stop 2023-10-18T12:19:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 shutdown_connections 2023-10-18T12:19:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3281057c0 0x7fd32819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3281008a0 0x7fd328198f70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3140a1440 0x7fd3140a3900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 --2- 172.21.15.70:0/1284919409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd328101260 0x7fd3281994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 >> 172.21.15.70:0/1284919409 conn(0x7fd3280779e0 msgr2=0x7fd32810bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:19:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 shutdown_connections 2023-10-18T12:19:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:12.230+0000 7fd32f561700 1 -- 172.21.15.70:0/1284919409 wait complete. 2023-10-18T12:19:12.728 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.728 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.728 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:12.728 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:12.729 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.729 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:12.729 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.729 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.730 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:12.731 DEBUG:teuthology.orchestra.run.smithi070:osd.0> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.0.service 2023-10-18T12:19:12.736 INFO:tasks.cephadm:Deploying osd.1 on smithi152 with /dev/vg_nvme/lv_4... 2023-10-18T12:19:12.736 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-18T12:19:12.769 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.770 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.770 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:12.770 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:12.771 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.771 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:12.771 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.771 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.772 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:12.812 INFO:journalctl@ceph.osd.0.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:19:12.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:12.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:12.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:12.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:12.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:13.104 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:19:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0[124289]: 2023-10-18T12:19:12.856+0000 7f9c3a9c8640 -1 osd.0 0 log_to_monitors true 2023-10-18T12:19:13.305 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:19:13.667 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:13 smithi070 ceph-mon[98639]: from='osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T12:19:13.667 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:13 smithi070 ceph-mon[98639]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T12:19:13.806 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:13 smithi152 ceph-mon[134767]: from='osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T12:19:13.807 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:13 smithi152 ceph-mon[134767]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T12:19:13.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:13 smithi186 ceph-mon[135253]: from='osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T12:19:13.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:13 smithi186 ceph-mon[135253]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T12:19:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:14 smithi152 ceph-mon[134767]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-18T12:19:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:14 smithi152 ceph-mon[134767]: osdmap e7: 1 total, 0 up, 1 in 2023-10-18T12:19:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:14 smithi152 ceph-mon[134767]: from='osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:19:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:14 smithi152 ceph-mon[134767]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:19:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:14 smithi152 ceph-mon[134767]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:14.798 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-18T12:19:14.799 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-18T12:19:14.799 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2023-10-18T12:19:14.799 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2023-10-18T12:19:14.799 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110091 s, 952 MB/s 2023-10-18T12:19:14.800 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2023-10-18T12:19:14.855 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0[124289]: 2023-10-18T12:19:14.512+0000 7f9c3003f700 -1 osd.0 0 waiting for initial osdmap 2023-10-18T12:19:14.855 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0[124289]: 2023-10-18T12:19:14.516+0000 7f9c29e30700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:19:14.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-mon[98639]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-18T12:19:14.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-mon[98639]: osdmap e7: 1 total, 0 up, 1 in 2023-10-18T12:19:14.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:14.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-mon[98639]: from='osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:19:14.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-mon[98639]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:19:14.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:14 smithi070 ceph-mon[98639]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:14.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:14 smithi186 ceph-mon[135253]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-18T12:19:14.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:14 smithi186 ceph-mon[135253]: osdmap e7: 1 total, 0 up, 1 in 2023-10-18T12:19:14.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:14.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:14 smithi186 ceph-mon[135253]: from='osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:19:14.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:14 smithi186 ceph-mon[135253]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:19:14.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:14 smithi186 ceph-mon[135253]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T12:19:15.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2023-10-18T12:19:15.842 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:19:15.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:15 smithi070 ceph-mon[98639]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:19:15.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:15 smithi070 ceph-mon[98639]: osdmap e8: 1 total, 0 up, 1 in 2023-10-18T12:19:15.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:15.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:15.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:15 smithi152 ceph-mon[134767]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:19:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:15 smithi152 ceph-mon[134767]: osdmap e8: 1 total, 0 up, 1 in 2023-10-18T12:19:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:15.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:15.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:15 smithi186 ceph-mon[135253]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:19:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:15 smithi186 ceph-mon[135253]: osdmap e8: 1 total, 0 up, 1 in 2023-10-18T12:19:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:15.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:16.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:19:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:19:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] boot 2023-10-18T12:19:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: osdmap e9: 1 total, 1 up, 1 in 2023-10-18T12:19:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:16.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:16.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:16.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:16.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:16.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:19:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:19:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] boot 2023-10-18T12:19:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: osdmap e9: 1 total, 1 up, 1 in 2023-10-18T12:19:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:16.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:16.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:16.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:16.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:16.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:19:16.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:19:16.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: osd.0 [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] boot 2023-10-18T12:19:16.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: osdmap e9: 1 total, 1 up, 1 in 2023-10-18T12:19:16.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T12:19:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:16.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:16.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:16.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:16.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:17.152 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 -- 172.21.15.152:0/1025039937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7950104720 msgr2=0x7f7950104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:17.152 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 --2- 172.21.15.152:0/1025039937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7950104720 0x7f7950104b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7944009990 tx=0x7f7944009ca0 comp rx=0 tx=0).stop 2023-10-18T12:19:17.152 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 -- 172.21.15.152:0/1025039937 shutdown_connections 2023-10-18T12:19:17.152 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 --2- 172.21.15.152:0/1025039937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7950100c60 0x7f795010c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:17.153 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 --2- 172.21.15.152:0/1025039937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7950104720 0x7f7950104b00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:17.153 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 --2- 172.21.15.152:0/1025039937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79501002a0 0x7f7950100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:17.153 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 -- 172.21.15.152:0/1025039937 >> 172.21.15.152:0/1025039937 conn(0x7f7950074a10 msgr2=0x7f7950074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:19:17.153 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 -- 172.21.15.152:0/1025039937 shutdown_connections 2023-10-18T12:19:17.153 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 -- 172.21.15.152:0/1025039937 wait complete. 2023-10-18T12:19:17.155 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 Processor -- start 2023-10-18T12:19:17.155 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.149+0000 7f7956481700 1 -- start start 2023-10-18T12:19:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7956481700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 0x7f7950197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7956481700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7950100c60 0x7f7950198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7956481700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7950104720 0x7f79501985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7956481700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79501a2df0 con 0x7f7950100c60 2023-10-18T12:19:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7956481700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79501a2f60 con 0x7f79501002a0 2023-10-18T12:19:17.157 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7956481700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79501a3120 con 0x7f7950104720 2023-10-18T12:19:17.157 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 0x7f7950197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:17.157 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 0x7f7950197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57640/0 (socket says 172.21.15.152:57640) 2023-10-18T12:19:17.157 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 -- 172.21.15.152:0/2740254759 learned_addr learned my addr 172.21.15.152:0/2740254759 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-10-18T12:19:17.157 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f7954a1e700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7950104720 0x7f79501985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:17.158 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 -- 172.21.15.152:0/2740254759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7950104720 msgr2=0x7f79501985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:17.158 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7950104720 0x7f79501985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:17.158 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 -- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7950100c60 msgr2=0x7f7950198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:17.158 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7950100c60 0x7f7950198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:17.158 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 -- 172.21.15.152:0/2740254759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440095f0 con 0x7f79501002a0 2023-10-18T12:19:17.159 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.150+0000 7f794ffff700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 0x7f7950197ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79440098e0 tx=0x7f794400f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:19:17.159 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.151+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794401d070 con 0x7f79501002a0 2023-10-18T12:19:17.159 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.151+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7944022470 con 0x7f79501002a0 2023-10-18T12:19:17.159 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.151+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79501a33a0 con 0x7f79501002a0 2023-10-18T12:19:17.159 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.151+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944017650 con 0x7f79501002a0 2023-10-18T12:19:17.160 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.151+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79501a3810 con 0x7f79501002a0 2023-10-18T12:19:17.160 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.152+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7950107c80 con 0x7f79501002a0 2023-10-18T12:19:17.160 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.152+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f79440225e0 con 0x7f79501002a0 2023-10-18T12:19:17.161 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.153+0000 7f794d7fa700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79380a14c0 0x7f79380a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:17.161 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.153+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f79440cc460 con 0x7f79501002a0 2023-10-18T12:19:17.161 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.153+0000 7f794f7fe700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79380a14c0 0x7f79380a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:17.161 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.154+0000 7f794f7fe700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79380a14c0 0x7f79380a3980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79500ff950 tx=0x7f7940006d20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:19:17.162 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.157+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f79440974f0 con 0x7f79501002a0 2023-10-18T12:19:17.354 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:17.352+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f79500ff700 con 0x7f79380a14c0 2023-10-18T12:19:17.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:17 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi070 2023-10-18T12:19:17.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:17 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi070 to 17271M 2023-10-18T12:19:17.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:17 smithi070 ceph-mon[98639]: osdmap e10: 1 total, 1 up, 1 in 2023-10-18T12:19:17.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:19:17.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:19:17.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:17.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:17 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi070 2023-10-18T12:19:17.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:17 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi070 to 17271M 2023-10-18T12:19:17.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:17 smithi152 ceph-mon[134767]: osdmap e10: 1 total, 1 up, 1 in 2023-10-18T12:19:17.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:19:17.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:19:17.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:17.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:17 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi070 2023-10-18T12:19:17.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:17 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi070 to 17271M 2023-10-18T12:19:17.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:17 smithi186 ceph-mon[135253]: osdmap e10: 1 total, 1 up, 1 in 2023-10-18T12:19:17.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:19:17.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:19:17.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:18.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:18 smithi070 ceph-mon[98639]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:19:18.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:18 smithi070 ceph-mon[98639]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:18.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:18 smithi152 ceph-mon[134767]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:19:18.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:18 smithi152 ceph-mon[134767]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:18.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:18 smithi186 ceph-mon[135253]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:19:18.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:18 smithi186 ceph-mon[135253]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:20 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/2415502690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]: dispatch 2023-10-18T12:19:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:20 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]: dispatch 2023-10-18T12:19:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:20 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]': finished 2023-10-18T12:19:20.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:20 smithi070 ceph-mon[98639]: osdmap e11: 2 total, 1 up, 2 in 2023-10-18T12:19:20.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:20.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:20 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/2415502690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]: dispatch 2023-10-18T12:19:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:20 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]: dispatch 2023-10-18T12:19:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:20 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]': finished 2023-10-18T12:19:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:20 smithi152 ceph-mon[134767]: osdmap e11: 2 total, 1 up, 2 in 2023-10-18T12:19:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:20 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/2415502690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]: dispatch 2023-10-18T12:19:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:20 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]: dispatch 2023-10-18T12:19:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:20 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48353e9d-5623-4c9b-b90a-35487c80253c"}]': finished 2023-10-18T12:19:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:20 smithi186 ceph-mon[135253]: osdmap e11: 2 total, 1 up, 2 in 2023-10-18T12:19:20.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:21.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:21 smithi070 ceph-mon[98639]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:21.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:21 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/582652411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:19:21.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:21 smithi152 ceph-mon[134767]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:21 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/582652411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:19:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:21 smithi186 ceph-mon[135253]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:21 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/582652411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:19:23.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:23 smithi070 ceph-mon[98639]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:23.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:23 smithi152 ceph-mon[134767]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:23 smithi186 ceph-mon[135253]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:25 smithi070 ceph-mon[98639]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:25.419 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:25 smithi152 ceph-mon[134767]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:25 smithi186 ceph-mon[135253]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:27.154 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:27 smithi152 ceph-mon[134767]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:27.155 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:27 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-18T12:19:27.155 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:27 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:27 smithi070 ceph-mon[98639]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:27 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-18T12:19:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:27 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:27 smithi186 ceph-mon[135253]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:27 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-18T12:19:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:27 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:28.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:28 smithi070 ceph-mon[98639]: Deploying daemon osd.1 on smithi152 2023-10-18T12:19:28.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:28 smithi152 ceph-mon[134767]: Deploying daemon osd.1 on smithi152 2023-10-18T12:19:28.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:28 smithi186 ceph-mon[135253]: Deploying daemon osd.1 on smithi152 2023-10-18T12:19:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:29 smithi070 ceph-mon[98639]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:29.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:29 smithi152 ceph-mon[134767]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:29 smithi186 ceph-mon[135253]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:31.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:31 smithi070 ceph-mon[98639]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:31 smithi152 ceph-mon[134767]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:31.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:31.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:31 smithi186 ceph-mon[135253]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:33.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:33 smithi152 ceph-mon[134767]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:33.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:33.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:33.325 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.322+0000 7f794d7fa700 1 -- 172.21.15.152:0/2740254759 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f79500ff700 con 0x7f79380a14c0 2023-10-18T12:19:33.326 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 1 on host 'smithi152' 2023-10-18T12:19:33.327 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.324+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79380a14c0 msgr2=0x7f79380a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:33.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.324+0000 7f7956481700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79380a14c0 0x7f79380a3980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79500ff950 tx=0x7f7940006d20 comp rx=0 tx=0).stop 2023-10-18T12:19:33.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 msgr2=0x7f7950197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:33.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 0x7f7950197ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79440098e0 tx=0x7f794400f670 comp rx=0 tx=0).stop 2023-10-18T12:19:33.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 shutdown_connections 2023-10-18T12:19:33.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7950104720 0x7f79501985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:33.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79380a14c0 0x7f79380a3980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:33.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501002a0 0x7f7950197ae0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:33.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 --2- 172.21.15.152:0/2740254759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7950100c60 0x7f7950198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:33.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 >> 172.21.15.152:0/2740254759 conn(0x7f7950074a10 msgr2=0x7f79500fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:19:33.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 shutdown_connections 2023-10-18T12:19:33.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:19:33.325+0000 7f7956481700 1 -- 172.21.15.152:0/2740254759 wait complete. 2023-10-18T12:19:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:33 smithi070 ceph-mon[98639]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:33 smithi186 ceph-mon[135253]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:33.736 DEBUG:teuthology.orchestra.run.smithi152:osd.1> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.1.service 2023-10-18T12:19:33.739 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2023-10-18T12:19:33.739 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-18T12:19:33.766 INFO:journalctl@ceph.osd.1.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:19:34.303 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:19:34.326 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:34.327 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:34.327 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.327 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.327 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.327 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:34.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:34.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:34.464 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:34.464 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:34.464 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:34.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:34.921 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:19:34 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1[140962]: 2023-10-18T12:19:34.599+0000 7f4ccfde3640 -1 osd.1 0 log_to_monitors true 2023-10-18T12:19:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:35 smithi070 ceph-mon[98639]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:35 smithi070 ceph-mon[98639]: from='osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T12:19:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:35 smithi070 ceph-mon[98639]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T12:19:35.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:35 smithi152 ceph-mon[134767]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:35.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:35 smithi152 ceph-mon[134767]: from='osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T12:19:35.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:35 smithi152 ceph-mon[134767]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T12:19:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:35 smithi186 ceph-mon[135253]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:35 smithi186 ceph-mon[135253]: from='osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T12:19:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:35 smithi186 ceph-mon[135253]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T12:19:35.714 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-18T12:19:35.714 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-18T12:19:35.715 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-18T12:19:35.715 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-18T12:19:35.715 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132473 s, 792 MB/s 2023-10-18T12:19:35.715 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-18T12:19:36.057 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2023-10-18T12:19:36.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:36 smithi070 ceph-mon[98639]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-18T12:19:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:36 smithi070 ceph-mon[98639]: osdmap e12: 2 total, 1 up, 2 in 2023-10-18T12:19:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:36 smithi070 ceph-mon[98639]: from='osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:19:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:36 smithi070 ceph-mon[98639]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:19:36.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:36 smithi070 ceph-mon[98639]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:36.613 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:19:36.671 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1[140962]: 2023-10-18T12:19:36.286+0000 7f4cc545a700 -1 osd.1 0 waiting for initial osdmap 2023-10-18T12:19:36.672 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1[140962]: 2023-10-18T12:19:36.291+0000 7f4cbf24b700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:19:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-mon[134767]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-18T12:19:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-mon[134767]: osdmap e12: 2 total, 1 up, 2 in 2023-10-18T12:19:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:36.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-mon[134767]: from='osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:19:36.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-mon[134767]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:19:36.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:36 smithi152 ceph-mon[134767]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:36.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:36 smithi186 ceph-mon[135253]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-18T12:19:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:36 smithi186 ceph-mon[135253]: osdmap e12: 2 total, 1 up, 2 in 2023-10-18T12:19:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:36 smithi186 ceph-mon[135253]: from='osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:19:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:36 smithi186 ceph-mon[135253]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:19:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:36 smithi186 ceph-mon[135253]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:19:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: osdmap e13: 2 total, 1 up, 2 in 2023-10-18T12:19:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi152 2023-10-18T12:19:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:37.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi152 to 17168M 2023-10-18T12:19:37.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:37.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:37.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:19:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: osdmap e13: 2 total, 1 up, 2 in 2023-10-18T12:19:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi152 2023-10-18T12:19:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi152 to 17168M 2023-10-18T12:19:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:37.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:37.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:19:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: osdmap e13: 2 total, 1 up, 2 in 2023-10-18T12:19:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi152 2023-10-18T12:19:37.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:37.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi152 to 17168M 2023-10-18T12:19:37.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:37.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:37.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:37.808 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 -- 172.21.15.186:0/1263141619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 msgr2=0x7f123c1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:37.809 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 --2- 172.21.15.186:0/1263141619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1059f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f122c009990 tx=0x7f122c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:19:37.809 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 -- 172.21.15.186:0/1263141619 shutdown_connections 2023-10-18T12:19:37.809 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 --2- 172.21.15.186:0/1263141619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c1002b0 0x7f123c10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:37.809 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 --2- 172.21.15.186:0/1263141619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1059f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:37.810 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 --2- 172.21.15.186:0/1263141619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c0ff8f0 0x7f123c0ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:37.810 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 -- 172.21.15.186:0/1263141619 >> 172.21.15.186:0/1263141619 conn(0x7f123c077410 msgr2=0x7f123c077820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:19:37.810 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 -- 172.21.15.186:0/1263141619 shutdown_connections 2023-10-18T12:19:37.810 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.805+0000 7f123b59e700 1 -- 172.21.15.186:0/1263141619 wait complete. 2023-10-18T12:19:37.813 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 Processor -- start 2023-10-18T12:19:37.813 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 -- start start 2023-10-18T12:19:37.814 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c0ff8f0 0x7f123c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:37.814 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c1002b0 0x7f123c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:37.814 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1a2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:37.814 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1986f0 con 0x7f123c1002b0 2023-10-18T12:19:37.814 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1a2eb0 con 0x7f123c0ff8f0 2023-10-18T12:19:37.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123b59e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1a2ff0 con 0x7f123c105610 2023-10-18T12:19:37.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:37.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1a2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:50428/0 (socket says 172.21.15.186:50428) 2023-10-18T12:19:37.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 -- 172.21.15.186:0/1006781075 learned_addr learned my addr 172.21.15.186:0/1006781075 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-18T12:19:37.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123a59c700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c0ff8f0 0x7f123c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:37.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f1239d9b700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c1002b0 0x7f123c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:37.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 -- 172.21.15.186:0/1006781075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c0ff8f0 msgr2=0x7f123c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:37.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c0ff8f0 0x7f123c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:37.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 -- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c1002b0 msgr2=0x7f123c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:37.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c1002b0 0x7f123c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:37.817 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123ad9d700 1 -- 172.21.15.186:0/1006781075 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0095f0 con 0x7f123c105610 2023-10-18T12:19:37.817 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f1239d9b700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c1002b0 0x7f123c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:19:37.817 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.806+0000 7f123a59c700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c0ff8f0 0x7f123c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:19:37.817 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.807+0000 7f123ad9d700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1a2970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1234007ee0 tx=0x7f123400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:19:37.817 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.807+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400ddd0 con 0x7f123c105610 2023-10-18T12:19:37.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.807+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123400f040 con 0x7f123c105610 2023-10-18T12:19:37.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.807+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c1a3270 con 0x7f123c105610 2023-10-18T12:19:37.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.807+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234015400 con 0x7f123c105610 2023-10-18T12:19:37.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.807+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c1a37c0 con 0x7f123c105610 2023-10-18T12:19:37.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.808+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c107ad0 con 0x7f123c105610 2023-10-18T12:19:37.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.809+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f1234004020 con 0x7f123c105610 2023-10-18T12:19:37.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.809+0000 7f122b7fe700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12240a14c0 0x7f12240a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:37.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.809+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2524+0+0 (secure 0 0 0) 0x7f12340cb680 con 0x7f123c105610 2023-10-18T12:19:37.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.809+0000 7f123a59c700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12240a14c0 0x7f12240a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:37.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.810+0000 7f123a59c700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12240a14c0 0x7f12240a3980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b5c0 tx=0x7f122c005fb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:19:37.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:37.813+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1234096460 con 0x7f123c105610 2023-10-18T12:19:38.011 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:38.008+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f123c068fe0 con 0x7f12240a14c0 2023-10-18T12:19:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:19:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:19:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] boot 2023-10-18T12:19:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: osdmap e14: 2 total, 2 up, 2 in 2023-10-18T12:19:38.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:38.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:38.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:19:38.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:19:38.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:19:38.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:38.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:19:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:19:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] boot 2023-10-18T12:19:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: osdmap e14: 2 total, 2 up, 2 in 2023-10-18T12:19:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:38.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:19:38.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:19:38.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:19:38.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:38.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:19:38.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:19:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: osd.1 [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] boot 2023-10-18T12:19:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: osdmap e14: 2 total, 2 up, 2 in 2023-10-18T12:19:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T12:19:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T12:19:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:19:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:19:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:19:38.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:39.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:39 smithi186 ceph-mon[135253]: osdmap e15: 2 total, 2 up, 2 in 2023-10-18T12:19:39.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:39 smithi070 ceph-mon[98639]: osdmap e15: 2 total, 2 up, 2 in 2023-10-18T12:19:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:39 smithi152 ceph-mon[134767]: osdmap e15: 2 total, 2 up, 2 in 2023-10-18T12:19:40.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:40 smithi070 ceph-mon[98639]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:40.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:40 smithi152 ceph-mon[134767]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:40 smithi186 ceph-mon[135253]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/2327127346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]: dispatch 2023-10-18T12:19:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:41 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]: dispatch 2023-10-18T12:19:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:41 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]': finished 2023-10-18T12:19:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:41 smithi070 ceph-mon[98639]: osdmap e16: 3 total, 2 up, 3 in 2023-10-18T12:19:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/1786335975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:19:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/2327127346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]: dispatch 2023-10-18T12:19:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:41 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]: dispatch 2023-10-18T12:19:41.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:41 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]': finished 2023-10-18T12:19:41.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:41 smithi152 ceph-mon[134767]: osdmap e16: 3 total, 2 up, 3 in 2023-10-18T12:19:41.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:41.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/1786335975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:19:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/2327127346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]: dispatch 2023-10-18T12:19:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:41 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]: dispatch 2023-10-18T12:19:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:41 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "972ea005-3f12-4646-896e-89622a35b19a"}]': finished 2023-10-18T12:19:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:41 smithi186 ceph-mon[135253]: osdmap e16: 3 total, 2 up, 3 in 2023-10-18T12:19:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/1786335975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:19:42.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:42 smithi070 ceph-mon[98639]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:42.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:42 smithi152 ceph-mon[134767]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:42.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:42 smithi186 ceph-mon[135253]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:45.310 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:45 smithi186 ceph-mon[135253]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:45 smithi070 ceph-mon[98639]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:45.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:45 smithi152 ceph-mon[134767]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:47.303 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:47 smithi186 ceph-mon[135253]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:47.303 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-18T12:19:47.303 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:47.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:47 smithi070 ceph-mon[98639]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:47.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-18T12:19:47.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:47.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:47 smithi152 ceph-mon[134767]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:47.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-18T12:19:47.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:48.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:48 smithi070 ceph-mon[98639]: Deploying daemon osd.2 on smithi186 2023-10-18T12:19:48.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:48 smithi152 ceph-mon[134767]: Deploying daemon osd.2 on smithi186 2023-10-18T12:19:48.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:48 smithi186 ceph-mon[135253]: Deploying daemon osd.2 on smithi186 2023-10-18T12:19:49.166 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:49 smithi186 ceph-mon[135253]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:49.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:49 smithi070 ceph-mon[98639]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:49.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:49 smithi152 ceph-mon[134767]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:51 smithi186 ceph-mon[135253]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:51.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:51 smithi070 ceph-mon[98639]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:51.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:51 smithi152 ceph-mon[134767]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:52.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:52.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:52.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:52.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:52.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:52.424 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:52.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:52.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:52.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:53.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:53 smithi186 ceph-mon[135253]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:53.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:53 smithi152 ceph-mon[134767]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:53.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:53 smithi070 ceph-mon[98639]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:53.918 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.915+0000 7f122b7fe700 1 -- 172.21.15.186:0/1006781075 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f123c068fe0 con 0x7f12240a14c0 2023-10-18T12:19:53.918 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2023-10-18T12:19:53.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12240a14c0 msgr2=0x7f12240a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:53.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12240a14c0 0x7f12240a3980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b5c0 tx=0x7f122c005fb0 comp rx=0 tx=0).stop 2023-10-18T12:19:53.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 msgr2=0x7f123c1a2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:53.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1a2970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1234007ee0 tx=0x7f123400d3b0 comp rx=0 tx=0).stop 2023-10-18T12:19:53.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 shutdown_connections 2023-10-18T12:19:53.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f123c105610 0x7f123c1a2970 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:53.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12240a14c0 0x7f12240a3980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:53.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c0ff8f0 0x7f123c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:53.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 --2- 172.21.15.186:0/1006781075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f123c1002b0 0x7f123c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:53.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 >> 172.21.15.186:0/1006781075 conn(0x7f123c077410 msgr2=0x7f123c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:19:53.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 shutdown_connections 2023-10-18T12:19:53.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:19:53.918+0000 7f123b59e700 1 -- 172.21.15.186:0/1006781075 wait complete. 2023-10-18T12:19:54.314 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.2.service 2023-10-18T12:19:54.317 INFO:tasks.cephadm:Deploying osd.3 on smithi070 with /dev/vg_nvme/lv_3... 2023-10-18T12:19:54.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-18T12:19:54.355 INFO:journalctl@ceph.osd.2.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:19:54.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:54.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:54.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:19:55.056 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:19:55.356 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:55 smithi186 ceph-mon[135253]: from='osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T12:19:55.356 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:55 smithi186 ceph-mon[135253]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T12:19:55.357 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:19:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2[142282]: 2023-10-18T12:19:55.061+0000 7ff72f9ae640 -1 osd.2 0 log_to_monitors true 2023-10-18T12:19:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:55 smithi070 ceph-mon[98639]: from='osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T12:19:55.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:55 smithi070 ceph-mon[98639]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T12:19:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:55 smithi152 ceph-mon[134767]: from='osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T12:19:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:55 smithi152 ceph-mon[134767]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T12:19:56.587 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2[142282]: 2023-10-18T12:19:56.315+0000 7ff725025700 -1 osd.2 0 waiting for initial osdmap 2023-10-18T12:19:56.588 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2[142282]: 2023-10-18T12:19:56.320+0000 7ff720e1a700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:19:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-mon[135253]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-18T12:19:56.588 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-mon[135253]: osdmap e17: 3 total, 2 up, 3 in 2023-10-18T12:19:56.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:56.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-mon[135253]: from='osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:19:56.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-mon[135253]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:19:56.589 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:56 smithi186 ceph-mon[135253]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:56.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:56 smithi070 ceph-mon[98639]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-18T12:19:56.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:56 smithi070 ceph-mon[98639]: osdmap e17: 3 total, 2 up, 3 in 2023-10-18T12:19:56.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:56 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:56.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:56 smithi070 ceph-mon[98639]: from='osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:19:56.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:56 smithi070 ceph-mon[98639]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:19:56.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:56 smithi070 ceph-mon[98639]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:56.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:56 smithi152 ceph-mon[134767]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-18T12:19:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:56 smithi152 ceph-mon[134767]: osdmap e17: 3 total, 2 up, 3 in 2023-10-18T12:19:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:56 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:56 smithi152 ceph-mon[134767]: from='osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:19:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:56 smithi152 ceph-mon[134767]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:19:56.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:56 smithi152 ceph-mon[134767]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:56.992 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-18T12:19:56.993 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-18T12:19:56.993 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-10-18T12:19:56.993 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-10-18T12:19:56.994 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0396638 s, 264 MB/s 2023-10-18T12:19:56.994 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-10-18T12:19:57.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2023-10-18T12:19:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:19:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: osdmap e18: 3 total, 2 up, 3 in 2023-10-18T12:19:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi186 2023-10-18T12:19:57.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:57.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-18T12:19:57.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:57.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:57.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:57 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:19:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: osdmap e18: 3 total, 2 up, 3 in 2023-10-18T12:19:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi186 2023-10-18T12:19:57.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:57.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-18T12:19:57.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:57.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:57.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:57 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:19:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: osdmap e18: 3 total, 2 up, 3 in 2023-10-18T12:19:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi186 2023-10-18T12:19:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:19:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-18T12:19:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:57.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:19:57.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:19:57.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:57 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:19:58.281 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:19:58.582 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:19:58.582 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:19:58.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] boot 2023-10-18T12:19:58.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: osdmap e19: 3 total, 3 up, 3 in 2023-10-18T12:19:58.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:58.583 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:58.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:19:58.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:19:58.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:19:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] boot 2023-10-18T12:19:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: osdmap e19: 3 total, 3 up, 3 in 2023-10-18T12:19:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:19:58.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:19:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:19:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: osd.2 [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] boot 2023-10-18T12:19:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: osdmap e19: 3 total, 3 up, 3 in 2023-10-18T12:19:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T12:19:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T12:19:58.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:19:59.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:59 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-18T12:19:59.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:59 smithi070 ceph-mon[98639]: osdmap e20: 3 total, 3 up, 3 in 2023-10-18T12:19:59.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:19:59 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:19:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:59 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-18T12:19:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:59 smithi152 ceph-mon[134767]: osdmap e20: 3 total, 3 up, 3 in 2023-10-18T12:19:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:19:59 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:19:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:59 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-18T12:19:59.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:59 smithi186 ceph-mon[135253]: osdmap e20: 3 total, 3 up, 3 in 2023-10-18T12:19:59.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:19:59 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:19:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 -- 172.21.15.70:0/711119517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 msgr2=0x7f0b58100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/711119517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58100d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009990 tx=0x7f0b44009ca0 comp rx=0 tx=0).stop 2023-10-18T12:19:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 -- 172.21.15.70:0/711119517 shutdown_connections 2023-10-18T12:19:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/711119517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b58101260 0x7f0b5810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/711119517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b581057c0 0x7f0b58105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/711119517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58100d20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 -- 172.21.15.70:0/711119517 >> 172.21.15.70:0/711119517 conn(0x7f0b580779e0 msgr2=0x7f0b58077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:19:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 -- 172.21.15.70:0/711119517 shutdown_connections 2023-10-18T12:19:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.785+0000 7f0b5edcf700 1 -- 172.21.15.70:0/711119517 wait complete. 2023-10-18T12:19:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.786+0000 7f0b5edcf700 1 Processor -- start 2023-10-18T12:19:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 -- start start 2023-10-18T12:19:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 0x7f0b58199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b581057c0 0x7f0b5819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819dd20 con 0x7f0b581008a0 2023-10-18T12:19:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819de90 con 0x7f0b58101260 2023-10-18T12:19:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.787+0000 7f0b5edcf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819e050 con 0x7f0b581057c0 2023-10-18T12:19:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b5cb6b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 0x7f0b58199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 0x7f0b58199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56204/0 (socket says 172.21.15.70:56204) 2023-10-18T12:19:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b5cb6b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50738/0 (socket says 172.21.15.70:50738) 2023-10-18T12:19:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 -- 172.21.15.70:0/1745035435 learned_addr learned my addr 172.21.15.70:0/1745035435 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:19:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b5d36c700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b581057c0 0x7f0b5819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 -- 172.21.15.70:0/1745035435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b581057c0 msgr2=0x7f0b5819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b581057c0 0x7f0b5819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 -- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 msgr2=0x7f0b58198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:19:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:19:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 -- 172.21.15.70:0/1745035435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440095f0 con 0x7f0b58101260 2023-10-18T12:19:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.788+0000 7f0b57fff700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 0x7f0b58199360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b44005770 tx=0x7f0b440057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:19:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.789+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4401d070 con 0x7f0b58101260 2023-10-18T12:19:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.789+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5819e2d0 con 0x7f0b58101260 2023-10-18T12:19:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.789+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b44021410 con 0x7f0b58101260 2023-10-18T12:19:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.789+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44017650 con 0x7f0b58101260 2023-10-18T12:19:59.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.789+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b581a5b90 con 0x7f0b58101260 2023-10-18T12:19:59.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.790+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b58101f30 con 0x7f0b58101260 2023-10-18T12:19:59.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.792+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f0b440217f0 con 0x7f0b58101260 2023-10-18T12:19:59.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.793+0000 7f0b55ffb700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b480a1440 0x7f0b480a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:19:59.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.793+0000 7f0b5cb6b700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b480a1440 0x7f0b480a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:19:59.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.793+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3452+0+0 (secure 0 0 0) 0x7f0b440ccc60 con 0x7f0b58101260 2023-10-18T12:19:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.794+0000 7f0b5cb6b700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b480a1440 0x7f0b480a3900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b58101930 tx=0x7f0b4c00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:19:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:19:59.797+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0b44097690 con 0x7f0b58101260 2023-10-18T12:20:00.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:00.014+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b5819a180 con 0x7f0b480a1440 2023-10-18T12:20:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-18T12:20:00.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: osdmap e21: 3 total, 3 up, 3 in 2023-10-18T12:20:00.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: pgmap v62: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:00.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: overall HEALTH_OK 2023-10-18T12:20:00.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:00.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:00.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:00.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:00 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:00.606 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130364]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T12:20:00.607 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:00.607 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130364]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:00.607 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130364]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:00.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-18T12:20:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: osdmap e21: 3 total, 3 up, 3 in 2023-10-18T12:20:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: pgmap v62: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: overall HEALTH_OK 2023-10-18T12:20:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:00 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-18T12:20:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: osdmap e21: 3 total, 3 up, 3 in 2023-10-18T12:20:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: pgmap v62: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: overall HEALTH_OK 2023-10-18T12:20:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:00 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:01.131 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130413]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T12:20:01.131 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130413]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:01.131 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130413]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:01.132 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:20:00 smithi070 sudo[130413]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:01.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:01 smithi070 ceph-mon[98639]: osdmap e22: 3 total, 3 up, 3 in 2023-10-18T12:20:01.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:01.405 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:01 smithi152 ceph-mon[134767]: osdmap e22: 3 total, 3 up, 3 in 2023-10-18T12:20:01.405 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:01.406 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146551]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T12:20:01.406 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:01.406 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146551]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:01.406 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146551]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:01.671 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146554]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T12:20:01.672 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:01.672 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146554]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:01.672 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:20:01 smithi152 sudo[146554]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:01.676 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:01 smithi186 ceph-mon[135253]: osdmap e22: 3 total, 3 up, 3 in 2023-10-18T12:20:01.676 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:01.929 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147091]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T12:20:01.930 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:01.930 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147091]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:01.930 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147091]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:02.218 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147094]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T12:20:02.219 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:02.219 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147094]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:02.219 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:20:01 smithi186 sudo[147094]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:02 smithi070 sudo[130532]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T12:20:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:02 smithi070 sudo[130532]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:02 smithi070 sudo[130532]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:02 smithi070 sudo[130532]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:03.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:03.053+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0b44097a60 con 0x7f0b58101260 2023-10-18T12:20:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 sudo[130567]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T12:20:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 sudo[130567]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 sudo[130567]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 sudo[130567]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:03.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:03.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:03.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:03 smithi070 ceph-mon[98639]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:03.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 ceph-mon[134767]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:03.535 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147097]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T12:20:03.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:03.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147097]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:03.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147097]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:03.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147100]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T12:20:03.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:03.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147100]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:03.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 sudo[147100]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:03.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:03.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:03 smithi186 ceph-mon[135253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.028 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 sudo[146557]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T12:20:04.029 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 sudo[146557]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:04.029 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 sudo[146557]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:04.029 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:03 smithi152 sudo[146557]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:04.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:04.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: mgrmap e20: x(active, since 109s), standbys: z, y 2023-10-18T12:20:04.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3216172523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]: dispatch 2023-10-18T12:20:04.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]: dispatch 2023-10-18T12:20:04.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]': finished 2023-10-18T12:20:04.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: osdmap e23: 4 total, 3 up, 4 in 2023-10-18T12:20:04.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:04.359 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:04 smithi070 ceph-mon[98639]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 sudo[146560]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T12:20:04.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 sudo[146560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T12:20:04.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 sudo[146560]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T12:20:04.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 sudo[146560]: pam_unix(sudo:session): session closed for user root 2023-10-18T12:20:04.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:04.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: mgrmap e20: x(active, since 109s), standbys: z, y 2023-10-18T12:20:04.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3216172523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]: dispatch 2023-10-18T12:20:04.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]: dispatch 2023-10-18T12:20:04.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.426 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.426 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]': finished 2023-10-18T12:20:04.426 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: osdmap e23: 4 total, 3 up, 4 in 2023-10-18T12:20:04.426 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:04.426 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:04 smithi152 ceph-mon[134767]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: mgrmap e20: x(active, since 109s), standbys: z, y 2023-10-18T12:20:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3216172523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]: dispatch 2023-10-18T12:20:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]: dispatch 2023-10-18T12:20:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T12:20:04.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T12:20:04.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T12:20:04.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T12:20:04.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8"}]': finished 2023-10-18T12:20:04.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: osdmap e23: 4 total, 3 up, 4 in 2023-10-18T12:20:04.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:04.473 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:04 smithi186 ceph-mon[135253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T12:20:05.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:05 smithi070 ceph-mon[98639]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:05 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/678991690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:05.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:05 smithi152 ceph-mon[134767]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:05 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/678991690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:05 smithi186 ceph-mon[135253]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:05 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/678991690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:07 smithi070 ceph-mon[98639]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:07.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:07 smithi152 ceph-mon[134767]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:07 smithi186 ceph-mon[135253]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:09 smithi070 ceph-mon[98639]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:09.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:09 smithi152 ceph-mon[134767]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:09 smithi186 ceph-mon[135253]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:11.309 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:11 smithi070 ceph-mon[98639]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-10-18T12:20:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:11 smithi152 ceph-mon[134767]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-10-18T12:20:11.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:11 smithi186 ceph-mon[135253]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-10-18T12:20:12.347 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-18T12:20:12.347 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-18T12:20:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:12.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-18T12:20:12.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:13.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:13 smithi070 ceph-mon[98639]: Deploying daemon osd.3 on smithi070 2023-10-18T12:20:13.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:13 smithi070 ceph-mon[98639]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-18T12:20:13.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:13 smithi152 ceph-mon[134767]: Deploying daemon osd.3 on smithi070 2023-10-18T12:20:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:13 smithi152 ceph-mon[134767]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-18T12:20:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:13 smithi186 ceph-mon[135253]: Deploying daemon osd.3 on smithi070 2023-10-18T12:20:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:13 smithi186 ceph-mon[135253]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-18T12:20:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:15 smithi070 ceph-mon[98639]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-18T12:20:15.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:15 smithi152 ceph-mon[134767]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-18T12:20:15.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:15 smithi186 ceph-mon[135253]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-18T12:20:17.338 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:17 smithi070 ceph-mon[98639]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:17.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:17 smithi152 ceph-mon[134767]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:17 smithi186 ceph-mon[135253]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:18.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:18 smithi070 ceph-mon[98639]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:18.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:18 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:18.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:18 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:18.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:18 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:18.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:18 smithi152 ceph-mon[134767]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:18.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:18 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:18.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:18 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:18.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:18 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:18.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:18 smithi186 ceph-mon[135253]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:18.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:18 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:18.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:18 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:18.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:18 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:20.933 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 3 on host 'smithi070' 2023-10-18T12:20:20.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.931+0000 7f0b55ffb700 1 -- 172.21.15.70:0/1745035435 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b5819a180 con 0x7f0b480a1440 2023-10-18T12:20:20.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.933+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b480a1440 msgr2=0x7f0b480a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:20.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.933+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b480a1440 0x7f0b480a3900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b58101930 tx=0x7f0b4c00b410 comp rx=0 tx=0).stop 2023-10-18T12:20:20.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.933+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 msgr2=0x7f0b58199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:20.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.933+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 0x7f0b58199360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b44005770 tx=0x7f0b440057a0 comp rx=0 tx=0).stop 2023-10-18T12:20:20.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 shutdown_connections 2023-10-18T12:20:20.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b581057c0 0x7f0b5819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:20.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b581008a0 0x7f0b58198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:20.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b480a1440 0x7f0b480a3900 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:20.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 --2- 172.21.15.70:0/1745035435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b58101260 0x7f0b58199360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:20.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 >> 172.21.15.70:0/1745035435 conn(0x7f0b580779e0 msgr2=0x7f0b5810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:20:20.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 shutdown_connections 2023-10-18T12:20:20.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:20:20.934+0000 7f0b5edcf700 1 -- 172.21.15.70:0/1745035435 wait complete. 2023-10-18T12:20:21.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:21.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:21.408 DEBUG:teuthology.orchestra.run.smithi070:osd.3> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.3.service 2023-10-18T12:20:21.412 INFO:tasks.cephadm:Deploying osd.4 on smithi152 with /dev/vg_nvme/lv_3... 2023-10-18T12:20:21.413 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-18T12:20:21.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-18T12:20:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:21.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:21.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:21.515 INFO:journalctl@ceph.osd.3.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:20:22.001 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:20:22.342 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:22 smithi070 ceph-mon[98639]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-18T12:20:22.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:22 smithi152 ceph-mon[134767]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-18T12:20:22.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:22 smithi186 ceph-mon[135253]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-18T12:20:23.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-mon[98639]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:23.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-mon[98639]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-18T12:20:23.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-mon[98639]: osdmap e24: 4 total, 3 up, 4 in 2023-10-18T12:20:23.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-mon[98639]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:20:23.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:23.356 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-3[132819]: 2023-10-18T12:20:23.073+0000 7f5df91de700 -1 osd.3 0 waiting for initial osdmap 2023-10-18T12:20:23.356 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:20:23 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-3[132819]: 2023-10-18T12:20:23.079+0000 7f5df37d0700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:20:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:23 smithi152 ceph-mon[134767]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:23 smithi152 ceph-mon[134767]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-18T12:20:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:23 smithi152 ceph-mon[134767]: osdmap e24: 4 total, 3 up, 4 in 2023-10-18T12:20:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:23 smithi152 ceph-mon[134767]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:20:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:23 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:23 smithi186 ceph-mon[135253]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:23 smithi186 ceph-mon[135253]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-18T12:20:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:23 smithi186 ceph-mon[135253]: osdmap e24: 4 total, 3 up, 4 in 2023-10-18T12:20:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:23 smithi186 ceph-mon[135253]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:20:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:23 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:23.494 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-18T12:20:23.494 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-18T12:20:23.494 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2023-10-18T12:20:23.495 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2023-10-18T12:20:23.495 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109704 s, 956 MB/s 2023-10-18T12:20:23.495 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2023-10-18T12:20:23.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2023-10-18T12:20:24.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:20:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: osdmap e25: 4 total, 3 up, 4 in 2023-10-18T12:20:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] boot 2023-10-18T12:20:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: osdmap e26: 4 total, 4 up, 4 in 2023-10-18T12:20:24.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:20:24.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: osdmap e25: 4 total, 3 up, 4 in 2023-10-18T12:20:24.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] boot 2023-10-18T12:20:24.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: osdmap e26: 4 total, 4 up, 4 in 2023-10-18T12:20:24.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.446 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:20:24.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: from='osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:20:24.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: osdmap e25: 4 total, 3 up, 4 in 2023-10-18T12:20:24.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:24.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: osd.3 [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] boot 2023-10-18T12:20:24.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: osdmap e26: 4 total, 4 up, 4 in 2023-10-18T12:20:24.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T12:20:25.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:20:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:20:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: pgmap v78: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:25.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:25.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:25.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:25.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:25 smithi152 ceph-mon[134767]: osdmap e27: 4 total, 4 up, 4 in 2023-10-18T12:20:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:20:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:20:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: pgmap v78: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:25.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:25.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:25.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:25.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:25 smithi070 ceph-mon[98639]: osdmap e27: 4 total, 4 up, 4 in 2023-10-18T12:20:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:20:25.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:20:25.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: pgmap v78: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T12:20:25.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:25.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:25.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:25.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:25 smithi186 ceph-mon[135253]: osdmap e27: 4 total, 4 up, 4 in 2023-10-18T12:20:25.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/4276927229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a00ff190 msgr2=0x7fc4a00ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:25.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/4276927229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a00ff190 0x7fc4a00ff5f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc494009990 tx=0x7fc494009ca0 comp rx=0 tx=0).stop 2023-10-18T12:20:25.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/4276927229 shutdown_connections 2023-10-18T12:20:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/4276927229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a00ffb30 0x7fc4a010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/4276927229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a01058d0 0x7fc4a0105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/4276927229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a00ff190 0x7fc4a00ff5f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/4276927229 >> 172.21.15.152:0/4276927229 conn(0x7fc4a0074a10 msgr2=0x7fc4a0074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:20:25.601 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/4276927229 shutdown_connections 2023-10-18T12:20:25.602 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/4276927229 wait complete. 2023-10-18T12:20:25.602 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.597+0000 7fc4a5c3e700 1 Processor -- start 2023-10-18T12:20:25.604 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 -- start start 2023-10-18T12:20:25.604 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a00ff190 0x7fc4a019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:25.605 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a00ffb30 0x7fc4a019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:25.605 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 0x7fc4a0196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:25.605 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0197100 con 0x7fc4a00ff190 2023-10-18T12:20:25.605 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0197270 con 0x7fc4a01058d0 2023-10-18T12:20:25.605 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc4a5c3e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0197430 con 0x7fc4a00ffb30 2023-10-18T12:20:25.606 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 0x7fc4a0196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:25.606 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 0x7fc4a0196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57726/0 (socket says 172.21.15.152:57726) 2023-10-18T12:20:25.606 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 -- 172.21.15.152:0/3401653620 learned_addr learned my addr 172.21.15.152:0/3401653620 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-10-18T12:20:25.606 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49effd700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a00ffb30 0x7fc4a019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:25.607 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49f7fe700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a00ff190 0x7fc4a019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:25.607 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 -- 172.21.15.152:0/3401653620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a00ffb30 msgr2=0x7fc4a019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:25.607 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a00ffb30 0x7fc4a019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:25.607 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 -- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a00ff190 msgr2=0x7fc4a019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:25.607 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a00ff190 0x7fc4a019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:25.608 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.598+0000 7fc49ffff700 1 -- 172.21.15.152:0/3401653620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940095f0 con 0x7fc4a01058d0 2023-10-18T12:20:25.608 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.599+0000 7fc49ffff700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 0x7fc4a0196aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc49800ada0 tx=0x7fc49800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:20:25.608 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.599+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800f070 con 0x7fc4a01058d0 2023-10-18T12:20:25.608 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.599+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc498007b20 con 0x7fc4a01058d0 2023-10-18T12:20:25.608 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.599+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a0197710 con 0x7fc4a01058d0 2023-10-18T12:20:25.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.599+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498013a30 con 0x7fc4a01058d0 2023-10-18T12:20:25.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.599+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a01a3900 con 0x7fc4a01058d0 2023-10-18T12:20:25.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.600+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a0108d30 con 0x7fc4a01058d0 2023-10-18T12:20:25.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.601+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc49800c8d0 con 0x7fc4a01058d0 2023-10-18T12:20:25.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.601+0000 7fc49cff9700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc4880a14d0 0x7fc4880a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:25.610 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.601+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3913+0+0 (secure 0 0 0) 0x7fc4980cb380 con 0x7fc4a01058d0 2023-10-18T12:20:25.610 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.601+0000 7fc49f7fe700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc4880a14d0 0x7fc4880a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:25.610 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.602+0000 7fc49f7fe700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc4880a14d0 0x7fc4880a3990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0068160 tx=0x7fc49000a380 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:20:25.611 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.606+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc498095b10 con 0x7fc4a01058d0 2023-10-18T12:20:25.806 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:25.803+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4a0197f60 con 0x7fc4880a14d0 2023-10-18T12:20:26.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:26 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi070 2023-10-18T12:20:26.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:26 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi070 to 8635M 2023-10-18T12:20:26.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:26.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:26.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:26.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:26 smithi070 ceph-mon[98639]: osdmap e28: 4 total, 4 up, 4 in 2023-10-18T12:20:26.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:26 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi070 2023-10-18T12:20:26.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:26 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi070 to 8635M 2023-10-18T12:20:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:26 smithi152 ceph-mon[134767]: osdmap e28: 4 total, 4 up, 4 in 2023-10-18T12:20:26.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:26 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi070 2023-10-18T12:20:26.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:26 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi070 to 8635M 2023-10-18T12:20:26.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:26.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:26.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:26.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:26 smithi186 ceph-mon[135253]: osdmap e28: 4 total, 4 up, 4 in 2023-10-18T12:20:27.161 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:27 smithi152 ceph-mon[134767]: pgmap v81: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:27.161 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:27 smithi152 ceph-mon[134767]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:27.161 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:27 smithi152 ceph-mon[134767]: osdmap e29: 4 total, 4 up, 4 in 2023-10-18T12:20:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:27 smithi070 ceph-mon[98639]: pgmap v81: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:27 smithi070 ceph-mon[98639]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:27 smithi070 ceph-mon[98639]: osdmap e29: 4 total, 4 up, 4 in 2023-10-18T12:20:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:27 smithi186 ceph-mon[135253]: pgmap v81: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:27 smithi186 ceph-mon[135253]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:27 smithi186 ceph-mon[135253]: osdmap e29: 4 total, 4 up, 4 in 2023-10-18T12:20:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: pgmap v84: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:29.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: osdmap e30: 4 total, 4 up, 4 in 2023-10-18T12:20:29.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/639412328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]: dispatch 2023-10-18T12:20:29.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]: dispatch 2023-10-18T12:20:29.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]': finished 2023-10-18T12:20:29.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: osdmap e31: 5 total, 4 up, 5 in 2023-10-18T12:20:29.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:29.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:29 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/4232810449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:29.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: pgmap v84: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:29.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: osdmap e30: 4 total, 4 up, 4 in 2023-10-18T12:20:29.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/639412328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]: dispatch 2023-10-18T12:20:29.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]: dispatch 2023-10-18T12:20:29.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]': finished 2023-10-18T12:20:29.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: osdmap e31: 5 total, 4 up, 5 in 2023-10-18T12:20:29.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:29.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:29 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/4232810449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: pgmap v84: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: osdmap e30: 4 total, 4 up, 4 in 2023-10-18T12:20:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/639412328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]: dispatch 2023-10-18T12:20:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]: dispatch 2023-10-18T12:20:29.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff61a4d2-db3e-4df1-abbc-daf21ea63965"}]': finished 2023-10-18T12:20:29.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: osdmap e31: 5 total, 4 up, 5 in 2023-10-18T12:20:29.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:29.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:29 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/4232810449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:31.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:31 smithi070 ceph-mon[98639]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:20:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:31 smithi152 ceph-mon[134767]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:20:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:31 smithi186 ceph-mon[135253]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:20:33.244 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:33 smithi152 ceph-mon[134767]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:20:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:33 smithi070 ceph-mon[98639]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:20:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:33 smithi186 ceph-mon[135253]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:20:35.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:35 smithi070 ceph-mon[98639]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-10-18T12:20:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:35 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-18T12:20:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:35 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:35.383 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:35 smithi152 ceph-mon[134767]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-10-18T12:20:35.383 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:35 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-18T12:20:35.383 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:35 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:35 smithi186 ceph-mon[135253]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-10-18T12:20:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:35 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-18T12:20:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:35 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:36.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:36 smithi070 ceph-mon[98639]: Deploying daemon osd.4 on smithi152 2023-10-18T12:20:36.396 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:36 smithi152 ceph-mon[134767]: Deploying daemon osd.4 on smithi152 2023-10-18T12:20:36.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:36 smithi186 ceph-mon[135253]: Deploying daemon osd.4 on smithi152 2023-10-18T12:20:37.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:37 smithi070 ceph-mon[98639]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:20:37.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:37 smithi152 ceph-mon[134767]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:20:37.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:37 smithi186 ceph-mon[135253]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:20:39.128 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:39 smithi152 ceph-mon[134767]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:20:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:39 smithi070 ceph-mon[98639]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:20:39.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:39 smithi186 ceph-mon[135253]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:20:40.418 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:40.418 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:40.418 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:40.419 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:40 smithi152 ceph-mon[134767]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-18T12:20:40.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:40.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:40.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:40.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:40 smithi186 ceph-mon[135253]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-18T12:20:40.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:40.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:40.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:40.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:40 smithi070 ceph-mon[98639]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-18T12:20:41.972 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.969+0000 7fc49cff9700 1 -- 172.21.15.152:0/3401653620 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc4a0197f60 con 0x7fc4880a14d0 2023-10-18T12:20:41.973 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 4 on host 'smithi152' 2023-10-18T12:20:41.975 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.971+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc4880a14d0 msgr2=0x7fc4880a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:41.975 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc4880a14d0 0x7fc4880a3990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0068160 tx=0x7fc49000a380 comp rx=0 tx=0).stop 2023-10-18T12:20:41.975 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 msgr2=0x7fc4a0196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:41.975 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 0x7fc4a0196aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc49800ada0 tx=0x7fc49800c330 comp rx=0 tx=0).stop 2023-10-18T12:20:41.975 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 shutdown_connections 2023-10-18T12:20:41.976 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc4880a14d0 0x7fc4880a3990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:41.976 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4a01058d0 0x7fc4a0196aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:41.976 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a00ff190 0x7fc4a019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:41.976 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 --2- 172.21.15.152:0/3401653620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a00ffb30 0x7fc4a019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:41.976 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 >> 172.21.15.152:0/3401653620 conn(0x7fc4a0074a10 msgr2=0x7fc4a00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:20:41.976 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 shutdown_connections 2023-10-18T12:20:41.977 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:20:41.972+0000 7fc4a5c3e700 1 -- 172.21.15.152:0/3401653620 wait complete. 2023-10-18T12:20:42.353 DEBUG:teuthology.orchestra.run.smithi152:osd.4> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.4.service 2023-10-18T12:20:42.355 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2023-10-18T12:20:42.356 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-18T12:20:42.387 INFO:journalctl@ceph.osd.4.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:20:42.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:42 smithi152 ceph-mon[134767]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:42 smithi186 ceph-mon[135253]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:42 smithi070 ceph-mon[98639]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:42.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:42.936 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:20:43.171 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:20:43 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:20:43.112+0000 7f62d2b37640 -1 osd.4 0 log_to_monitors true 2023-10-18T12:20:43.716 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:43 smithi152 ceph-mon[134767]: from='osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T12:20:43.717 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:43 smithi152 ceph-mon[134767]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T12:20:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:43 smithi186 ceph-mon[135253]: from='osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T12:20:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:43 smithi186 ceph-mon[135253]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T12:20:43.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:43 smithi070 ceph-mon[98639]: from='osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T12:20:43.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:43 smithi070 ceph-mon[98639]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T12:20:44.396 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-18T12:20:44.396 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-18T12:20:44.397 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-18T12:20:44.397 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-18T12:20:44.397 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111149 s, 943 MB/s 2023-10-18T12:20:44.397 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 2023-10-18T12:20:44.397 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-18T12:20:44.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:44 smithi186 ceph-mon[135253]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:44.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:44 smithi186 ceph-mon[135253]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-18T12:20:44.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:44 smithi186 ceph-mon[135253]: osdmap e32: 5 total, 4 up, 5 in 2023-10-18T12:20:44.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:44 smithi186 ceph-mon[135253]: from='osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:20:44.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:44 smithi186 ceph-mon[135253]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:20:44.776 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2023-10-18T12:20:44.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:44 smithi070 ceph-mon[98639]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:44.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:44 smithi070 ceph-mon[98639]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-18T12:20:44.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:44 smithi070 ceph-mon[98639]: osdmap e32: 5 total, 4 up, 5 in 2023-10-18T12:20:44.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:44.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:44 smithi070 ceph-mon[98639]: from='osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:20:44.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:44 smithi070 ceph-mon[98639]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:20:44.921 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:20:44.603+0000 7f62c81ae700 -1 osd.4 0 waiting for initial osdmap 2023-10-18T12:20:44.922 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:20:44.610+0000 7f62c1f9f700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:20:44.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-mon[134767]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:44.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-mon[134767]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-18T12:20:44.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-mon[134767]: osdmap e32: 5 total, 4 up, 5 in 2023-10-18T12:20:44.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:44.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-mon[134767]: from='osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:20:44.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:44 smithi152 ceph-mon[134767]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:20:45.338 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:20:45.822 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:20:45.822 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: osdmap e33: 5 total, 4 up, 5 in 2023-10-18T12:20:45.822 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:45.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:45.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi152 2023-10-18T12:20:45.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:45.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:45.824 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi152 to 8584M 2023-10-18T12:20:45.824 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.824 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:45.824 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:45.824 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.825 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:45.825 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:45.825 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:45.825 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:45 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:20:45.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: osdmap e33: 5 total, 4 up, 5 in 2023-10-18T12:20:45.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:45.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:45.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi152 2023-10-18T12:20:45.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:45.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:45.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi152 to 8584M 2023-10-18T12:20:45.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:45.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:45.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:45.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:45.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:45.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:45.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:45 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:46.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:20:46.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: osdmap e33: 5 total, 4 up, 5 in 2023-10-18T12:20:46.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:46.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:46.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi152 2023-10-18T12:20:46.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:46.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:46.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:46.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:20:46.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi152 to 8584M 2023-10-18T12:20:46.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:46.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:46.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:46.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:46.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:20:46.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:46.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:20:46.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:45 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:20:46.529 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 -- 172.21.15.186:0/3973564761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50740ff750 msgr2=0x7f50740ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:46.530 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 --2- 172.21.15.186:0/3973564761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50740ff750 0x7f50740ffbd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5064009990 tx=0x7f5064009ca0 comp rx=0 tx=0).stop 2023-10-18T12:20:46.530 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 -- 172.21.15.186:0/3973564761 shutdown_connections 2023-10-18T12:20:46.530 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 --2- 172.21.15.186:0/3973564761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5074100110 0x7f507410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:46.530 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 --2- 172.21.15.186:0/3973564761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50741056b0 0x7f5074105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:46.531 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 --2- 172.21.15.186:0/3973564761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50740ff750 0x7f50740ffbd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:46.531 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 -- 172.21.15.186:0/3973564761 >> 172.21.15.186:0/3973564761 conn(0x7f5074075790 msgr2=0x7f5074075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:20:46.531 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 -- 172.21.15.186:0/3973564761 shutdown_connections 2023-10-18T12:20:46.531 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.527+0000 7f507988c700 1 -- 172.21.15.186:0/3973564761 wait complete. 2023-10-18T12:20:46.533 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 Processor -- start 2023-10-18T12:20:46.533 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 -- start start 2023-10-18T12:20:46.534 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50740ff750 0x7f507419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:46.534 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5074100110 0x7f507419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:46.534 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 0x7f5074195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:46.534 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741960f0 con 0x7f5074100110 2023-10-18T12:20:46.534 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074196260 con 0x7f50740ff750 2023-10-18T12:20:46.535 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f507988c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741963a0 con 0x7f50741056b0 2023-10-18T12:20:46.535 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f50737fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 0x7f5074195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:46.535 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f50737fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 0x7f5074195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:50504/0 (socket says 172.21.15.186:50504) 2023-10-18T12:20:46.535 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f50737fe700 1 -- 172.21.15.186:0/2500004618 learned_addr learned my addr 172.21.15.186:0/2500004618 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-18T12:20:46.535 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f5072ffd700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50740ff750 0x7f507419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:46.536 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.528+0000 7f50727fc700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5074100110 0x7f507419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:46.536 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f50737fe700 1 -- 172.21.15.186:0/2500004618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50740ff750 msgr2=0x7f507419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:46.536 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f50737fe700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50740ff750 0x7f507419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:46.536 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f50737fe700 1 -- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5074100110 msgr2=0x7f507419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:20:46.536 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f50737fe700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5074100110 0x7f507419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:20:46.536 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f50737fe700 1 -- 172.21.15.186:0/2500004618 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640095f0 con 0x7f50741056b0 2023-10-18T12:20:46.537 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f50737fe700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 0x7f5074195940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f506c007ee0 tx=0x7f506c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:20:46.537 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c017070 con 0x7f50741056b0 2023-10-18T12:20:46.537 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074196680 con 0x7f50741056b0 2023-10-18T12:20:46.537 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506c00f040 con 0x7f50741056b0 2023-10-18T12:20:46.537 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c005740 con 0x7f50741056b0 2023-10-18T12:20:46.538 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.529+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5074068360 con 0x7f50741056b0 2023-10-18T12:20:46.538 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.530+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5074100df0 con 0x7f50741056b0 2023-10-18T12:20:46.538 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.531+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f506c00dc60 con 0x7f50741056b0 2023-10-18T12:20:46.539 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.532+0000 7f507888a700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f505c0a14d0 0x7f505c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:20:46.539 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.532+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4316+0+0 (secure 0 0 0) 0x7f506c0cb690 con 0x7f50741056b0 2023-10-18T12:20:46.540 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.532+0000 7f5072ffd700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f505c0a14d0 0x7f505c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:20:46.540 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.532+0000 7f5072ffd700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f505c0a14d0 0x7f505c0a3990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5068005950 tx=0x7f506800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:20:46.540 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.536+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f506c095bf0 con 0x7f50741056b0 2023-10-18T12:20:46.736 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:20:46.734+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f507410f6a0 con 0x7f505c0a14d0 2023-10-18T12:20:46.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:46 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:20:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:46 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:20:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:46 smithi152 ceph-mon[134767]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:46 smithi152 ceph-mon[134767]: osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] boot 2023-10-18T12:20:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:46 smithi152 ceph-mon[134767]: osdmap e34: 5 total, 5 up, 5 in 2023-10-18T12:20:46.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:46 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:20:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:46 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:20:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:46 smithi186 ceph-mon[135253]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:46 smithi186 ceph-mon[135253]: osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] boot 2023-10-18T12:20:46.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:46 smithi186 ceph-mon[135253]: osdmap e34: 5 total, 5 up, 5 in 2023-10-18T12:20:46.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:47.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:46 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:20:47.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:46 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:20:47.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:46 smithi070 ceph-mon[98639]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T12:20:47.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:46 smithi070 ceph-mon[98639]: osd.4 [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] boot 2023-10-18T12:20:47.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:46 smithi070 ceph-mon[98639]: osdmap e34: 5 total, 5 up, 5 in 2023-10-18T12:20:47.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T12:20:47.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:47 smithi186 ceph-mon[135253]: osdmap e35: 5 total, 5 up, 5 in 2023-10-18T12:20:47.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:47 smithi186 ceph-mon[135253]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:47.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:47.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:47.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:47.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:47 smithi152 ceph-mon[134767]: osdmap e35: 5 total, 5 up, 5 in 2023-10-18T12:20:47.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:47 smithi152 ceph-mon[134767]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:47.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:47.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:47.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:48.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:47 smithi070 ceph-mon[98639]: osdmap e35: 5 total, 5 up, 5 in 2023-10-18T12:20:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:47 smithi070 ceph-mon[98639]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:20:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:20:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:20:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:48.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:48 smithi152 ceph-mon[134767]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:48.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:48 smithi186 ceph-mon[135253]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:49.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:48 smithi070 ceph-mon[98639]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:50 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/2976501521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]: dispatch 2023-10-18T12:20:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:50 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]: dispatch 2023-10-18T12:20:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:50 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]': finished 2023-10-18T12:20:50.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:50 smithi070 ceph-mon[98639]: osdmap e36: 6 total, 5 up, 6 in 2023-10-18T12:20:50.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:20:50.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:50 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/90821843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:50.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:50 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/2976501521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]: dispatch 2023-10-18T12:20:50.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:50 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]: dispatch 2023-10-18T12:20:50.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:50 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]': finished 2023-10-18T12:20:50.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:50 smithi152 ceph-mon[134767]: osdmap e36: 6 total, 5 up, 6 in 2023-10-18T12:20:50.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:20:50.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:50 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/90821843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:50.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:50 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/2976501521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]: dispatch 2023-10-18T12:20:50.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:50 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]: dispatch 2023-10-18T12:20:50.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:50 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd152ca6-76f9-4773-887d-9a751fd85319"}]': finished 2023-10-18T12:20:50.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:50 smithi186 ceph-mon[135253]: osdmap e36: 6 total, 5 up, 6 in 2023-10-18T12:20:50.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:20:50.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:50 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/90821843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:20:51.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:51 smithi070 ceph-mon[98639]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:51.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:51 smithi152 ceph-mon[134767]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:51 smithi186 ceph-mon[135253]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:53.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:53 smithi070 ceph-mon[98639]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:53.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:53 smithi152 ceph-mon[134767]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:53 smithi186 ceph-mon[135253]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:55.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:55 smithi070 ceph-mon[98639]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:55.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:55 smithi152 ceph-mon[134767]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:55.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:55 smithi186 ceph-mon[135253]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:56.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:56 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-18T12:20:56.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:56 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:56.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:56 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-18T12:20:56.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:56 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:56.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:56 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-18T12:20:56.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:56 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:20:57.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:57 smithi152 ceph-mon[134767]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:57.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:57 smithi152 ceph-mon[134767]: Deploying daemon osd.5 on smithi186 2023-10-18T12:20:57.443 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:57 smithi186 ceph-mon[135253]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:57.443 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:57 smithi186 ceph-mon[135253]: Deploying daemon osd.5 on smithi186 2023-10-18T12:20:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:57 smithi070 ceph-mon[98639]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:57 smithi070 ceph-mon[98639]: Deploying daemon osd.5 on smithi186 2023-10-18T12:20:59.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:20:59 smithi152 ceph-mon[134767]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:59.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:20:59 smithi186 ceph-mon[135253]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:20:59.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:20:59 smithi070 ceph-mon[98639]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:01.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:01 smithi186 ceph-mon[135253]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:01.204 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:01.204 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:01.204 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:01 smithi070 ceph-mon[98639]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:01.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:01 smithi152 ceph-mon[134767]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:02.950 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2023-10-18T12:21:02.950 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.947+0000 7f507888a700 1 -- 172.21.15.186:0/2500004618 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f507410f6a0 con 0x7f505c0a14d0 2023-10-18T12:21:02.952 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.949+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f505c0a14d0 msgr2=0x7f505c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:02.952 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.949+0000 7f507988c700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f505c0a14d0 0x7f505c0a3990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5068005950 tx=0x7f506800b410 comp rx=0 tx=0).stop 2023-10-18T12:21:02.952 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 msgr2=0x7f5074195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:02.952 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 0x7f5074195940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f506c007ee0 tx=0x7f506c00d3b0 comp rx=0 tx=0).stop 2023-10-18T12:21:02.953 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 shutdown_connections 2023-10-18T12:21:02.953 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50741056b0 0x7f5074195940 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:02.953 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f505c0a14d0 0x7f505c0a3990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:02.953 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50740ff750 0x7f507419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:02.953 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 --2- 172.21.15.186:0/2500004618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5074100110 0x7f507419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:02.954 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 >> 172.21.15.186:0/2500004618 conn(0x7f5074075790 msgr2=0x7f50740fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:21:02.954 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 shutdown_connections 2023-10-18T12:21:02.954 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:02.950+0000 7f507988c700 1 -- 172.21.15.186:0/2500004618 wait complete. 2023-10-18T12:21:03.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:03.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.309 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:03.310 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:03.310 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.310 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.310 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:03.310 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.328 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.5.service 2023-10-18T12:21:03.330 INFO:tasks.cephadm:Deploying osd.6 on smithi070 with /dev/vg_nvme/lv_2... 2023-10-18T12:21:03.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-18T12:21:03.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:03.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:03.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:03 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.359 INFO:journalctl@ceph.osd.5.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:21:03.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:03.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:03.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:03.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:03 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:04.072 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:21:04.136 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:04 smithi186 ceph-mon[135253]: from='osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T12:21:04.136 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:04 smithi186 ceph-mon[135253]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T12:21:04.136 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:21:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:21:03.944+0000 7fb82fc26640 -1 osd.5 0 log_to_monitors true 2023-10-18T12:21:04.338 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:04 smithi070 ceph-mon[98639]: from='osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T12:21:04.338 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:04 smithi070 ceph-mon[98639]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T12:21:04.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:04 smithi152 ceph-mon[134767]: from='osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T12:21:04.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:04 smithi152 ceph-mon[134767]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T12:21:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:05 smithi070 ceph-mon[98639]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:05 smithi070 ceph-mon[98639]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-18T12:21:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:05 smithi070 ceph-mon[98639]: osdmap e37: 6 total, 5 up, 6 in 2023-10-18T12:21:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:05 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:05.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:05 smithi070 ceph-mon[98639]: from='osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:21:05.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:05 smithi070 ceph-mon[98639]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:21:05.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:05 smithi152 ceph-mon[134767]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:05.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:05 smithi152 ceph-mon[134767]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-18T12:21:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:05 smithi152 ceph-mon[134767]: osdmap e37: 6 total, 5 up, 6 in 2023-10-18T12:21:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:05 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:05 smithi152 ceph-mon[134767]: from='osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:21:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:05 smithi152 ceph-mon[134767]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:21:05.452 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:21:05.047+0000 7fb82529d700 -1 osd.5 0 waiting for initial osdmap 2023-10-18T12:21:05.453 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:21:05.054+0000 7fb81f08e700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:21:05.453 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-mon[135253]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:05.453 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-mon[135253]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-18T12:21:05.453 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-mon[135253]: osdmap e37: 6 total, 5 up, 6 in 2023-10-18T12:21:05.453 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:05.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-mon[135253]: from='osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:21:05.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:05 smithi186 ceph-mon[135253]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:21:06.126 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-18T12:21:06.126 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-18T12:21:06.126 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-10-18T12:21:06.126 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-10-18T12:21:06.127 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0393537 s, 266 MB/s 2023-10-18T12:21:06.127 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-10-18T12:21:06.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:21:06.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: osdmap e38: 6 total, 5 up, 6 in 2023-10-18T12:21:06.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:06.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:06.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:06.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:06.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:21:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: osdmap e38: 6 total, 5 up, 6 in 2023-10-18T12:21:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:06.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:06.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:06.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:06.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:21:06.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: osdmap e38: 6 total, 5 up, 6 in 2023-10-18T12:21:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:06.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:06.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:06.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:06.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:06.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2023-10-18T12:21:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:21:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:21:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:21:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi186 2023-10-18T12:21:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-18T12:21:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] boot 2023-10-18T12:21:07.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: osdmap e39: 6 total, 6 up, 6 in 2023-10-18T12:21:07.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:07 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:07.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:21:07.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:21:07.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:07.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi186 2023-10-18T12:21:07.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-18T12:21:07.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:07.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] boot 2023-10-18T12:21:07.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: osdmap e39: 6 total, 6 up, 6 in 2023-10-18T12:21:07.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:07 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:21:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:21:07.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T12:21:07.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi186 2023-10-18T12:21:07.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-18T12:21:07.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:07.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: osd.5 [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] boot 2023-10-18T12:21:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: osdmap e39: 6 total, 6 up, 6 in 2023-10-18T12:21:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:07 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T12:21:08.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:08 smithi070 ceph-mon[98639]: osdmap e40: 6 total, 6 up, 6 in 2023-10-18T12:21:08.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:08 smithi152 ceph-mon[134767]: osdmap e40: 6 total, 6 up, 6 in 2023-10-18T12:21:08.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:08 smithi186 ceph-mon[135253]: osdmap e40: 6 total, 6 up, 6 in 2023-10-18T12:21:08.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 -- 172.21.15.70:0/689763954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac101260 msgr2=0x7ff4ac10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:08.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/689763954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac101260 0x7ff4ac10d650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400b0a0 tx=0x7ff4a400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:21:08.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 -- 172.21.15.70:0/689763954 shutdown_connections 2023-10-18T12:21:08.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/689763954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac101260 0x7ff4ac10d650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:08.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/689763954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1057c0 0x7ff4ac105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:08.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/689763954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:08.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 -- 172.21.15.70:0/689763954 >> 172.21.15.70:0/689763954 conn(0x7ff4ac0779e0 msgr2=0x7ff4ac077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:21:08.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 -- 172.21.15.70:0/689763954 shutdown_connections 2023-10-18T12:21:08.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.842+0000 7ff4b11b7700 1 -- 172.21.15.70:0/689763954 wait complete. 2023-10-18T12:21:08.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.843+0000 7ff4b11b7700 1 Processor -- start 2023-10-18T12:21:08.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 -- start start 2023-10-18T12:21:08.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:08.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac101260 0x7ff4ac199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:08.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac1057c0 0x7ff4ac19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:08.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac19de10 con 0x7ff4ac1057c0 2023-10-18T12:21:08.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac19df80 con 0x7ff4ac101260 2023-10-18T12:21:08.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4b11b7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac19e140 con 0x7ff4ac1008a0 2023-10-18T12:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:43166/0 (socket says 172.21.15.70:43166) 2023-10-18T12:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4aad9d700 1 -- 172.21.15.70:0/2029554064 learned_addr learned my addr 172.21.15.70:0/2029554064 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4ab59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac1057c0 0x7ff4ac19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.844+0000 7ff4aa59c700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac101260 0x7ff4ac199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:08.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aad9d700 1 -- 172.21.15.70:0/2029554064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac101260 msgr2=0x7ff4ac199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:08.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aad9d700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac101260 0x7ff4ac199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:08.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aad9d700 1 -- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac1057c0 msgr2=0x7ff4ac19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:08.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aad9d700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac1057c0 0x7ff4ac19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aad9d700 1 -- 172.21.15.70:0/2029554064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009d00 con 0x7ff4ac1008a0 2023-10-18T12:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4ab59e700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac1057c0 0x7ff4ac19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aa59c700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac101260 0x7ff4ac199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4aad9d700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac198f10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e970 tx=0x7ff49c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c009790 con 0x7ff4ac1008a0 2023-10-18T12:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac19e420 con 0x7ff4ac1008a0 2023-10-18T12:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c00eec0 con 0x7ff4ac1008a0 2023-10-18T12:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c003cf0 con 0x7ff4ac1008a0 2023-10-18T12:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.845+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac1a5c80 con 0x7ff4ac1008a0 2023-10-18T12:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.846+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac101f30 con 0x7ff4ac1008a0 2023-10-18T12:21:08.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.848+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff49c00ccd0 con 0x7ff4ac1008a0 2023-10-18T12:21:08.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.850+0000 7ff493fff700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff4940a14d0 0x7ff4940a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:08.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.850+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4777+0+0 (secure 0 0 0) 0x7ff49c014070 con 0x7ff4ac1008a0 2023-10-18T12:21:08.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.850+0000 7ff4aa59c700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff4940a14d0 0x7ff4940a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:08.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.850+0000 7ff4aa59c700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff4940a14d0 0x7ff4940a3990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac19a470 tx=0x7ff4a0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:21:08.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:08.852+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff49c096780 con 0x7ff4ac1008a0 2023-10-18T12:21:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:09.059+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4ac110780 con 0x7ff4940a14d0 2023-10-18T12:21:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:09 smithi070 ceph-mon[98639]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:09 smithi070 ceph-mon[98639]: osdmap e41: 6 total, 6 up, 6 in 2023-10-18T12:21:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:09 smithi070 ceph-mon[98639]: osdmap e42: 6 total, 6 up, 6 in 2023-10-18T12:21:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:09 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:09.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:09 smithi152 ceph-mon[134767]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:09.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:09 smithi152 ceph-mon[134767]: osdmap e41: 6 total, 6 up, 6 in 2023-10-18T12:21:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:09 smithi152 ceph-mon[134767]: osdmap e42: 6 total, 6 up, 6 in 2023-10-18T12:21:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:09 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:09 smithi186 ceph-mon[135253]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:09 smithi186 ceph-mon[135253]: osdmap e41: 6 total, 6 up, 6 in 2023-10-18T12:21:09.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:09 smithi186 ceph-mon[135253]: osdmap e42: 6 total, 6 up, 6 in 2023-10-18T12:21:09.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:09 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:10.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:10 smithi070 ceph-mon[98639]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:10.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:10 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:10.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:10 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:10.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:10 smithi070 ceph-mon[98639]: osdmap e43: 6 total, 6 up, 6 in 2023-10-18T12:21:10.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:10 smithi152 ceph-mon[134767]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:10 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:10 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:10 smithi152 ceph-mon[134767]: osdmap e43: 6 total, 6 up, 6 in 2023-10-18T12:21:10.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:10 smithi186 ceph-mon[135253]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:10.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:10 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:10.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:10 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:10.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:10 smithi186 ceph-mon[135253]: osdmap e43: 6 total, 6 up, 6 in 2023-10-18T12:21:11.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:11 smithi070 ceph-mon[98639]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:21:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:11 smithi152 ceph-mon[134767]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:21:11.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:11 smithi186 ceph-mon[135253]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:21:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1225329632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09373159-f65f-4a33-a0cd-7adaba830621"}]: dispatch 2023-10-18T12:21:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1225329632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09373159-f65f-4a33-a0cd-7adaba830621"}]': finished 2023-10-18T12:21:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:12 smithi070 ceph-mon[98639]: osdmap e44: 7 total, 6 up, 7 in 2023-10-18T12:21:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:12 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1225329632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09373159-f65f-4a33-a0cd-7adaba830621"}]: dispatch 2023-10-18T12:21:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:12 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1225329632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09373159-f65f-4a33-a0cd-7adaba830621"}]': finished 2023-10-18T12:21:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:12 smithi152 ceph-mon[134767]: osdmap e44: 7 total, 6 up, 7 in 2023-10-18T12:21:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:12.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:12 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1225329632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09373159-f65f-4a33-a0cd-7adaba830621"}]: dispatch 2023-10-18T12:21:12.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:12 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1225329632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09373159-f65f-4a33-a0cd-7adaba830621"}]': finished 2023-10-18T12:21:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:12 smithi186 ceph-mon[135253]: osdmap e44: 7 total, 6 up, 7 in 2023-10-18T12:21:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:13.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:13 smithi070 ceph-mon[98639]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-10-18T12:21:13.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:13 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3263583470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:13.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:13 smithi152 ceph-mon[134767]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-10-18T12:21:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:13 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3263583470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:13.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:13 smithi186 ceph-mon[135253]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-10-18T12:21:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:13 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3263583470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:13.854 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:21:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:21:13.537+0000 7f3330d1e700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-18T12:21:15.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:15 smithi070 ceph-mon[98639]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:21:15.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:15 smithi152 ceph-mon[134767]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:21:15.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:15 smithi186 ceph-mon[135253]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:21:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:17 smithi070 ceph-mon[98639]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:17.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:17 smithi152 ceph-mon[134767]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:17 smithi186 ceph-mon[135253]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:19 smithi070 ceph-mon[98639]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:19.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:19 smithi152 ceph-mon[134767]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:19.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:19 smithi186 ceph-mon[135253]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:20.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-18T12:21:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:20.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-18T12:21:20.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:20.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-18T12:21:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:21.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:21 smithi070 ceph-mon[98639]: Deploying daemon osd.6 on smithi070 2023-10-18T12:21:21.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:21 smithi070 ceph-mon[98639]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:21.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:21 smithi152 ceph-mon[134767]: Deploying daemon osd.6 on smithi070 2023-10-18T12:21:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:21 smithi152 ceph-mon[134767]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:21 smithi186 ceph-mon[135253]: Deploying daemon osd.6 on smithi070 2023-10-18T12:21:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:21 smithi186 ceph-mon[135253]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:23.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:23 smithi070 ceph-mon[98639]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:23.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:23 smithi152 ceph-mon[134767]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:23 smithi186 ceph-mon[135253]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:25.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:25 smithi152 ceph-mon[134767]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:25 smithi070 ceph-mon[98639]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:25 smithi186 ceph-mon[135253]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:26.905 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:26 smithi070 ceph-mon[98639]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:26.905 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:26.906 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:26.906 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:26 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:26.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:26 smithi152 ceph-mon[134767]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:26.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:26.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:26.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:26 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:26.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:26 smithi186 ceph-mon[135253]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:26.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:26.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:26.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:26 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:28.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:28 smithi070 ceph-mon[98639]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:28.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:28 smithi152 ceph-mon[134767]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:28.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:28 smithi186 ceph-mon[135253]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.019+0000 7ff493fff700 1 -- 172.21.15.70:0/2029554064 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff4ac110780 con 0x7ff4940a14d0 2023-10-18T12:21:29.022 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 6 on host 'smithi070' 2023-10-18T12:21:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff4940a14d0 msgr2=0x7ff4940a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff4940a14d0 0x7ff4940a3990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac19a470 tx=0x7ff4a0009260 comp rx=0 tx=0).stop 2023-10-18T12:21:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 msgr2=0x7ff4ac198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac198f10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e970 tx=0x7ff49c00ec80 comp rx=0 tx=0).stop 2023-10-18T12:21:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 shutdown_connections 2023-10-18T12:21:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4ac1057c0 0x7ff4ac19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4ac1008a0 0x7ff4ac198f10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff4940a14d0 0x7ff4940a3990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:29.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 --2- 172.21.15.70:0/2029554064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ac101260 0x7ff4ac199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:29.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 >> 172.21.15.70:0/2029554064 conn(0x7ff4ac0779e0 msgr2=0x7ff4ac10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:21:29.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 shutdown_connections 2023-10-18T12:21:29.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:21:29.023+0000 7ff4b11b7700 1 -- 172.21.15.70:0/2029554064 wait complete. 2023-10-18T12:21:29.454 DEBUG:teuthology.orchestra.run.smithi070:osd.6> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.6.service 2023-10-18T12:21:29.458 INFO:tasks.cephadm:Deploying osd.7 on smithi152 with /dev/vg_nvme/lv_2... 2023-10-18T12:21:29.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-18T12:21:29.531 INFO:journalctl@ceph.osd.6.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:21:29.730 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:21:29.665+0000 7f181a639640 -1 osd.6 0 log_to_monitors true 2023-10-18T12:21:29.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:29.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:29.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:29.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:29.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:29.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:29.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:29.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:29.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:29 smithi152 ceph-mon[134767]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-18T12:21:30.031 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:21:30.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:30.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:30.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:30.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:29 smithi070 ceph-mon[98639]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-18T12:21:30.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:30.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:30.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:29 smithi186 ceph-mon[135253]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-18T12:21:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-mon[98639]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-mon[98639]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-18T12:21:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-mon[98639]: osdmap e45: 7 total, 6 up, 7 in 2023-10-18T12:21:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-mon[98639]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:21:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:31.356 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:21:31.097+0000 7f180fcb0700 -1 osd.6 0 waiting for initial osdmap 2023-10-18T12:21:31.356 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:21:31 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:21:31.107+0000 7f1807a9d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:21:31.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:31 smithi152 ceph-mon[134767]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:31.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:31 smithi152 ceph-mon[134767]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-18T12:21:31.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:31 smithi152 ceph-mon[134767]: osdmap e45: 7 total, 6 up, 7 in 2023-10-18T12:21:31.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:31 smithi152 ceph-mon[134767]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:21:31.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:31 smithi186 ceph-mon[135253]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:31 smithi186 ceph-mon[135253]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-18T12:21:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:31 smithi186 ceph-mon[135253]: osdmap e45: 7 total, 6 up, 7 in 2023-10-18T12:21:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:31 smithi186 ceph-mon[135253]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:21:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:31.539 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-18T12:21:31.540 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-18T12:21:31.540 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2023-10-18T12:21:31.540 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2023-10-18T12:21:31.540 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124024 s, 845 MB/s 2023-10-18T12:21:31.540 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2023-10-18T12:21:31.885 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2023-10-18T12:21:32.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:32 smithi152 ceph-mon[134767]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:21:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:32 smithi152 ceph-mon[134767]: osdmap e46: 7 total, 6 up, 7 in 2023-10-18T12:21:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:32 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:32 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:32 smithi070 ceph-mon[98639]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:21:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:32 smithi070 ceph-mon[98639]: osdmap e46: 7 total, 6 up, 7 in 2023-10-18T12:21:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:32 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:32 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:32.445 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:21:32.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:32 smithi186 ceph-mon[135253]: from='osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:21:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:32 smithi186 ceph-mon[135253]: osdmap e46: 7 total, 6 up, 7 in 2023-10-18T12:21:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:32 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:32 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:21:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:21:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:33.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] boot 2023-10-18T12:21:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: osdmap e47: 7 total, 7 up, 7 in 2023-10-18T12:21:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:33.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:33.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:21:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:21:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] boot 2023-10-18T12:21:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: osdmap e47: 7 total, 7 up, 7 in 2023-10-18T12:21:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:33.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:33.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:21:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:21:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T12:21:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: osd.6 [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] boot 2023-10-18T12:21:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: osdmap e47: 7 total, 7 up, 7 in 2023-10-18T12:21:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T12:21:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:33.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:33.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:33.746 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- 172.21.15.152:0/3302649089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580ff750 msgr2=0x7fe7580ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:33.752 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- 172.21.15.152:0/3302649089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580ff750 0x7fe7580ffbd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe744009990 tx=0x7fe744009ca0 comp rx=0 tx=0).stop 2023-10-18T12:21:33.753 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- 172.21.15.152:0/3302649089 shutdown_connections 2023-10-18T12:21:33.753 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- 172.21.15.152:0/3302649089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 0x7fe75810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:33.753 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- 172.21.15.152:0/3302649089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7581056b0 0x7fe758105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:33.753 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- 172.21.15.152:0/3302649089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7580ff750 0x7fe7580ffbd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:33.753 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- 172.21.15.152:0/3302649089 >> 172.21.15.152:0/3302649089 conn(0x7fe758075790 msgr2=0x7fe758075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:21:33.754 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- 172.21.15.152:0/3302649089 shutdown_connections 2023-10-18T12:21:33.754 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- 172.21.15.152:0/3302649089 wait complete. 2023-10-18T12:21:33.754 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 Processor -- start 2023-10-18T12:21:33.754 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- start start 2023-10-18T12:21:33.755 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580ff750 0x7fe75819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:33.755 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 0x7fe75819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:33.755 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 0x7fe758196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7581971d0 con 0x7fe7580ff750 2023-10-18T12:21:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758197340 con 0x7fe7581056b0 2023-10-18T12:21:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.742+0000 7fe75eef9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758197500 con 0x7fe758100110 2023-10-18T12:21:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 0x7fe758196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:33.757 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 0x7fe758196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57838/0 (socket says 172.21.15.152:57838) 2023-10-18T12:21:33.757 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 -- 172.21.15.152:0/1506976719 learned_addr learned my addr 172.21.15.152:0/1506976719 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-10-18T12:21:33.757 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75cc95700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580ff750 0x7fe75819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:33.757 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe757fff700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 0x7fe75819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:33.757 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 -- 172.21.15.152:0/1506976719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 msgr2=0x7fe75819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 0x7fe75819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 -- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580ff750 msgr2=0x7fe75819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580ff750 0x7fe75819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 -- 172.21.15.152:0/1506976719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440095f0 con 0x7fe7581056b0 2023-10-18T12:21:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe757fff700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 0x7fe75819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:21:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75d496700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 0x7fe758196b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe75000b970 tx=0x7fe75000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:21:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000c930 con 0x7fe7581056b0 2023-10-18T12:21:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe750014440 con 0x7fe7581056b0 2023-10-18T12:21:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7581977e0 con 0x7fe7581056b0 2023-10-18T12:21:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.743+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500075d0 con 0x7fe7581056b0 2023-10-18T12:21:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.744+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7581a39d0 con 0x7fe7581056b0 2023-10-18T12:21:33.760 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.744+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe758100df0 con 0x7fe7581056b0 2023-10-18T12:21:33.761 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.745+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe7500145b0 con 0x7fe7581056b0 2023-10-18T12:21:33.761 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.746+0000 7fe755ffb700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7480a14d0 0x7fe7480a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:33.761 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.746+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5180+0+0 (secure 0 0 0) 0x7fe750006e40 con 0x7fe7581056b0 2023-10-18T12:21:33.761 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.746+0000 7fe75cc95700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7480a14d0 0x7fe7480a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:33.761 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.747+0000 7fe75cc95700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7480a14d0 0x7fe7480a3990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009a10 tx=0x7fe74c009190 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:21:33.762 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.750+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe75004f100 con 0x7fe7581056b0 2023-10-18T12:21:33.949 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:33.948+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe758198050 con 0x7fe7480a14d0 2023-10-18T12:21:34.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:34 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi070 2023-10-18T12:21:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:34 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi070 to 5757M 2023-10-18T12:21:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:34 smithi070 ceph-mon[98639]: osdmap e48: 7 total, 7 up, 7 in 2023-10-18T12:21:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:34.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:34.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:34 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi070 2023-10-18T12:21:34.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:34 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi070 to 5757M 2023-10-18T12:21:34.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:34 smithi152 ceph-mon[134767]: osdmap e48: 7 total, 7 up, 7 in 2023-10-18T12:21:34.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:34.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:34.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:34.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:34 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi070 2023-10-18T12:21:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:34 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi070 to 5757M 2023-10-18T12:21:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:34 smithi186 ceph-mon[135253]: osdmap e48: 7 total, 7 up, 7 in 2023-10-18T12:21:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:35.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:35 smithi152 ceph-mon[134767]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:35.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:35 smithi152 ceph-mon[134767]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:35 smithi186 ceph-mon[135253]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:35 smithi186 ceph-mon[135253]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:35 smithi070 ceph-mon[98639]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:35.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:35 smithi070 ceph-mon[98639]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:37.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:37.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/3580767114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]: dispatch 2023-10-18T12:21:37.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]: dispatch 2023-10-18T12:21:37.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]': finished 2023-10-18T12:21:37.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: osdmap e49: 8 total, 7 up, 8 in 2023-10-18T12:21:37.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:37.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:37 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/1526997305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:37.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/3580767114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]: dispatch 2023-10-18T12:21:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]: dispatch 2023-10-18T12:21:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]': finished 2023-10-18T12:21:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: osdmap e49: 8 total, 7 up, 8 in 2023-10-18T12:21:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:37 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/1526997305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/3580767114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]: dispatch 2023-10-18T12:21:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]: dispatch 2023-10-18T12:21:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2979cb11-038c-482c-b796-7d8dc3f6da42"}]': finished 2023-10-18T12:21:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: osdmap e49: 8 total, 7 up, 8 in 2023-10-18T12:21:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:37 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/1526997305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:39.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:39 smithi152 ceph-mon[134767]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:39.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:39 smithi186 ceph-mon[135253]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:39.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:39 smithi070 ceph-mon[98639]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:41.411 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:41 smithi152 ceph-mon[134767]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:41.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:41 smithi186 ceph-mon[135253]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:41 smithi070 ceph-mon[98639]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:43.372 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:43 smithi152 ceph-mon[134767]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:43.372 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:43 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-18T12:21:43.373 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:43 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:43.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:43 smithi186 ceph-mon[135253]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:43.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:43 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-18T12:21:43.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:43 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:43.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:43 smithi070 ceph-mon[98639]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:43.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:43 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-18T12:21:43.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:43 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:44.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:44 smithi152 ceph-mon[134767]: Deploying daemon osd.7 on smithi152 2023-10-18T12:21:44.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:44 smithi186 ceph-mon[135253]: Deploying daemon osd.7 on smithi152 2023-10-18T12:21:44.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:44 smithi070 ceph-mon[98639]: Deploying daemon osd.7 on smithi152 2023-10-18T12:21:45.325 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:45 smithi152 ceph-mon[134767]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:45.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:45 smithi186 ceph-mon[135253]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:45 smithi070 ceph-mon[98639]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:47.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:47 smithi152 ceph-mon[134767]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:47.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:47 smithi186 ceph-mon[135253]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:47.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:47 smithi070 ceph-mon[98639]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:48.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:48 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:48.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:48 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:48.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:48 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:48.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:48 smithi152 ceph-mon[134767]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:48.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:48 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:48 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:48 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:48 smithi070 ceph-mon[98639]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:48.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:48 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:48 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:48 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:48 smithi186 ceph-mon[135253]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:50.223 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 7 on host 'smithi152' 2023-10-18T12:21:50.224 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.220+0000 7fe755ffb700 1 -- 172.21.15.152:0/1506976719 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe758198050 con 0x7fe7480a14d0 2023-10-18T12:21:50.226 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7480a14d0 msgr2=0x7fe7480a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:50.226 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7480a14d0 0x7fe7480a3990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009a10 tx=0x7fe74c009190 comp rx=0 tx=0).stop 2023-10-18T12:21:50.226 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 msgr2=0x7fe758196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:50.226 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 0x7fe758196b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe75000b970 tx=0x7fe75000bc80 comp rx=0 tx=0).stop 2023-10-18T12:21:50.226 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 shutdown_connections 2023-10-18T12:21:50.227 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7480a14d0 0x7fe7480a3990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:50.227 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7581056b0 0x7fe758196b70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:50.227 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580ff750 0x7fe75819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:50.227 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 --2- 172.21.15.152:0/1506976719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe758100110 0x7fe75819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:50.227 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 >> 172.21.15.152:0/1506976719 conn(0x7fe758075790 msgr2=0x7fe758103500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:21:50.228 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 shutdown_connections 2023-10-18T12:21:50.228 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:21:50.223+0000 7fe75eef9700 1 -- 172.21.15.152:0/1506976719 wait complete. 2023-10-18T12:21:50.627 DEBUG:teuthology.orchestra.run.smithi152:osd.7> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.7.service 2023-10-18T12:21:50.629 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2023-10-18T12:21:50.629 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-18T12:21:50.659 INFO:journalctl@ceph.osd.7.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:21:50.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:50 smithi152 ceph-mon[134767]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:50.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:50 smithi186 ceph-mon[135253]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:50.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:50.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:51.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:50 smithi070 ceph-mon[98639]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:51.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:51.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:51.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:51.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:51.199 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:21:51.482 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:21:51 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:21:51.238+0000 7f04d610a640 -1 osd.7 0 log_to_monitors true 2023-10-18T12:21:51.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:51 smithi186 ceph-mon[135253]: from='osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T12:21:51.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:51 smithi186 ceph-mon[135253]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T12:21:51.995 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:51 smithi152 ceph-mon[134767]: from='osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T12:21:51.995 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:51 smithi152 ceph-mon[134767]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T12:21:52.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:51 smithi070 ceph-mon[98639]: from='osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T12:21:52.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:51 smithi070 ceph-mon[98639]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T12:21:52.659 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-18T12:21:52.660 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-18T12:21:52.660 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-18T12:21:52.660 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-18T12:21:52.660 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111647 s, 939 MB/s 2023-10-18T12:21:52.660 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-18T12:21:52.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:52 smithi186 ceph-mon[135253]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:52.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:52 smithi186 ceph-mon[135253]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-18T12:21:52.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:52 smithi186 ceph-mon[135253]: from='osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:21:52.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:52 smithi186 ceph-mon[135253]: osdmap e50: 8 total, 7 up, 8 in 2023-10-18T12:21:52.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:52.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:52 smithi186 ceph-mon[135253]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:21:53.039 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2023-10-18T12:21:53.048 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:21:52.740+0000 7f04cb781700 -1 osd.7 0 waiting for initial osdmap 2023-10-18T12:21:53.048 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:21:52.752+0000 7f04c5572700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:21:53.048 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-mon[134767]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:53.048 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-mon[134767]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-18T12:21:53.049 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-mon[134767]: from='osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:21:53.049 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-mon[134767]: osdmap e50: 8 total, 7 up, 8 in 2023-10-18T12:21:53.049 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:53.049 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:52 smithi152 ceph-mon[134767]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:21:53.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:52 smithi070 ceph-mon[98639]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:52 smithi070 ceph-mon[98639]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-18T12:21:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:52 smithi070 ceph-mon[98639]: from='osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:21:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:52 smithi070 ceph-mon[98639]: osdmap e50: 8 total, 7 up, 8 in 2023-10-18T12:21:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:52 smithi070 ceph-mon[98639]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:21:53.598 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:21:54.001 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:21:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: osdmap e51: 8 total, 7 up, 8 in 2023-10-18T12:21:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:54.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:54.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:54.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:54.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:21:54.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: osdmap e51: 8 total, 7 up, 8 in 2023-10-18T12:21:54.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:54.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:54.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:54.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:54.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:54.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:21:54.190 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: osdmap e51: 8 total, 7 up, 8 in 2023-10-18T12:21:54.190 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.191 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.191 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.191 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.191 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.191 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.192 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:21:54.192 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.192 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:54.192 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:54.192 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.193 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:21:54.193 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:54.193 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:21:54.193 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:21:54.193 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 -- 172.21.15.186:0/2437430285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e40ff750 msgr2=0x7fa1e40ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:54.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 --2- 172.21.15.186:0/2437430285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e40ff750 0x7fa1e40ffbd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009990 tx=0x7fa1d4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:21:54.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 -- 172.21.15.186:0/2437430285 shutdown_connections 2023-10-18T12:21:54.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 --2- 172.21.15.186:0/2437430285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e4100110 0x7fa1e410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:54.769 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 --2- 172.21.15.186:0/2437430285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e41056b0 0x7fa1e4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:54.770 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 --2- 172.21.15.186:0/2437430285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e40ff750 0x7fa1e40ffbd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:54.770 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 -- 172.21.15.186:0/2437430285 >> 172.21.15.186:0/2437430285 conn(0x7fa1e4075790 msgr2=0x7fa1e4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:21:54.770 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 -- 172.21.15.186:0/2437430285 shutdown_connections 2023-10-18T12:21:54.770 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.766+0000 7fa1e9137700 1 -- 172.21.15.186:0/2437430285 wait complete. 2023-10-18T12:21:54.772 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 Processor -- start 2023-10-18T12:21:54.773 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 -- start start 2023-10-18T12:21:54.773 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e40ff750 0x7fa1e419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:54.773 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 0x7fa1e419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:54.773 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e41056b0 0x7fa1e4195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:54.773 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41960b0 con 0x7fa1e40ff750 2023-10-18T12:21:54.774 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e4196220 con 0x7fa1e41056b0 2023-10-18T12:21:54.774 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e9137700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41963e0 con 0x7fa1e4100110 2023-10-18T12:21:54.774 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 0x7fa1e419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:54.774 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 0x7fa1e419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:50614/0 (socket says 172.21.15.186:50614) 2023-10-18T12:21:54.775 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.767+0000 7fa1e259c700 1 -- 172.21.15.186:0/1151307677 learned_addr learned my addr 172.21.15.186:0/1151307677 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-18T12:21:54.775 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e2d9d700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e40ff750 0x7fa1e419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:54.775 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e359e700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e41056b0 0x7fa1e4195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:54.775 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e259c700 1 -- 172.21.15.186:0/1151307677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e41056b0 msgr2=0x7fa1e4195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:54.775 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e259c700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e41056b0 0x7fa1e4195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:54.776 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e259c700 1 -- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e40ff750 msgr2=0x7fa1e419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:21:54.776 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e259c700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e40ff750 0x7fa1e419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:21:54.776 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e259c700 1 -- 172.21.15.186:0/1151307677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40095f0 con 0x7fa1e4100110 2023-10-18T12:21:54.776 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e259c700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 0x7fa1e419bd90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4000c00 tx=0x7fa1d400f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:21:54.776 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d401d070 con 0x7fa1e4100110 2023-10-18T12:21:54.776 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e4196660 con 0x7fa1e4100110 2023-10-18T12:21:54.777 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d400fd30 con 0x7fa1e4100110 2023-10-18T12:21:54.777 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4017630 con 0x7fa1e4100110 2023-10-18T12:21:54.777 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.768+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e41a27d0 con 0x7fa1e4100110 2023-10-18T12:21:54.777 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.769+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e4107cd0 con 0x7fa1e4100110 2023-10-18T12:21:54.777 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.770+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa1d4004830 con 0x7fa1e4100110 2023-10-18T12:21:54.778 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.771+0000 7fa1dbfff700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1d00a14d0 0x7fa1d00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:21:54.778 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.771+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5641+0+0 (secure 0 0 0) 0x7fa1d40cd8d0 con 0x7fa1e4100110 2023-10-18T12:21:54.779 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.771+0000 7fa1e2d9d700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1d00a14d0 0x7fa1d00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:21:54.779 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.772+0000 7fa1e2d9d700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1d00a14d0 0x7fa1d00a3990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009a10 tx=0x7fa1cc009190 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:21:54.779 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.774+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa1d4097a30 con 0x7fa1e4100110 2023-10-18T12:21:54.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:21:54.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:21:54.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi152 2023-10-18T12:21:54.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi152 to 5722M 2023-10-18T12:21:54.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:54.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] boot 2023-10-18T12:21:54.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: osdmap e52: 8 total, 8 up, 8 in 2023-10-18T12:21:54.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:54.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:54 smithi186 ceph-mon[135253]: osdmap e53: 8 total, 8 up, 8 in 2023-10-18T12:21:54.974 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:21:54.972+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1e404e530 con 0x7fa1d00a14d0 2023-10-18T12:21:55.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:21:55.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:21:55.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi152 2023-10-18T12:21:55.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi152 to 5722M 2023-10-18T12:21:55.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:55.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] boot 2023-10-18T12:21:55.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: osdmap e52: 8 total, 8 up, 8 in 2023-10-18T12:21:55.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:55.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:54 smithi070 ceph-mon[98639]: osdmap e53: 8 total, 8 up, 8 in 2023-10-18T12:21:55.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:21:55.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:21:55.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi152 2023-10-18T12:21:55.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi152 to 5722M 2023-10-18T12:21:55.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T12:21:55.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: osd.7 [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] boot 2023-10-18T12:21:55.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: osdmap e52: 8 total, 8 up, 8 in 2023-10-18T12:21:55.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T12:21:55.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:54 smithi152 ceph-mon[134767]: osdmap e53: 8 total, 8 up, 8 in 2023-10-18T12:21:56.095 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:55 smithi186 ceph-mon[135253]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:56.095 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:56.095 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:56.096 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:56.096 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:55 smithi186 ceph-mon[135253]: osdmap e54: 8 total, 8 up, 8 in 2023-10-18T12:21:56.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:55 smithi070 ceph-mon[98639]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:56.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:56.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:56.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:56.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:55 smithi070 ceph-mon[98639]: osdmap e54: 8 total, 8 up, 8 in 2023-10-18T12:21:56.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:55 smithi152 ceph-mon[134767]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:21:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:21:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:21:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:21:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:55 smithi152 ceph-mon[134767]: osdmap e54: 8 total, 8 up, 8 in 2023-10-18T12:21:57.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:57 smithi070 ceph-mon[98639]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:21:57.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:57 smithi070 ceph-mon[98639]: osdmap e55: 8 total, 8 up, 8 in 2023-10-18T12:21:57.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:57 smithi152 ceph-mon[134767]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:21:57.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:57 smithi152 ceph-mon[134767]: osdmap e55: 8 total, 8 up, 8 in 2023-10-18T12:21:57.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:57 smithi186 ceph-mon[135253]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:21:57.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:57 smithi186 ceph-mon[135253]: osdmap e55: 8 total, 8 up, 8 in 2023-10-18T12:21:58.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: osdmap e56: 8 total, 8 up, 8 in 2023-10-18T12:21:58.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/2463383384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]: dispatch 2023-10-18T12:21:58.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]: dispatch 2023-10-18T12:21:58.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]': finished 2023-10-18T12:21:58.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: osdmap e57: 9 total, 8 up, 9 in 2023-10-18T12:21:58.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:21:58.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/32031945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:58.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: osdmap e56: 8 total, 8 up, 8 in 2023-10-18T12:21:58.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/2463383384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]: dispatch 2023-10-18T12:21:58.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]: dispatch 2023-10-18T12:21:58.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]': finished 2023-10-18T12:21:58.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: osdmap e57: 9 total, 8 up, 9 in 2023-10-18T12:21:58.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:21:58.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:58 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/32031945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:58.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: osdmap e56: 8 total, 8 up, 8 in 2023-10-18T12:21:58.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/2463383384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]: dispatch 2023-10-18T12:21:58.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]: dispatch 2023-10-18T12:21:58.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba13df86-7cc5-4b47-8890-c35b0f2b2ce0"}]': finished 2023-10-18T12:21:58.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: osdmap e57: 9 total, 8 up, 9 in 2023-10-18T12:21:58.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:21:58.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/32031945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:21:59.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:21:59 smithi070 ceph-mon[98639]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:21:59.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:21:59 smithi152 ceph-mon[134767]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:21:59.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:21:59 smithi186 ceph-mon[135253]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:01 smithi070 ceph-mon[98639]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:01.357 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:01 smithi186 ceph-mon[135253]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:01.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:01 smithi152 ceph-mon[134767]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:03.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:03 smithi070 ceph-mon[98639]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:03 smithi152 ceph-mon[134767]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:03 smithi186 ceph-mon[135253]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:04.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-18T12:22:04.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:04.452 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-18T12:22:04.452 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-18T12:22:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:05.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:05 smithi186 ceph-mon[135253]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:05.402 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:05 smithi186 ceph-mon[135253]: Deploying daemon osd.8 on smithi186 2023-10-18T12:22:05.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:05 smithi152 ceph-mon[134767]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:05.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:05 smithi152 ceph-mon[134767]: Deploying daemon osd.8 on smithi186 2023-10-18T12:22:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:05 smithi070 ceph-mon[98639]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:05.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:05 smithi070 ceph-mon[98639]: Deploying daemon osd.8 on smithi186 2023-10-18T12:22:07.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:07 smithi186 ceph-mon[135253]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:07.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:07 smithi152 ceph-mon[134767]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:07.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:07 smithi070 ceph-mon[98639]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:08.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:08 smithi186 ceph-mon[135253]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:08.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:08.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:08.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:09.099 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:08 smithi152 ceph-mon[134767]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:09.099 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:09.100 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:09.100 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:09.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:08 smithi070 ceph-mon[98639]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:09.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:09.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:09.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:10.587 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2023-10-18T12:22:10.588 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.583+0000 7fa1dbfff700 1 -- 172.21.15.186:0/1151307677 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa1e404e530 con 0x7fa1d00a14d0 2023-10-18T12:22:10.589 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1d00a14d0 msgr2=0x7fa1d00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:10.589 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1d00a14d0 0x7fa1d00a3990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009a10 tx=0x7fa1cc009190 comp rx=0 tx=0).stop 2023-10-18T12:22:10.590 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 msgr2=0x7fa1e419bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:10.590 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 0x7fa1e419bd90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4000c00 tx=0x7fa1d400f7d0 comp rx=0 tx=0).stop 2023-10-18T12:22:10.590 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 shutdown_connections 2023-10-18T12:22:10.590 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1d00a14d0 0x7fa1d00a3990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:10.591 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e41056b0 0x7fa1e4195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:10.591 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e40ff750 0x7fa1e419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:10.591 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 --2- 172.21.15.186:0/1151307677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e4100110 0x7fa1e419bd90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:10.591 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 >> 172.21.15.186:0/1151307677 conn(0x7fa1e4075790 msgr2=0x7fa1e40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:22:10.591 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 shutdown_connections 2023-10-18T12:22:10.592 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:22:10.586+0000 7fa1e9137700 1 -- 172.21.15.186:0/1151307677 wait complete. 2023-10-18T12:22:10.979 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.8.service 2023-10-18T12:22:10.981 INFO:tasks.cephadm:Deploying osd.9 on smithi070 with /dev/vg_nvme/lv_1... 2023-10-18T12:22:10.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-18T12:22:11.015 INFO:journalctl@ceph.osd.8.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:22:11.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:11.726 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:22:12.030 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:22:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:22:11.759+0000 7fe15a0a0640 -1 osd.8 0 log_to_monitors true 2023-10-18T12:22:12.292 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:12 smithi070 ceph-mon[98639]: from='osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T12:22:12.293 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:12 smithi070 ceph-mon[98639]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T12:22:12.298 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:12 smithi186 ceph-mon[135253]: from='osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T12:22:12.298 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:12 smithi186 ceph-mon[135253]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T12:22:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:12 smithi152 ceph-mon[134767]: from='osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T12:22:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:12 smithi152 ceph-mon[134767]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T12:22:13.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:13 smithi070 ceph-mon[98639]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:13.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:13 smithi070 ceph-mon[98639]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-18T12:22:13.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:13 smithi070 ceph-mon[98639]: osdmap e58: 9 total, 8 up, 9 in 2023-10-18T12:22:13.359 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:13 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:13.359 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:13 smithi070 ceph-mon[98639]: from='osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:22:13.359 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:13 smithi070 ceph-mon[98639]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:22:13.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:13 smithi152 ceph-mon[134767]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:13 smithi152 ceph-mon[134767]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-18T12:22:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:13 smithi152 ceph-mon[134767]: osdmap e58: 9 total, 8 up, 9 in 2023-10-18T12:22:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:13 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:13 smithi152 ceph-mon[134767]: from='osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:22:13.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:13 smithi152 ceph-mon[134767]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:22:13.468 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:22:13.178+0000 7fe14f717700 -1 osd.8 0 waiting for initial osdmap 2023-10-18T12:22:13.469 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:22:13.190+0000 7fe149508700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:22:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-mon[135253]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-mon[135253]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-18T12:22:13.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-mon[135253]: osdmap e58: 9 total, 8 up, 9 in 2023-10-18T12:22:13.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:13.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-mon[135253]: from='osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:22:13.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:13 smithi186 ceph-mon[135253]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:22:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-18T12:22:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-18T12:22:13.727 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-10-18T12:22:13.727 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-10-18T12:22:13.727 INFO:teuthology.orchestra.run.smithi070.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0477084 s, 220 MB/s 2023-10-18T12:22:13.727 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-10-18T12:22:14.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi070:vg_nvme/lv_1 2023-10-18T12:22:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:22:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: osdmap e59: 9 total, 8 up, 9 in 2023-10-18T12:22:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] boot 2023-10-18T12:22:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: osdmap e60: 9 total, 9 up, 9 in 2023-10-18T12:22:14.472 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:22:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: osdmap e59: 9 total, 8 up, 9 in 2023-10-18T12:22:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:14.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:14.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] boot 2023-10-18T12:22:14.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: osdmap e60: 9 total, 9 up, 9 in 2023-10-18T12:22:14.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:22:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: osdmap e59: 9 total, 8 up, 9 in 2023-10-18T12:22:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: osd.8 [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] boot 2023-10-18T12:22:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: osdmap e60: 9 total, 9 up, 9 in 2023-10-18T12:22:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T12:22:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:22:15.459 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:15 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:22:15.459 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:15 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:22:15.460 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:15 smithi070 ceph-mon[98639]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:15.460 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:15 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi186 2023-10-18T12:22:15.460 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:15 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-18T12:22:15.460 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:15 smithi070 ceph-mon[98639]: osdmap e61: 9 total, 9 up, 9 in 2023-10-18T12:22:15.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:15 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:22:15.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:15 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:22:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:15 smithi186 ceph-mon[135253]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:15 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi186 2023-10-18T12:22:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:15 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-18T12:22:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:15 smithi186 ceph-mon[135253]: osdmap e61: 9 total, 9 up, 9 in 2023-10-18T12:22:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:15 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:22:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:15 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:22:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:15 smithi152 ceph-mon[134767]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T12:22:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:15 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi186 2023-10-18T12:22:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:15 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-18T12:22:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:15 smithi152 ceph-mon[134767]: osdmap e61: 9 total, 9 up, 9 in 2023-10-18T12:22:16.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 -- 172.21.15.70:0/2345122415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 msgr2=0x7f9ac410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:16.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 --2- 172.21.15.70:0/2345122415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac410d650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b0a0 tx=0x7f9abc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:22:16.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 -- 172.21.15.70:0/2345122415 shutdown_connections 2023-10-18T12:22:16.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 --2- 172.21.15.70:0/2345122415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac410d650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:16.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 --2- 172.21.15.70:0/2345122415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:16.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 --2- 172.21.15.70:0/2345122415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac41008a0 0x7f9ac4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:16.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 -- 172.21.15.70:0/2345122415 >> 172.21.15.70:0/2345122415 conn(0x7f9ac40779e0 msgr2=0x7f9ac4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:22:16.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 -- 172.21.15.70:0/2345122415 shutdown_connections 2023-10-18T12:22:16.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.435+0000 7f9aca691700 1 -- 172.21.15.70:0/2345122415 wait complete. 2023-10-18T12:22:16.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.436+0000 7f9aca691700 1 Processor -- start 2023-10-18T12:22:16.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.436+0000 7f9aca691700 1 -- start start 2023-10-18T12:22:16.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9aca691700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac41008a0 0x7f9ac4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:16.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9aca691700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:16.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9aca691700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:16.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9aca691700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419de10 con 0x7f9ac4101260 2023-10-18T12:22:16.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9aca691700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419df80 con 0x7f9ac41008a0 2023-10-18T12:22:16.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9aca691700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419e140 con 0x7f9ac41057c0 2023-10-18T12:22:16.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9ac37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:16.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9ac8c2e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:16.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9ac8c2e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60994/0 (socket says 172.21.15.70:60994) 2023-10-18T12:22:16.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9ac37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac4199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50262/0 (socket says 172.21.15.70:50262) 2023-10-18T12:22:16.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9ac8c2e700 1 -- 172.21.15.70:0/2654936639 learned_addr learned my addr 172.21.15.70:0/2654936639 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:22:16.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.437+0000 7f9ac3fff700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac41008a0 0x7f9ac4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:16.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac8c2e700 1 -- 172.21.15.70:0/2654936639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac41008a0 msgr2=0x7f9ac4198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:16.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac8c2e700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac41008a0 0x7f9ac4198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:16.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac8c2e700 1 -- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 msgr2=0x7f9ac4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac8c2e700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac8c2e700 1 -- 172.21.15.70:0/2654936639 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc009d00 con 0x7f9ac41057c0 2023-10-18T12:22:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac8c2e700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac419d800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b070 tx=0x7f9abc007860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:22:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00e040 con 0x7f9ac41057c0 2023-10-18T12:22:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac419e3c0 con 0x7f9ac41057c0 2023-10-18T12:22:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9abc01d470 con 0x7f9ac41057c0 2023-10-18T12:22:16.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0052a0 con 0x7f9ac41057c0 2023-10-18T12:22:16.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.438+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac41a5c80 con 0x7f9ac41057c0 2023-10-18T12:22:16.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.439+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac4101f30 con 0x7f9ac41057c0 2023-10-18T12:22:16.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.441+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9abc01d5e0 con 0x7f9ac41057c0 2023-10-18T12:22:16.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.442+0000 7f9ac17fa700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9aac0a14d0 0x7f9aac0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:16.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.442+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9abc0cd7e0 con 0x7f9ac41057c0 2023-10-18T12:22:16.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.443+0000 7f9ac3fff700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9aac0a14d0 0x7f9aac0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:16.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.443+0000 7f9ac3fff700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9aac0a14d0 0x7f9aac0a3990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4101930 tx=0x7f9ab4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:22:16.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.445+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9abc0976f0 con 0x7f9ac41057c0 2023-10-18T12:22:16.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:16.656+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac419a290 con 0x7f9aac0a14d0 2023-10-18T12:22:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:17 smithi070 ceph-mon[98639]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:17 smithi070 ceph-mon[98639]: osdmap e62: 9 total, 9 up, 9 in 2023-10-18T12:22:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:22:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:22:17.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:17 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:17.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:17 smithi152 ceph-mon[134767]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:17.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:17 smithi152 ceph-mon[134767]: osdmap e62: 9 total, 9 up, 9 in 2023-10-18T12:22:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:22:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:22:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:17 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:17 smithi186 ceph-mon[135253]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:17 smithi186 ceph-mon[135253]: osdmap e62: 9 total, 9 up, 9 in 2023-10-18T12:22:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:22:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:22:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:17 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:18.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:18 smithi070 ceph-mon[98639]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:22:18.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:18 smithi070 ceph-mon[98639]: osdmap e63: 9 total, 9 up, 9 in 2023-10-18T12:22:18.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:18 smithi152 ceph-mon[134767]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:22:18.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:18 smithi152 ceph-mon[134767]: osdmap e63: 9 total, 9 up, 9 in 2023-10-18T12:22:18.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:18 smithi186 ceph-mon[135253]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:22:18.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:18 smithi186 ceph-mon[135253]: osdmap e63: 9 total, 9 up, 9 in 2023-10-18T12:22:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:19 smithi070 ceph-mon[98639]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:19 smithi070 ceph-mon[98639]: osdmap e64: 9 total, 9 up, 9 in 2023-10-18T12:22:19.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:19 smithi152 ceph-mon[134767]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:19.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:19 smithi152 ceph-mon[134767]: osdmap e64: 9 total, 9 up, 9 in 2023-10-18T12:22:19.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:19 smithi186 ceph-mon[135253]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:19.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:19 smithi186 ceph-mon[135253]: osdmap e64: 9 total, 9 up, 9 in 2023-10-18T12:22:20.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:20 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2011646135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6976bc71-bb5c-49ac-8b46-9b64c66fd4ed"}]: dispatch 2023-10-18T12:22:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:20 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2011646135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6976bc71-bb5c-49ac-8b46-9b64c66fd4ed"}]': finished 2023-10-18T12:22:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:20 smithi070 ceph-mon[98639]: osdmap e65: 10 total, 9 up, 10 in 2023-10-18T12:22:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:20 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:20.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:20 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2011646135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6976bc71-bb5c-49ac-8b46-9b64c66fd4ed"}]: dispatch 2023-10-18T12:22:20.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:20 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2011646135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6976bc71-bb5c-49ac-8b46-9b64c66fd4ed"}]': finished 2023-10-18T12:22:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:20 smithi152 ceph-mon[134767]: osdmap e65: 10 total, 9 up, 10 in 2023-10-18T12:22:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:20 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:20.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:20 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2011646135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6976bc71-bb5c-49ac-8b46-9b64c66fd4ed"}]: dispatch 2023-10-18T12:22:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:20 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2011646135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6976bc71-bb5c-49ac-8b46-9b64c66fd4ed"}]': finished 2023-10-18T12:22:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:20 smithi186 ceph-mon[135253]: osdmap e65: 10 total, 9 up, 10 in 2023-10-18T12:22:20.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:20 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:21.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:21 smithi070 ceph-mon[98639]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:21.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:21 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/167076661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:22:21.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:21 smithi152 ceph-mon[134767]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:21 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/167076661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:22:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:21 smithi186 ceph-mon[135253]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:21 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/167076661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:22:23.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:23 smithi070 ceph-mon[98639]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:22:23.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:23 smithi152 ceph-mon[134767]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:22:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:23 smithi186 ceph-mon[135253]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-10-18T12:22:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:25 smithi070 ceph-mon[98639]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-18T12:22:25.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:25 smithi152 ceph-mon[134767]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-18T12:22:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:25 smithi186 ceph-mon[135253]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-18T12:22:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:27 smithi070 ceph-mon[98639]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:22:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:27 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-18T12:22:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:27 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:27.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:27 smithi152 ceph-mon[134767]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:22:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:27 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-18T12:22:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:27 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:27 smithi186 ceph-mon[135253]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:22:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:27 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-18T12:22:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:27 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:28.222 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:28 smithi070 ceph-mon[98639]: Deploying daemon osd.9 on smithi070 2023-10-18T12:22:28.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:28 smithi152 ceph-mon[134767]: Deploying daemon osd.9 on smithi070 2023-10-18T12:22:28.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:28 smithi186 ceph-mon[135253]: Deploying daemon osd.9 on smithi070 2023-10-18T12:22:29.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:29 smithi070 ceph-mon[98639]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:22:29.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:29 smithi152 ceph-mon[134767]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:22:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:29 smithi186 ceph-mon[135253]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:22:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:31 smithi152 ceph-mon[134767]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T12:22:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:31 smithi186 ceph-mon[135253]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T12:22:31.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:31 smithi070 ceph-mon[98639]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T12:22:33.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:33 smithi070 ceph-mon[98639]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-18T12:22:33.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:33 smithi152 ceph-mon[134767]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-18T12:22:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:33 smithi186 ceph-mon[135253]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-18T12:22:34.550 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:34.550 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:34.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:34.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:34 smithi070 ceph-mon[98639]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:34.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:34 smithi152 ceph-mon[134767]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:34.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:34 smithi186 ceph-mon[135253]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:36.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.612+0000 7f9ac17fa700 1 -- 172.21.15.70:0/2654936639 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9ac419a290 con 0x7f9aac0a14d0 2023-10-18T12:22:36.616 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 9 on host 'smithi070' 2023-10-18T12:22:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9aac0a14d0 msgr2=0x7f9aac0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9aac0a14d0 0x7f9aac0a3990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4101930 tx=0x7f9ab4009260 comp rx=0 tx=0).stop 2023-10-18T12:22:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 msgr2=0x7f9ac419d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac419d800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b070 tx=0x7f9abc007860 comp rx=0 tx=0).stop 2023-10-18T12:22:36.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 shutdown_connections 2023-10-18T12:22:36.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac41057c0 0x7f9ac419d800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:36.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9aac0a14d0 0x7f9aac0a3990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:36.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac41008a0 0x7f9ac4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:36.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 --2- 172.21.15.70:0/2654936639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ac4101260 0x7f9ac4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 >> 172.21.15.70:0/2654936639 conn(0x7f9ac40779e0 msgr2=0x7f9ac410ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:22:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 shutdown_connections 2023-10-18T12:22:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:22:36.616+0000 7f9aca691700 1 -- 172.21.15.70:0/2654936639 wait complete. 2023-10-18T12:22:36.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:36.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:36.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:36.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:36.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:36.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:37.063 DEBUG:teuthology.orchestra.run.smithi070:osd.9> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.9.service 2023-10-18T12:22:37.068 INFO:tasks.cephadm:Deploying osd.10 on smithi152 with /dev/vg_nvme/lv_1... 2023-10-18T12:22:37.068 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-18T12:22:37.140 INFO:journalctl@ceph.osd.9.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:22:37.355 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:22:37 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:22:37.243+0000 7fe5d55f8640 -1 osd.9 0 log_to_monitors true 2023-10-18T12:22:37.644 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:22:37.889 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:37 smithi152 ceph-mon[134767]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T12:22:37.954 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:37 smithi070 ceph-mon[98639]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T12:22:37.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:37 smithi186 ceph-mon[135253]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T12:22:38.738 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:38 smithi152 ceph-mon[134767]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:38.739 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:38 smithi152 ceph-mon[134767]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-18T12:22:38.739 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:38 smithi152 ceph-mon[134767]: osdmap e66: 10 total, 9 up, 10 in 2023-10-18T12:22:38.739 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:38 smithi152 ceph-mon[134767]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:22:38.739 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:38.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:38 smithi186 ceph-mon[135253]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:38.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:38 smithi186 ceph-mon[135253]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-18T12:22:38.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:38 smithi186 ceph-mon[135253]: osdmap e66: 10 total, 9 up, 10 in 2023-10-18T12:22:38.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:38 smithi186 ceph-mon[135253]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:22:38.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:39.105 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:22:38.650+0000 7fe5cac6f700 -1 osd.9 0 waiting for initial osdmap 2023-10-18T12:22:39.105 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:22:38.664+0000 7fe5c425f700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:22:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-mon[98639]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-mon[98639]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-18T12:22:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-mon[98639]: osdmap e66: 10 total, 9 up, 10 in 2023-10-18T12:22:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-mon[98639]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-10-18T12:22:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:39.241 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-18T12:22:39.241 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-18T12:22:39.242 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2023-10-18T12:22:39.242 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2023-10-18T12:22:39.242 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127579 s, 822 MB/s 2023-10-18T12:22:39.242 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2023-10-18T12:22:39.600 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi152:vg_nvme/lv_1 2023-10-18T12:22:39.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:39 smithi152 ceph-mon[134767]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:22:39.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:39 smithi152 ceph-mon[134767]: osdmap e67: 10 total, 9 up, 10 in 2023-10-18T12:22:39.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:39.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:39.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:39 smithi186 ceph-mon[135253]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:22:39.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:39 smithi186 ceph-mon[135253]: osdmap e67: 10 total, 9 up, 10 in 2023-10-18T12:22:39.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:39.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:39 smithi070 ceph-mon[98639]: from='osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-10-18T12:22:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:39 smithi070 ceph-mon[98639]: osdmap e67: 10 total, 9 up, 10 in 2023-10-18T12:22:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.159 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:22:40.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:22:40.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:22:40.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:40.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] boot 2023-10-18T12:22:40.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: osdmap e68: 10 total, 10 up, 10 in 2023-10-18T12:22:40.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:40.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:40.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:22:40.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:22:40.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:40.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] boot 2023-10-18T12:22:40.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: osdmap e68: 10 total, 10 up, 10 in 2023-10-18T12:22:40.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.924 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:40.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:40.925 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:22:40.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:22:40.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T12:22:40.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: osd.9 [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] boot 2023-10-18T12:22:40.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: osdmap e68: 10 total, 10 up, 10 in 2023-10-18T12:22:40.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T12:22:40.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:22:40.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:40.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:40.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:22:40.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 -- 172.21.15.152:0/1769423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ffb30 msgr2=0x7fb17810d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:41.281 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 --2- 172.21.15.152:0/1769423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ffb30 0x7fb17810d5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb17000b0a0 tx=0x7fb17000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:22:41.282 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 -- 172.21.15.152:0/1769423745 shutdown_connections 2023-10-18T12:22:41.282 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 --2- 172.21.15.152:0/1769423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ffb30 0x7fb17810d5c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:41.282 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 --2- 172.21.15.152:0/1769423745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781058d0 0x7fb178105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:41.282 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 --2- 172.21.15.152:0/1769423745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ff190 0x7fb1780ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:41.282 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 -- 172.21.15.152:0/1769423745 >> 172.21.15.152:0/1769423745 conn(0x7fb178074a10 msgr2=0x7fb178074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:22:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 -- 172.21.15.152:0/1769423745 shutdown_connections 2023-10-18T12:22:41.283 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.278+0000 7fb17cd01700 1 -- 172.21.15.152:0/1769423745 wait complete. 2023-10-18T12:22:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 Processor -- start 2023-10-18T12:22:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 -- start start 2023-10-18T12:22:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 0x7fb178196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ffb30 0x7fb178197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:41.285 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781058d0 0x7fb17819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:41.286 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17819bca0 con 0x7fb1781058d0 2023-10-18T12:22:41.286 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17819be10 con 0x7fb1780ff190 2023-10-18T12:22:41.286 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17cd01700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17819bfd0 con 0x7fb1780ffb30 2023-10-18T12:22:41.286 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 0x7fb178196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:41.287 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 0x7fb178196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57976/0 (socket says 172.21.15.152:57976) 2023-10-18T12:22:41.287 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.279+0000 7fb17659c700 1 -- 172.21.15.152:0/3593709411 learned_addr learned my addr 172.21.15.152:0/3593709411 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-10-18T12:22:41.287 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb175d9b700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ffb30 0x7fb178197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:41.287 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb176d9d700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781058d0 0x7fb17819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:41.287 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17659c700 1 -- 172.21.15.152:0/3593709411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ffb30 msgr2=0x7fb178197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:41.288 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17659c700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ffb30 0x7fb178197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:41.288 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17659c700 1 -- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781058d0 msgr2=0x7fb17819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:41.288 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17659c700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781058d0 0x7fb17819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:41.288 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17659c700 1 -- 172.21.15.152:0/3593709411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170009d00 con 0x7fb1780ff190 2023-10-18T12:22:41.288 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb175d9b700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ffb30 0x7fb178197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:22:41.289 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17659c700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 0x7fb178196df0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00b500 tx=0x7fb16c00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:22:41.289 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c011830 con 0x7fb1780ff190 2023-10-18T12:22:41.289 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16c011e50 con 0x7fb1780ff190 2023-10-18T12:22:41.289 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17819c2b0 con 0x7fb1780ff190 2023-10-18T12:22:41.289 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00f2e0 con 0x7fb1780ff190 2023-10-18T12:22:41.290 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.280+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1781a3b10 con 0x7fb1780ff190 2023-10-18T12:22:41.290 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.281+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb178108d30 con 0x7fb1780ff190 2023-10-18T12:22:41.290 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.282+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb16c010580 con 0x7fb1780ff190 2023-10-18T12:22:41.291 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.282+0000 7fb1677fe700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a14d0 0x7fb1600a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:22:41.291 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.283+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6476+0+0 (secure 0 0 0) 0x7fb16c0cc100 con 0x7fb1780ff190 2023-10-18T12:22:41.291 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.283+0000 7fb175d9b700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a14d0 0x7fb1600a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:22:41.292 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.283+0000 7fb175d9b700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a14d0 0x7fb1600a3990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb168005950 tx=0x7fb1680058e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:22:41.292 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.287+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb16c095e90 con 0x7fb1780ff190 2023-10-18T12:22:41.488 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:41.485+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb17804e560 con 0x7fb1600a14d0 2023-10-18T12:22:42.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:42 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi070 2023-10-18T12:22:42.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:42 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi070 to 4317M 2023-10-18T12:22:42.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:42 smithi070 ceph-mon[98639]: osdmap e69: 10 total, 10 up, 10 in 2023-10-18T12:22:42.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:22:42.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:22:42.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:42.363 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:42 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi070 2023-10-18T12:22:42.363 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:42 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi070 to 4317M 2023-10-18T12:22:42.363 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:42 smithi152 ceph-mon[134767]: osdmap e69: 10 total, 10 up, 10 in 2023-10-18T12:22:42.364 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:22:42.364 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:22:42.364 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:42.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:42 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi070 2023-10-18T12:22:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:42 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi070 to 4317M 2023-10-18T12:22:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:42 smithi186 ceph-mon[135253]: osdmap e69: 10 total, 10 up, 10 in 2023-10-18T12:22:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:22:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:22:42.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:43.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:43 smithi070 ceph-mon[98639]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:22:43.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:43 smithi070 ceph-mon[98639]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:43.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:43 smithi152 ceph-mon[134767]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:22:43.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:43 smithi152 ceph-mon[134767]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:43.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:43 smithi186 ceph-mon[135253]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:22:43.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:43 smithi186 ceph-mon[135253]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:44.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:44 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/3703463862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]: dispatch 2023-10-18T12:22:44.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:44 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]: dispatch 2023-10-18T12:22:44.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:44 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]': finished 2023-10-18T12:22:44.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:44 smithi070 ceph-mon[98639]: osdmap e70: 11 total, 10 up, 11 in 2023-10-18T12:22:44.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:22:44.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:44 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/3703463862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]: dispatch 2023-10-18T12:22:44.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:44 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]: dispatch 2023-10-18T12:22:44.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:44 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]': finished 2023-10-18T12:22:44.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:44 smithi152 ceph-mon[134767]: osdmap e70: 11 total, 10 up, 11 in 2023-10-18T12:22:44.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:22:44.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:44 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/3703463862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]: dispatch 2023-10-18T12:22:44.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:44 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]: dispatch 2023-10-18T12:22:44.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:44 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "310462a3-9997-49ff-ab18-0a1fbc14518d"}]': finished 2023-10-18T12:22:44.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:44 smithi186 ceph-mon[135253]: osdmap e70: 11 total, 10 up, 11 in 2023-10-18T12:22:44.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:22:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:45 smithi070 ceph-mon[98639]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:45.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:45 smithi070 ceph-mon[98639]: from='client.? 172.21.15.152:0/2261228928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:22:45.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:45 smithi152 ceph-mon[134767]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:45.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:45 smithi152 ceph-mon[134767]: from='client.? 172.21.15.152:0/2261228928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:22:45.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:45 smithi186 ceph-mon[135253]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:45.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:45 smithi186 ceph-mon[135253]: from='client.? 172.21.15.152:0/2261228928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:22:47.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:47 smithi152 ceph-mon[134767]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:47.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:47 smithi186 ceph-mon[135253]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:47.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:47 smithi070 ceph-mon[98639]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:49.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:49 smithi152 ceph-mon[134767]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:49.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:49 smithi186 ceph-mon[135253]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:49 smithi070 ceph-mon[98639]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:51.356 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:51 smithi152 ceph-mon[134767]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:51.357 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-18T12:22:51.357 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:51.357 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:51 smithi152 ceph-mon[134767]: Deploying daemon osd.10 on smithi152 2023-10-18T12:22:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:51 smithi186 ceph-mon[135253]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-18T12:22:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:51 smithi186 ceph-mon[135253]: Deploying daemon osd.10 on smithi152 2023-10-18T12:22:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:51 smithi070 ceph-mon[98639]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-18T12:22:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:22:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:51 smithi070 ceph-mon[98639]: Deploying daemon osd.10 on smithi152 2023-10-18T12:22:53.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:53 smithi152 ceph-mon[134767]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:53 smithi186 ceph-mon[135253]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:53.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:53 smithi070 ceph-mon[98639]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:55.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:55 smithi152 ceph-mon[134767]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:55.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:55.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:55.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:55.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:55 smithi186 ceph-mon[135253]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:55.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:55.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:55.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:55.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:55 smithi070 ceph-mon[98639]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:55.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:55.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:55.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:22:57.345 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:57 smithi152 ceph-mon[134767]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:57.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.398+0000 7fb1677fe700 1 -- 172.21.15.152:0/3593709411 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb17804e560 con 0x7fb1600a14d0 2023-10-18T12:22:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 10 on host 'smithi152' 2023-10-18T12:22:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a14d0 msgr2=0x7fb1600a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a14d0 0x7fb1600a3990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb168005950 tx=0x7fb1680058e0 comp rx=0 tx=0).stop 2023-10-18T12:22:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 msgr2=0x7fb178196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:22:57.403 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 0x7fb178196df0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00b500 tx=0x7fb16c00b8c0 comp rx=0 tx=0).stop 2023-10-18T12:22:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 shutdown_connections 2023-10-18T12:22:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781058d0 0x7fb17819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a14d0 0x7fb1600a3990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1780ff190 0x7fb178196df0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:57.404 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 --2- 172.21.15.152:0/3593709411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780ffb30 0x7fb178197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:22:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 >> 172.21.15.152:0/3593709411 conn(0x7fb178074a10 msgr2=0x7fb1780fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:22:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 shutdown_connections 2023-10-18T12:22:57.405 INFO:teuthology.orchestra.run.smithi152.stderr:2023-10-18T12:22:57.401+0000 7fb17cd01700 1 -- 172.21.15.152:0/3593709411 wait complete. 2023-10-18T12:22:57.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:57 smithi186 ceph-mon[135253]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:57 smithi070 ceph-mon[98639]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:57.849 DEBUG:teuthology.orchestra.run.smithi152:osd.10> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.10.service 2023-10-18T12:22:57.851 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2023-10-18T12:22:57.852 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-18T12:22:57.881 INFO:journalctl@ceph.osd.10.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:22:58.421 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:22:58.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:58 smithi186 ceph-mon[135253]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:58.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:58 smithi152 ceph-mon[134767]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:58.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:22:58.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:58 smithi070 ceph-mon[98639]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:22:59.069 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:22:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:22:58.678+0000 7fc615f5b640 -1 osd.10 0 log_to_monitors true 2023-10-18T12:22:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:59 smithi152 ceph-mon[134767]: from='osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T12:22:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:22:59 smithi152 ceph-mon[134767]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T12:22:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:59 smithi186 ceph-mon[135253]: from='osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T12:22:59.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:22:59 smithi186 ceph-mon[135253]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T12:22:59.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:59 smithi070 ceph-mon[98639]: from='osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T12:22:59.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:22:59 smithi070 ceph-mon[98639]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T12:22:59.916 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-18T12:22:59.916 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-18T12:22:59.917 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-18T12:22:59.917 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-18T12:22:59.917 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110819 s, 946 MB/s 2023-10-18T12:22:59.917 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 2023-10-18T12:22:59.917 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-18T12:23:00.265 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2023-10-18T12:23:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-mon[134767]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-18T12:23:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-mon[134767]: osdmap e71: 11 total, 10 up, 11 in 2023-10-18T12:23:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-mon[134767]: from='osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:23:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-mon[134767]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:23:00.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-mon[134767]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:23:00.673 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:23:00.399+0000 7fc60b5d2700 -1 osd.10 0 waiting for initial osdmap 2023-10-18T12:23:00.674 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:23:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:23:00.411+0000 7fc6043c1700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:23:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:00 smithi186 ceph-mon[135253]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-18T12:23:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:00 smithi186 ceph-mon[135253]: osdmap e71: 11 total, 10 up, 11 in 2023-10-18T12:23:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:00 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:00 smithi186 ceph-mon[135253]: from='osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:23:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:00 smithi186 ceph-mon[135253]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:23:00.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:00 smithi186 ceph-mon[135253]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:23:00.828 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:23:00.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:00 smithi070 ceph-mon[98639]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-18T12:23:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:00 smithi070 ceph-mon[98639]: osdmap e71: 11 total, 10 up, 11 in 2023-10-18T12:23:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:00 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:00 smithi070 ceph-mon[98639]: from='osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:23:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:00 smithi070 ceph-mon[98639]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2023-10-18T12:23:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:00 smithi070 ceph-mon[98639]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 894 GiB / 894 GiB avail 2023-10-18T12:23:01.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:23:01.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: osdmap e72: 11 total, 10 up, 11 in 2023-10-18T12:23:01.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi152 2023-10-18T12:23:01.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi152 to 4292M 2023-10-18T12:23:01.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:01.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:01.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:01.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:01.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:01.676 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.676 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] boot 2023-10-18T12:23:01.676 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: osdmap e73: 11 total, 11 up, 11 in 2023-10-18T12:23:01.676 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:23:01.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: osdmap e72: 11 total, 10 up, 11 in 2023-10-18T12:23:01.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi152 2023-10-18T12:23:01.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.686 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.686 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.686 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.686 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.686 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi152 to 4292M 2023-10-18T12:23:01.687 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.687 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:01.687 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:01.687 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.687 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:01.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:01.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:01.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] boot 2023-10-18T12:23:01.688 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: osdmap e73: 11 total, 11 up, 11 in 2023-10-18T12:23:01.689 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2023-10-18T12:23:01.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: osdmap e72: 11 total, 10 up, 11 in 2023-10-18T12:23:01.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:01.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi152 2023-10-18T12:23:01.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:01.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi152 to 4292M 2023-10-18T12:23:01.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:01.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:01.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:01.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:01.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:01.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:01.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: osd.10 [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] boot 2023-10-18T12:23:01.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: osdmap e73: 11 total, 11 up, 11 in 2023-10-18T12:23:01.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T12:23:02.133 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 -- 172.21.15.186:0/3907325842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c068b30 msgr2=0x7f533c10d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:02.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 --2- 172.21.15.186:0/3907325842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c068b30 0x7f533c10d670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f533400b0a0 tx=0x7f533400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:23:02.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 -- 172.21.15.186:0/3907325842 shutdown_connections 2023-10-18T12:23:02.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 --2- 172.21.15.186:0/3907325842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c068b30 0x7f533c10d670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:02.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 --2- 172.21.15.186:0/3907325842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1067a0 0x7f533c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:02.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 --2- 172.21.15.186:0/3907325842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c068190 0x7f533c0685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:02.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 -- 172.21.15.186:0/3907325842 >> 172.21.15.186:0/3907325842 conn(0x7f533c075790 msgr2=0x7f533c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:02.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 -- 172.21.15.186:0/3907325842 shutdown_connections 2023-10-18T12:23:02.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.130+0000 7f5341ae4700 1 -- 172.21.15.186:0/3907325842 wait complete. 2023-10-18T12:23:02.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 Processor -- start 2023-10-18T12:23:02.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 -- start start 2023-10-18T12:23:02.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c068190 0x7f533c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:02.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c068b30 0x7f533c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:02.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 0x7f533c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:02.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c19ddf0 con 0x7f533c068190 2023-10-18T12:23:02.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c19df60 con 0x7f533c068b30 2023-10-18T12:23:02.137 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f5341ae4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c19e120 con 0x7f533c1067a0 2023-10-18T12:23:02.137 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f533affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c068b30 0x7f533c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:02.137 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f533bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 0x7f533c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:02.137 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f533affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c068b30 0x7f533c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.186:40162/0 (socket says 172.21.15.186:40162) 2023-10-18T12:23:02.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f533b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c068190 0x7f533c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:02.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f533bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 0x7f533c19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:50770/0 (socket says 172.21.15.186:50770) 2023-10-18T12:23:02.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.131+0000 7f533affd700 1 -- 172.21.15.186:0/1595512083 learned_addr learned my addr 172.21.15.186:0/1595512083 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-18T12:23:02.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f533bfff700 1 -- 172.21.15.186:0/1595512083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c068b30 msgr2=0x7f533c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:02.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f533bfff700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c068b30 0x7f533c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:02.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f533bfff700 1 -- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c068190 msgr2=0x7f533c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:02.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f533bfff700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c068190 0x7f533c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:02.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f533bfff700 1 -- 172.21.15.186:0/1595512083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009d00 con 0x7f533c1067a0 2023-10-18T12:23:02.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f533bfff700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 0x7f533c19d7e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f533400b0a0 tx=0x7f5334007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:02.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400e040 con 0x7f533c1067a0 2023-10-18T12:23:02.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c19e3a0 con 0x7f533c1067a0 2023-10-18T12:23:02.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5334004c60 con 0x7f533c1067a0 2023-10-18T12:23:02.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340052a0 con 0x7f533c1067a0 2023-10-18T12:23:02.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.132+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1a5c60 con 0x7f533c1067a0 2023-10-18T12:23:02.142 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.133+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c101ee0 con 0x7f533c1067a0 2023-10-18T12:23:02.142 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.134+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f533401b450 con 0x7f533c1067a0 2023-10-18T12:23:02.142 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.134+0000 7f5338ff9700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53240a14d0 0x7f53240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:02.142 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.135+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6942+0+0 (secure 0 0 0) 0x7f53340ce6c0 con 0x7f533c1067a0 2023-10-18T12:23:02.142 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.135+0000 7f533b7fe700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53240a14d0 0x7f53240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:02.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.135+0000 7f533b7fe700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53240a14d0 0x7f53240a3990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5330009a10 tx=0x7f5330009190 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:02.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.138+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5334098310 con 0x7f533c1067a0 2023-10-18T12:23:02.337 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:02.334+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f533c19a270 con 0x7f53240a14d0 2023-10-18T12:23:02.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:23:02.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:23:02.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:02.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:23:02.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:23:02.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:02.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:02 smithi152 ceph-mon[134767]: osdmap e74: 11 total, 11 up, 11 in 2023-10-18T12:23:02.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:23:02.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:23:02.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:02.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:23:02.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:23:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:02.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:02 smithi186 ceph-mon[135253]: osdmap e74: 11 total, 11 up, 11 in 2023-10-18T12:23:02.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:23:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:23:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:02.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T12:23:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T12:23:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:02.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:02 smithi070 ceph-mon[98639]: osdmap e74: 11 total, 11 up, 11 in 2023-10-18T12:23:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:03 smithi152 ceph-mon[134767]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:23:03.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:03 smithi152 ceph-mon[134767]: osdmap e75: 11 total, 11 up, 11 in 2023-10-18T12:23:03.682 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:03 smithi186 ceph-mon[135253]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:23:03.683 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:03 smithi186 ceph-mon[135253]: osdmap e75: 11 total, 11 up, 11 in 2023-10-18T12:23:03.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:03 smithi070 ceph-mon[98639]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:23:03.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:03 smithi070 ceph-mon[98639]: osdmap e75: 11 total, 11 up, 11 in 2023-10-18T12:23:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: osdmap e76: 11 total, 11 up, 11 in 2023-10-18T12:23:05.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/4084959232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]: dispatch 2023-10-18T12:23:05.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]: dispatch 2023-10-18T12:23:05.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]': finished 2023-10-18T12:23:05.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: osdmap e77: 12 total, 11 up, 12 in 2023-10-18T12:23:05.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:05 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: osdmap e76: 11 total, 11 up, 11 in 2023-10-18T12:23:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/4084959232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]: dispatch 2023-10-18T12:23:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]: dispatch 2023-10-18T12:23:05.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]': finished 2023-10-18T12:23:05.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: osdmap e77: 12 total, 11 up, 12 in 2023-10-18T12:23:05.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:05 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:05.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:05.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: osdmap e76: 11 total, 11 up, 11 in 2023-10-18T12:23:05.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/4084959232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]: dispatch 2023-10-18T12:23:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]: dispatch 2023-10-18T12:23:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3695b252-80cc-489f-a5d5-48f7ce64a1f2"}]': finished 2023-10-18T12:23:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: osdmap e77: 12 total, 11 up, 12 in 2023-10-18T12:23:05.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:05 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:06 smithi070 ceph-mon[98639]: from='client.? 172.21.15.186:0/3161271442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:23:06.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:06 smithi152 ceph-mon[134767]: from='client.? 172.21.15.186:0/3161271442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:23:06.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:06 smithi186 ceph-mon[135253]: from='client.? 172.21.15.186:0/3161271442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T12:23:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:07 smithi070 ceph-mon[98639]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:23:07.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:07 smithi152 ceph-mon[134767]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:23:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:07 smithi186 ceph-mon[135253]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T12:23:09.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:09 smithi152 ceph-mon[134767]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-10-18T12:23:09.456 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:09 smithi186 ceph-mon[135253]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-10-18T12:23:09.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:09 smithi070 ceph-mon[98639]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-10-18T12:23:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:11 smithi152 ceph-mon[134767]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-10-18T12:23:11.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:11 smithi186 ceph-mon[135253]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-10-18T12:23:11.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:11 smithi070 ceph-mon[98639]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-10-18T12:23:12.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-18T12:23:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:12 smithi152 ceph-mon[134767]: Deploying daemon osd.11 on smithi186 2023-10-18T12:23:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-18T12:23:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:12 smithi186 ceph-mon[135253]: Deploying daemon osd.11 on smithi186 2023-10-18T12:23:12.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-18T12:23:12.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:12.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:12 smithi070 ceph-mon[98639]: Deploying daemon osd.11 on smithi186 2023-10-18T12:23:13.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:13 smithi186 ceph-mon[135253]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:23:13.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:13 smithi152 ceph-mon[134767]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:23:13.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:13 smithi070 ceph-mon[98639]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-18T12:23:15.378 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:15 smithi186 ceph-mon[135253]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:23:15.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:15 smithi152 ceph-mon[134767]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:23:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:15 smithi070 ceph-mon[98639]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-18T12:23:16.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:16 smithi186 ceph-mon[135253]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-10-18T12:23:16.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:16.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:16.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:16.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:16 smithi070 ceph-mon[98639]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-10-18T12:23:16.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:16.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:16.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:16.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:16 smithi152 ceph-mon[134767]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-10-18T12:23:18.401 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2023-10-18T12:23:18.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.397+0000 7f5338ff9700 1 -- 172.21.15.186:0/1595512083 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f533c19a270 con 0x7f53240a14d0 2023-10-18T12:23:18.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53240a14d0 msgr2=0x7f53240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:18.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53240a14d0 0x7f53240a3990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5330009a10 tx=0x7f5330009190 comp rx=0 tx=0).stop 2023-10-18T12:23:18.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 msgr2=0x7f533c19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:18.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 0x7f533c19d7e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f533400b0a0 tx=0x7f5334007870 comp rx=0 tx=0).stop 2023-10-18T12:23:18.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 shutdown_connections 2023-10-18T12:23:18.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c1067a0 0x7f533c19d7e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:18.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c068190 0x7f533c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:18.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53240a14d0 0x7f53240a3990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:18.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 --2- 172.21.15.186:0/1595512083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c068b30 0x7f533c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:18.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.400+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 >> 172.21.15.186:0/1595512083 conn(0x7f533c075790 msgr2=0x7f533c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:18.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.401+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 shutdown_connections 2023-10-18T12:23:18.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:18.401+0000 7f5341ae4700 1 -- 172.21.15.186:0/1595512083 wait complete. 2023-10-18T12:23:18.823 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.11.service 2023-10-18T12:23:18.825 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-18T12:23:18.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-18T12:23:18.857 INFO:journalctl@ceph.osd.11.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:23:19.173 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:19.173 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.173 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.173 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:19.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:19.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.174 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:19.175 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:19.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:19.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:19.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:19 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:19.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:19.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:19.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:19.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:19 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:19.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:23:19 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:23:19.168+0000 7f101d4af640 -1 osd.11 0 log_to_monitors true 2023-10-18T12:23:19.569 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:23:20.110 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:20 smithi070 ceph-mon[98639]: from='osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T12:23:20.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:20 smithi070 ceph-mon[98639]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T12:23:20.131 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:20 smithi186 ceph-mon[135253]: from='osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T12:23:20.131 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:20 smithi186 ceph-mon[135253]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T12:23:20.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:20 smithi152 ceph-mon[134767]: from='osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T12:23:20.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:20 smithi152 ceph-mon[134767]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T12:23:20.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 -- 172.21.15.70:0/1016216779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 msgr2=0x7ff580105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:20.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 --2- 172.21.15.70:0/1016216779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580105ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009990 tx=0x7ff57c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:20.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 -- 172.21.15.70:0/1016216779 shutdown_connections 2023-10-18T12:23:20.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 --2- 172.21.15.70:0/1016216779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 0x7ff58010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:20.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 --2- 172.21.15.70:0/1016216779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580105ba0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:20.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 --2- 172.21.15.70:0/1016216779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5801008a0 0x7ff580100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:20.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 -- 172.21.15.70:0/1016216779 >> 172.21.15.70:0/1016216779 conn(0x7ff5800779e0 msgr2=0x7ff580077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:20.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 -- 172.21.15.70:0/1016216779 shutdown_connections 2023-10-18T12:23:20.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.967+0000 7ff587817700 1 -- 172.21.15.70:0/1016216779 wait complete. 2023-10-18T12:23:20.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.968+0000 7ff587817700 1 Processor -- start 2023-10-18T12:23:20.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 -- start start 2023-10-18T12:23:20.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5801008a0 0x7ff580072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:20.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 0x7ff580073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:20.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:20.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5801a5130 con 0x7ff580101260 2023-10-18T12:23:20.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5801a52a0 con 0x7ff5801008a0 2023-10-18T12:23:20.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff587817700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5801a5460 con 0x7ff5801057c0 2023-10-18T12:23:20.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff584db2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 0x7ff580073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:20.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff585db4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:20.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff5855b3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5801008a0 0x7ff580072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:20.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff585db4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:32802/0 (socket says 172.21.15.70:32802) 2023-10-18T12:23:20.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff585db4700 1 -- 172.21.15.70:0/392263548 learned_addr learned my addr 172.21.15.70:0/392263548 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:20.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff585db4700 1 -- 172.21.15.70:0/392263548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5801008a0 msgr2=0x7ff580072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:20.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff585db4700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5801008a0 0x7ff580072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:20.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.969+0000 7ff585db4700 1 -- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 msgr2=0x7ff580073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:20.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff585db4700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 0x7ff580073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:20.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff585db4700 1 -- 172.21.15.70:0/392263548 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c0095f0 con 0x7ff5801057c0 2023-10-18T12:23:20.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff584db2700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 0x7ff580073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:23:20.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff585db4700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580073570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff57800efc0 tx=0x7ff57800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:20.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578018070 con 0x7ff5801057c0 2023-10-18T12:23:20.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5801a56e0 con 0x7ff5801057c0 2023-10-18T12:23:20.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff57800f040 con 0x7ff5801057c0 2023-10-18T12:23:20.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800cc10 con 0x7ff5801057c0 2023-10-18T12:23:20.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.970+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5801a5b90 con 0x7ff5801057c0 2023-10-18T12:23:20.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.971+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff580108d80 con 0x7ff5801057c0 2023-10-18T12:23:20.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.973+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff578003a00 con 0x7ff5801057c0 2023-10-18T12:23:20.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.974+0000 7ff5767fc700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff56c0a14d0 0x7ff56c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:20.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.975+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7046+0+0 (secure 0 0 0) 0x7ff5780cc3c0 con 0x7ff5801057c0 2023-10-18T12:23:20.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.975+0000 7ff5855b3700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff56c0a14d0 0x7ff56c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:20.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.975+0000 7ff5855b3700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff56c0a14d0 0x7ff56c0a3990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00b2e0 tx=0x7ff57c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:20.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:20.977+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff578095f00 con 0x7ff5801057c0 2023-10-18T12:23:21.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.180+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff580101f30 con 0x7ff5801057c0 2023-10-18T12:23:21.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.181+0000 7ff5767fc700 1 -- 172.21.15.70:0/392263548 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff578005020 con 0x7ff5801057c0 2023-10-18T12:23:21.184 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:21.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.183+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff56c0a14d0 msgr2=0x7ff56c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.183+0000 7ff587817700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff56c0a14d0 0x7ff56c0a3990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00b2e0 tx=0x7ff57c005fd0 comp rx=0 tx=0).stop 2023-10-18T12:23:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.183+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 msgr2=0x7ff580073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.183+0000 7ff587817700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580073570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff57800efc0 tx=0x7ff57800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:23:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 shutdown_connections 2023-10-18T12:23:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5801057c0 0x7ff580073570 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:21.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff56c0a14d0 0x7ff56c0a3990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:21.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5801008a0 0x7ff580072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:21.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 --2- 172.21.15.70:0/392263548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff580101260 0x7ff580073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:21.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 >> 172.21.15.70:0/392263548 conn(0x7ff5800779e0 msgr2=0x7ff5800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:21.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 shutdown_connections 2023-10-18T12:23:21.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:21.184+0000 7ff587817700 1 -- 172.21.15.70:0/392263548 wait complete. 2023-10-18T12:23:21.371 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:21 smithi070 ceph-mon[98639]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:21.371 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:21 smithi070 ceph-mon[98639]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-18T12:23:21.372 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:21 smithi070 ceph-mon[98639]: osdmap e78: 12 total, 11 up, 12 in 2023-10-18T12:23:21.372 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:21 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:21.372 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:21 smithi070 ceph-mon[98639]: from='osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:23:21.372 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:21 smithi070 ceph-mon[98639]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:23:21.407 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:23:21.109+0000 7f1013327700 -1 osd.11 0 waiting for initial osdmap 2023-10-18T12:23:21.407 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:23:21.123+0000 7f100a913700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T12:23:21.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-mon[135253]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:21.408 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-mon[135253]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-18T12:23:21.409 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-mon[135253]: osdmap e78: 12 total, 11 up, 12 in 2023-10-18T12:23:21.409 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:21.409 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-mon[135253]: from='osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:23:21.409 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:21 smithi186 ceph-mon[135253]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:23:21.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:21 smithi152 ceph-mon[134767]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:21 smithi152 ceph-mon[134767]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-18T12:23:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:21 smithi152 ceph-mon[134767]: osdmap e78: 12 total, 11 up, 12 in 2023-10-18T12:23:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:21 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:21 smithi152 ceph-mon[134767]: from='osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:23:21.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:21 smithi152 ceph-mon[134767]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-18T12:23:21.656 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1697631781,"num_in_osds":12,"osd_in_since":1697631784,"num_remapped_pgs":0} 2023-10-18T12:23:22.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:23:22.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: osdmap e79: 12 total, 11 up, 12 in 2023-10-18T12:23:22.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:22.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:22.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/392263548' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T12:23:22.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:22.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:22.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:22 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:23:22.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: osdmap e79: 12 total, 11 up, 12 in 2023-10-18T12:23:22.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:22.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:22.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/392263548' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T12:23:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:22.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:22 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-18T12:23:22.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: osdmap e79: 12 total, 11 up, 12 in 2023-10-18T12:23:22.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:22.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:22.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/392263548' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T12:23:22.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:23:22.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:22.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:22.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:22 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:22.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-18T12:23:23.402 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:23:23.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: purged_snaps scrub starts 2023-10-18T12:23:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: purged_snaps scrub ok 2023-10-18T12:23:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: Detected new or changed devices on smithi186 2023-10-18T12:23:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-18T12:23:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] boot 2023-10-18T12:23:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: osdmap e80: 12 total, 12 up, 12 in 2023-10-18T12:23:23.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:23 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: purged_snaps scrub starts 2023-10-18T12:23:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: purged_snaps scrub ok 2023-10-18T12:23:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: Detected new or changed devices on smithi186 2023-10-18T12:23:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-18T12:23:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:23.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] boot 2023-10-18T12:23:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: osdmap e80: 12 total, 12 up, 12 in 2023-10-18T12:23:23.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:23 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:23.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: purged_snaps scrub starts 2023-10-18T12:23:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: purged_snaps scrub ok 2023-10-18T12:23:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: Detected new or changed devices on smithi186 2023-10-18T12:23:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-18T12:23:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-18T12:23:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: osd.11 [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] boot 2023-10-18T12:23:23.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: osdmap e80: 12 total, 12 up, 12 in 2023-10-18T12:23:23.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:23 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T12:23:24.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:24 smithi070 ceph-mon[98639]: osdmap e81: 12 total, 12 up, 12 in 2023-10-18T12:23:24.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:24 smithi152 ceph-mon[134767]: osdmap e81: 12 total, 12 up, 12 in 2023-10-18T12:23:24.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:24 smithi186 ceph-mon[135253]: osdmap e81: 12 total, 12 up, 12 in 2023-10-18T12:23:24.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 -- 172.21.15.70:0/2874609649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481057c0 msgr2=0x7f8848105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:24.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 --2- 172.21.15.70:0/2874609649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481057c0 0x7f8848105ba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f883c0098f0 tx=0x7f883c009c00 comp rx=0 tx=0).stop 2023-10-18T12:23:24.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 -- 172.21.15.70:0/2874609649 shutdown_connections 2023-10-18T12:23:24.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 --2- 172.21.15.70:0/2874609649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8848101260 0x7f884810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:24.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 --2- 172.21.15.70:0/2874609649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481057c0 0x7f8848105ba0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:24.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 --2- 172.21.15.70:0/2874609649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88481008a0 0x7f8848100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:24.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 -- 172.21.15.70:0/2874609649 >> 172.21.15.70:0/2874609649 conn(0x7f88480779e0 msgr2=0x7f8848077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:24.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 -- 172.21.15.70:0/2874609649 shutdown_connections 2023-10-18T12:23:24.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.935+0000 7f884ec0c700 1 -- 172.21.15.70:0/2874609649 wait complete. 2023-10-18T12:23:24.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.936+0000 7f884ec0c700 1 Processor -- start 2023-10-18T12:23:24.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.936+0000 7f884ec0c700 1 -- start start 2023-10-18T12:23:24.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884ec0c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 0x7f884819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:24.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884ec0c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8848101260 0x7f884819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:24.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884ec0c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88481057c0 0x7f8848196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:24.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884ec0c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8848197120 con 0x7f88481008a0 2023-10-18T12:23:24.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884ec0c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8848197290 con 0x7f8848101260 2023-10-18T12:23:24.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884ec0c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8848197450 con 0x7f88481057c0 2023-10-18T12:23:24.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 0x7f884819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:24.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 0x7f884819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43284/0 (socket says 172.21.15.70:43284) 2023-10-18T12:23:24.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 -- 172.21.15.70:0/18088384 learned_addr learned my addr 172.21.15.70:0/18088384 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:24.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884d1a9700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88481057c0 0x7f8848196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:24.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f8847fff700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8848101260 0x7f884819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:24.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 -- 172.21.15.70:0/18088384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88481057c0 msgr2=0x7f8848196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:24.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88481057c0 0x7f8848196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:24.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 -- 172.21.15.70:0/18088384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8848101260 msgr2=0x7f884819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:24.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.937+0000 7f884c9a8700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8848101260 0x7f884819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:24.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f884c9a8700 1 -- 172.21.15.70:0/18088384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c009520 con 0x7f88481008a0 2023-10-18T12:23:24.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f8847fff700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8848101260 0x7f884819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:23:24.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f884c9a8700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 0x7f884819c980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f883c009230 tx=0x7f883c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:24.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c01c070 con 0x7f88481008a0 2023-10-18T12:23:24.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883c00bb20 con 0x7f88481008a0 2023-10-18T12:23:24.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88481976d0 con 0x7f88481008a0 2023-10-18T12:23:24.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0038e0 con 0x7f88481008a0 2023-10-18T12:23:24.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.938+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88481a3970 con 0x7f88481008a0 2023-10-18T12:23:24.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.939+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8848108d80 con 0x7f88481008a0 2023-10-18T12:23:24.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.941+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f883c020070 con 0x7f88481008a0 2023-10-18T12:23:24.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.943+0000 7f8845ffb700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f88380a1550 0x7f88380a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:24.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.943+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f883c0cd390 con 0x7f88481008a0 2023-10-18T12:23:24.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.943+0000 7f8847fff700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f88380a1550 0x7f88380a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:24.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.944+0000 7f8847fff700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f88380a1550 0x7f88380a3a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8848197c80 tx=0x7f8834009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:24.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:24.948+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f883c096db0 con 0x7f88481008a0 2023-10-18T12:23:25.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.151+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8848101f30 con 0x7f88481008a0 2023-10-18T12:23:25.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.151+0000 7f8845ffb700 1 -- 172.21.15.70:0/18088384 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f883c09a3d0 con 0x7f88481008a0 2023-10-18T12:23:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:25.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f88380a1550 msgr2=0x7f88380a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:25.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f88380a1550 0x7f88380a3a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8848197c80 tx=0x7f8834009260 comp rx=0 tx=0).stop 2023-10-18T12:23:25.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 msgr2=0x7f884819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:25.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 0x7f884819c980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f883c009230 tx=0x7f883c00fa00 comp rx=0 tx=0).stop 2023-10-18T12:23:25.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 shutdown_connections 2023-10-18T12:23:25.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88481057c0 0x7f8848196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:25.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88481008a0 0x7f884819c980 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:25.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f88380a1550 0x7f88380a3a10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:25.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 --2- 172.21.15.70:0/18088384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8848101260 0x7f884819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:25.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 >> 172.21.15.70:0/18088384 conn(0x7f88480779e0 msgr2=0x7f88480fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:25.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 shutdown_connections 2023-10-18T12:23:25.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:25.154+0000 7f884ec0c700 1 -- 172.21.15.70:0/18088384 wait complete. 2023-10-18T12:23:25.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:25 smithi152 ceph-mon[134767]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:25 smithi186 ceph-mon[135253]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:25.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:25 smithi070 ceph-mon[98639]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:25.593 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1697631802,"num_in_osds":12,"osd_in_since":1697631784,"num_remapped_pgs":0} 2023-10-18T12:23:25.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-18T12:23:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:23:26.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:26 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/18088384' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T12:23:26.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:26 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/18088384' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T12:23:26.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:26 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/18088384' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T12:23:27.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:27 smithi152 ceph-mon[134767]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:27 smithi186 ceph-mon[135253]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:27.499 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:27 smithi070 ceph-mon[98639]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:27.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 -- 172.21.15.70:0/1351852367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f181057c0 msgr2=0x7f8f18105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:27.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/1351852367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f181057c0 0x7f8f18105ba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009990 tx=0x7f8f0c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 -- 172.21.15.70:0/1351852367 shutdown_connections 2023-10-18T12:23:27.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/1351852367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f18101260 0x7f8f1810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/1351852367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f181057c0 0x7f8f18105ba0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/1351852367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f181008a0 0x7f8f18100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 -- 172.21.15.70:0/1351852367 >> 172.21.15.70:0/1351852367 conn(0x7f8f180779e0 msgr2=0x7f8f18077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:27.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 -- 172.21.15.70:0/1351852367 shutdown_connections 2023-10-18T12:23:27.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.743+0000 7f8f1f21b700 1 -- 172.21.15.70:0/1351852367 wait complete. 2023-10-18T12:23:27.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.744+0000 7f8f1f21b700 1 Processor -- start 2023-10-18T12:23:27.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.744+0000 7f8f1f21b700 1 -- start start 2023-10-18T12:23:27.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1f21b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f181008a0 0x7f8f18072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:27.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1f21b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 0x7f8f18073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:27.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1f21b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f181057c0 0x7f8f18073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:27.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1f21b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f181a5030 con 0x7f8f18101260 2023-10-18T12:23:27.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1f21b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f181a51a0 con 0x7f8f181057c0 2023-10-18T12:23:27.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1f21b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f181a5360 con 0x7f8f181008a0 2023-10-18T12:23:27.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 0x7f8f18073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:27.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 0x7f8f18073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43312/0 (socket says 172.21.15.70:43312) 2023-10-18T12:23:27.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 -- 172.21.15.70:0/3851557822 learned_addr learned my addr 172.21.15.70:0/3851557822 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:27.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1cfb7700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f181008a0 0x7f8f18072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:27.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f1d7b8700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f181057c0 0x7f8f18073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:27.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 -- 172.21.15.70:0/3851557822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f181008a0 msgr2=0x7f8f18072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:27.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f181008a0 0x7f8f18072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 -- 172.21.15.70:0/3851557822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f181057c0 msgr2=0x7f8f18073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:27.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f181057c0 0x7f8f18073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.745+0000 7f8f17fff700 1 -- 172.21.15.70:0/3851557822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0095f0 con 0x7f8f18101260 2023-10-18T12:23:27.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.746+0000 7f8f17fff700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 0x7f8f18073030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400e9a0 tx=0x7f8f0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:27.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.746+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04009790 con 0x7f8f18101260 2023-10-18T12:23:27.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.746+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f04010430 con 0x7f8f18101260 2023-10-18T12:23:27.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.746+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f181a5640 con 0x7f8f18101260 2023-10-18T12:23:27.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.746+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04018680 con 0x7f8f18101260 2023-10-18T12:23:27.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.746+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f181a5b90 con 0x7f8f18101260 2023-10-18T12:23:27.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.747+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f18108d80 con 0x7f8f18101260 2023-10-18T12:23:27.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.749+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8f040105a0 con 0x7f8f18101260 2023-10-18T12:23:27.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.750+0000 7f8f15ffb700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f080a1550 0x7f8f080a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:27.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.750+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f8f04014070 con 0x7f8f18101260 2023-10-18T12:23:27.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.750+0000 7f8f1cfb7700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f080a1550 0x7f8f080a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:27.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.751+0000 7f8f1cfb7700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f080a1550 0x7f8f080a3a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00b2e0 tx=0x7f8f0c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:27.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.754+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f04096510 con 0x7f8f18101260 2023-10-18T12:23:27.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.956+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f18101f30 con 0x7f8f18101260 2023-10-18T12:23:27.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.957+0000 7f8f15ffb700 1 -- 172.21.15.70:0/3851557822 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18748 (secure 0 0 0) 0x7f8f04099b30 con 0x7f8f18101260 2023-10-18T12:23:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:27.961 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":81,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:23:23.115758+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:23:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f080a1550 msgr2=0x7f8f080a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f080a1550 0x7f8f080a3a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00b2e0 tx=0x7f8f0c005fd0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 msgr2=0x7f8f18073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 0x7f8f18073030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400e9a0 tx=0x7f8f0400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 shutdown_connections 2023-10-18T12:23:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f080a1550 0x7f8f080a3a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f181057c0 0x7f8f18073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f181008a0 0x7f8f18072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 --2- 172.21.15.70:0/3851557822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18101260 0x7f8f18073030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 >> 172.21.15.70:0/3851557822 conn(0x7f8f180779e0 msgr2=0x7f8f180fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.959+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 shutdown_connections 2023-10-18T12:23:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:27.960+0000 7f8f1f21b700 1 -- 172.21.15.70:0/3851557822 wait complete. 2023-10-18T12:23:28.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:28 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3851557822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:23:28.407 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:23:28.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-18T12:23:28.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:28 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3851557822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:23:28.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:28 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3851557822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:23:29.139 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:23:29.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:29 smithi070 ceph-mon[98639]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:29.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:29 smithi152 ceph-mon[134767]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:29 smithi186 ceph-mon[135253]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:30.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 -- 172.21.15.70:0/1899196794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741057c0 msgr2=0x7f2074105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:30.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 --2- 172.21.15.70:0/1899196794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741057c0 0x7f2074105ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2070009990 tx=0x7f2070009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 -- 172.21.15.70:0/1899196794 shutdown_connections 2023-10-18T12:23:30.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 --2- 172.21.15.70:0/1899196794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2074101260 0x7f207410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 --2- 172.21.15.70:0/1899196794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741057c0 0x7f2074105ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 --2- 172.21.15.70:0/1899196794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20741008a0 0x7f2074100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.552+0000 7f207c743700 1 -- 172.21.15.70:0/1899196794 >> 172.21.15.70:0/1899196794 conn(0x7f20740779e0 msgr2=0x7f2074077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:30.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.553+0000 7f207c743700 1 -- 172.21.15.70:0/1899196794 shutdown_connections 2023-10-18T12:23:30.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.553+0000 7f207c743700 1 -- 172.21.15.70:0/1899196794 wait complete. 2023-10-18T12:23:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 Processor -- start 2023-10-18T12:23:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 -- start start 2023-10-18T12:23:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 0x7f2074198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074101260 0x7f2074199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20741057c0 0x7f207419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419de00 con 0x7f20741057c0 2023-10-18T12:23:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419df70 con 0x7f20741008a0 2023-10-18T12:23:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.554+0000 7f207c743700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419e130 con 0x7f2074101260 2023-10-18T12:23:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 0x7f2074198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 0x7f2074198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59604/0 (socket says 172.21.15.70:59604) 2023-10-18T12:23:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 -- 172.21.15.70:0/4002200081 learned_addr learned my addr 172.21.15.70:0/4002200081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207ace0700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20741057c0 0x7f207419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f2079cde700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074101260 0x7f2074199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 -- 172.21.15.70:0/4002200081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074101260 msgr2=0x7f2074199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074101260 0x7f2074199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 -- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20741057c0 msgr2=0x7f207419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20741057c0 0x7f207419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 -- 172.21.15.70:0/4002200081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c009c40 con 0x7f20741008a0 2023-10-18T12:23:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f2079cde700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074101260 0x7f2074199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:23:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207a4df700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 0x7f2074198f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2070000c00 tx=0x7f2070005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207001d070 con 0x7f20741008a0 2023-10-18T12:23:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2070022470 con 0x7f20741008a0 2023-10-18T12:23:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20700095f0 con 0x7f20741008a0 2023-10-18T12:23:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.555+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070017650 con 0x7f20741008a0 2023-10-18T12:23:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.556+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207419e630 con 0x7f20741008a0 2023-10-18T12:23:30.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.557+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2074101f30 con 0x7f20741008a0 2023-10-18T12:23:30.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.559+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f207000fad0 con 0x7f20741008a0 2023-10-18T12:23:30.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.560+0000 7f206b7fe700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f20600a14d0 0x7f20600a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:30.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.560+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f20700ce470 con 0x7f20741008a0 2023-10-18T12:23:30.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.560+0000 7f2079cde700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f20600a14d0 0x7f20600a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:30.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.561+0000 7f2079cde700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f20600a14d0 0x7f20600a3990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f207419a460 tx=0x7f2064009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:30.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.562+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2070097e90 con 0x7f20741008a0 2023-10-18T12:23:30.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.761+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2074110760 con 0x7f20741008a0 2023-10-18T12:23:30.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.761+0000 7f206b7fe700 1 -- 172.21.15.70:0/4002200081 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f207009b4b0 con 0x7f20741008a0 2023-10-18T12:23:30.765 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:23:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f20600a14d0 msgr2=0x7f20600a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f20600a14d0 0x7f20600a3990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f207419a460 tx=0x7f2064009260 comp rx=0 tx=0).stop 2023-10-18T12:23:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 msgr2=0x7f2074198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 0x7f2074198f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2070000c00 tx=0x7f2070005740 comp rx=0 tx=0).stop 2023-10-18T12:23:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 shutdown_connections 2023-10-18T12:23:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20741057c0 0x7f207419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f20600a14d0 0x7f20600a3990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20741008a0 0x7f2074198f50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 --2- 172.21.15.70:0/4002200081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074101260 0x7f2074199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 >> 172.21.15.70:0/4002200081 conn(0x7f20740779e0 msgr2=0x7f207410bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 shutdown_connections 2023-10-18T12:23:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:30.764+0000 7f207c743700 1 -- 172.21.15.70:0/4002200081 wait complete. 2023-10-18T12:23:31.218 INFO:tasks.cephadm:Adding mds.a on smithi070 2023-10-18T12:23:31.218 INFO:tasks.cephadm:Adding mds.d on smithi070 2023-10-18T12:23:31.219 INFO:tasks.cephadm:Adding mds.g on smithi070 2023-10-18T12:23:31.219 INFO:tasks.cephadm:Adding mds.j on smithi070 2023-10-18T12:23:31.219 INFO:tasks.cephadm:Adding mds.b on smithi152 2023-10-18T12:23:31.219 INFO:tasks.cephadm:Adding mds.e on smithi152 2023-10-18T12:23:31.220 INFO:tasks.cephadm:Adding mds.h on smithi152 2023-10-18T12:23:31.220 INFO:tasks.cephadm:Adding mds.k on smithi152 2023-10-18T12:23:31.220 INFO:tasks.cephadm:Adding mds.c on smithi186 2023-10-18T12:23:31.220 INFO:tasks.cephadm:Adding mds.f on smithi186 2023-10-18T12:23:31.221 INFO:tasks.cephadm:Adding mds.i on smithi186 2023-10-18T12:23:31.221 INFO:tasks.cephadm:Adding mds.l on smithi186 2023-10-18T12:23:31.221 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph orch apply mds all '12;smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2023-10-18T12:23:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:31 smithi152 ceph-mon[134767]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:31.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:31 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4002200081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:23:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:31 smithi186 ceph-mon[135253]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:31 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4002200081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:23:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:31 smithi070 ceph-mon[98639]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:31.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:31 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4002200081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:23:31.794 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:23:33.180 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 -- 172.21.15.186:0/3316733589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0103e70 msgr2=0x7fddf0104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:33.181 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 --2- 172.21.15.186:0/3316733589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0103e70 0x7fddf0104250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009990 tx=0x7fdde0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.181 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 -- 172.21.15.186:0/3316733589 shutdown_connections 2023-10-18T12:23:33.181 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 --2- 172.21.15.186:0/3316733589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf00ffb50 0x7fddf010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.181 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 --2- 172.21.15.186:0/3316733589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0103e70 0x7fddf0104250 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.181 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 --2- 172.21.15.186:0/3316733589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf00ff190 0x7fddf00ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.182 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 -- 172.21.15.186:0/3316733589 >> 172.21.15.186:0/3316733589 conn(0x7fddf0074a10 msgr2=0x7fddf0074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:33.182 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 -- 172.21.15.186:0/3316733589 shutdown_connections 2023-10-18T12:23:33.182 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.178+0000 7fddf4886700 1 -- 172.21.15.186:0/3316733589 wait complete. 2023-10-18T12:23:33.184 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 Processor -- start 2023-10-18T12:23:33.184 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 -- start start 2023-10-18T12:23:33.185 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf00ff190 0x7fddf019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:33.185 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 0x7fddf019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:33.185 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf0103e70 0x7fddf0195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:33.185 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf01960a0 con 0x7fddf00ff190 2023-10-18T12:23:33.185 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0196210 con 0x7fddf0103e70 2023-10-18T12:23:33.186 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddf4886700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf01963d0 con 0x7fddf00ffb50 2023-10-18T12:23:33.186 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 0x7fddf019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:33.186 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 0x7fddf019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:50890/0 (socket says 172.21.15.186:50890) 2023-10-18T12:23:33.186 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 -- 172.21.15.186:0/1899702588 learned_addr learned my addr 172.21.15.186:0/1899702588 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-18T12:23:33.187 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 -- 172.21.15.186:0/1899702588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf0103e70 msgr2=0x7fddf0195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:33.187 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf0103e70 0x7fddf0195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.187 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 -- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf00ff190 msgr2=0x7fddf019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:33.187 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf00ff190 0x7fddf019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.187 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 -- 172.21.15.186:0/1899702588 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00095f0 con 0x7fddf00ffb50 2023-10-18T12:23:33.188 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fddedd9b700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 0x7fddf019bdf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdde400d9e0 tx=0x7fdde400dda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:33.188 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.179+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde400ecd0 con 0x7fddf00ffb50 2023-10-18T12:23:33.188 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.180+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde400ee30 con 0x7fddf00ffb50 2023-10-18T12:23:33.188 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.180+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4014800 con 0x7fddf00ffb50 2023-10-18T12:23:33.188 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.180+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf01966b0 con 0x7fddf00ffb50 2023-10-18T12:23:33.189 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.180+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf0100c70 con 0x7fddf00ffb50 2023-10-18T12:23:33.189 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.181+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf01005b0 con 0x7fddf00ffb50 2023-10-18T12:23:33.189 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.181+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fdde4016b20 con 0x7fddf00ffb50 2023-10-18T12:23:33.189 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.182+0000 7fdddf7fe700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddd80a14d0 0x7fddd80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:33.189 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.182+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fdde40ce260 con 0x7fddf00ffb50 2023-10-18T12:23:33.189 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.182+0000 7fddee59c700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddd80a14d0 0x7fddd80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:33.190 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.183+0000 7fddee59c700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddd80a14d0 0x7fddd80a3990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdde00095c0 tx=0x7fdde0009510 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:33.190 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.187+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdde4097d20 con 0x7fddf00ffb50 2023-10-18T12:23:33.387 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.384+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fddf010eec0 con 0x7fddd80a14d0 2023-10-18T12:23:33.395 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.393+0000 7fdddf7fe700 1 -- 172.21.15.186:0/1899702588 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fddf010eec0 con 0x7fddd80a14d0 2023-10-18T12:23:33.395 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2023-10-18T12:23:33.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddd80a14d0 msgr2=0x7fddd80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:33.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddd80a14d0 0x7fddd80a3990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdde00095c0 tx=0x7fdde0009510 comp rx=0 tx=0).stop 2023-10-18T12:23:33.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 msgr2=0x7fddf019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:33.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 0x7fddf019bdf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdde400d9e0 tx=0x7fdde400dda0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 shutdown_connections 2023-10-18T12:23:33.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddd80a14d0 0x7fddd80a3990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf0103e70 0x7fddf0195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf00ff190 0x7fddf019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.399 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 --2- 172.21.15.186:0/1899702588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf00ffb50 0x7fddf019bdf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:33.399 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 >> 172.21.15.186:0/1899702588 conn(0x7fddf0074a10 msgr2=0x7fddf00fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:33.399 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 shutdown_connections 2023-10-18T12:23:33.399 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-18T12:23:33.395+0000 7fddf4886700 1 -- 172.21.15.186:0/1899702588 wait complete. 2023-10-18T12:23:33.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:33 smithi186 ceph-mon[135253]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:33.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:33 smithi152 ceph-mon[134767]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:33.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:33 smithi070 ceph-mon[98639]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:33.749 DEBUG:teuthology.orchestra.run.smithi070:mds.a> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.a.service 2023-10-18T12:23:33.754 DEBUG:teuthology.orchestra.run.smithi070:mds.d> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.d.service 2023-10-18T12:23:33.800 DEBUG:teuthology.orchestra.run.smithi070:mds.g> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.g.service 2023-10-18T12:23:33.833 INFO:journalctl@ceph.mds.a.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:23:33.837 DEBUG:teuthology.orchestra.run.smithi070:mds.j> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.j.service 2023-10-18T12:23:33.872 INFO:journalctl@ceph.mds.d.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:23:33.875 DEBUG:teuthology.orchestra.run.smithi152:mds.b> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.b.service 2023-10-18T12:23:33.878 DEBUG:teuthology.orchestra.run.smithi152:mds.e> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.e.service 2023-10-18T12:23:33.908 INFO:journalctl@ceph.mds.g.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:23:33.915 INFO:journalctl@ceph.mds.b.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:23:33.917 DEBUG:teuthology.orchestra.run.smithi152:mds.h> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.h.service 2023-10-18T12:23:33.943 INFO:journalctl@ceph.mds.j.smithi070.stdout:-- Logs begin at Wed 2023-10-18 11:58:38 UTC. -- 2023-10-18T12:23:33.945 INFO:journalctl@ceph.mds.e.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:23:33.946 DEBUG:teuthology.orchestra.run.smithi152:mds.k> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.k.service 2023-10-18T12:23:33.973 INFO:journalctl@ceph.mds.h.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:23:33.974 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.c.service 2023-10-18T12:23:33.977 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.f.service 2023-10-18T12:23:34.001 INFO:journalctl@ceph.mds.k.smithi152.stdout:-- Logs begin at Wed 2023-10-18 11:54:49 UTC. -- 2023-10-18T12:23:34.006 INFO:journalctl@ceph.mds.c.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:23:34.007 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.i.service 2023-10-18T12:23:34.032 INFO:journalctl@ceph.mds.f.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:23:34.033 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.l.service 2023-10-18T12:23:34.059 INFO:journalctl@ceph.mds.i.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:23:34.060 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-18T12:23:34.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:23:34.085 INFO:journalctl@ceph.mds.l.smithi186.stdout:-- Logs begin at Wed 2023-10-18 11:54:33 UTC. -- 2023-10-18T12:23:34.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 -- 172.21.15.70:0/2744374493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 msgr2=0x7fe3bc10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/2744374493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 0x7fe3bc10d570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400b0a0 tx=0x7fe3b400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 -- 172.21.15.70:0/2744374493 shutdown_connections 2023-10-18T12:23:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/2744374493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 0x7fe3bc10d570 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/2744374493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc1066b0 0x7fe3bc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/2744374493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 -- 172.21.15.70:0/2744374493 >> 172.21.15.70:0/2744374493 conn(0x7fe3bc0fc170 msgr2=0x7fe3bc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 -- 172.21.15.70:0/2744374493 shutdown_connections 2023-10-18T12:23:34.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.286+0000 7fe3c3fff700 1 -- 172.21.15.70:0/2744374493 wait complete. 2023-10-18T12:23:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 Processor -- start 2023-10-18T12:23:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 -- start start 2023-10-18T12:23:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 0x7fe3bc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc1066b0 0x7fe3bc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1992a0 con 0x7fe3bc101060 2023-10-18T12:23:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc199410 con 0x7fe3bc1066b0 2023-10-18T12:23:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1995d0 con 0x7fe3bc1006a0 2023-10-18T12:23:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.287+0000 7fe3c1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41368/0 (socket says 172.21.15.70:41368) 2023-10-18T12:23:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 -- 172.21.15.70:0/1572731487 learned_addr learned my addr 172.21.15.70:0/1572731487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c159a700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 0x7fe3bc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c259c700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc1066b0 0x7fe3bc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 -- 172.21.15.70:0/1572731487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc1066b0 msgr2=0x7fe3bc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc1066b0 0x7fe3bc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 -- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 msgr2=0x7fe3bc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 0x7fe3bc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 -- 172.21.15.70:0/1572731487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4009d00 con 0x7fe3bc1006a0 2023-10-18T12:23:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c1d9b700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc19ea30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800e9a0 tx=0x7fe3b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8009790 con 0x7fe3bc1006a0 2023-10-18T12:23:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc1998b0 con 0x7fe3bc1006a0 2023-10-18T12:23:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3b8010430 con 0x7fe3bc1006a0 2023-10-18T12:23:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.288+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8018680 con 0x7fe3bc1006a0 2023-10-18T12:23:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.289+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc1a5a00 con 0x7fe3bc1006a0 2023-10-18T12:23:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.289+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc101de0 con 0x7fe3bc1006a0 2023-10-18T12:23:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.291+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe3b80105a0 con 0x7fe3bc1006a0 2023-10-18T12:23:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.293+0000 7fe3b2ffd700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3a80a14d0 0x7fe3a80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.293+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fe3b8014070 con 0x7fe3bc1006a0 2023-10-18T12:23:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.293+0000 7fe3c159a700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3a80a14d0 0x7fe3a80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.293+0000 7fe3c159a700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3a80a14d0 0x7fe3a80a3990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc19a240 tx=0x7fe3ac009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:34.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.295+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe3b8096540 con 0x7fe3bc1006a0 2023-10-18T12:23:34.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.500+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3bc067530 con 0x7fe3bc1006a0 2023-10-18T12:23:34.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.502+0000 7fe3b2ffd700 1 -- 172.21.15.70:0/1572731487 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18748 (secure 0 0 0) 0x7fe3b8099b60 con 0x7fe3bc1006a0 2023-10-18T12:23:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:34.505 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":81,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:23:23.115758+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:23:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3a80a14d0 msgr2=0x7fe3a80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3a80a14d0 0x7fe3a80a3990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc19a240 tx=0x7fe3ac009260 comp rx=0 tx=0).stop 2023-10-18T12:23:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 msgr2=0x7fe3bc19ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc19ea30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800e9a0 tx=0x7fe3b800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 shutdown_connections 2023-10-18T12:23:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3a80a14d0 0x7fe3a80a3990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc1066b0 0x7fe3bc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3bc1006a0 0x7fe3bc19ea30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 --2- 172.21.15.70:0/1572731487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3bc101060 0x7fe3bc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 >> 172.21.15.70:0/1572731487 conn(0x7fe3bc0fc170 msgr2=0x7fe3bc10bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 shutdown_connections 2023-10-18T12:23:34.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.504+0000 7fe3c3fff700 1 -- 172.21.15.70:0/1572731487 wait complete. 2023-10-18T12:23:34.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:23:34.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:23:34.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:23:34.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: Saving service mds.all spec with placement smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-18T12:23:34.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:34.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:34.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:34.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:34.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:34.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:34.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:34.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:34.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: Deploying daemon mds.l on smithi186 2023-10-18T12:23:34.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:34 smithi186 ceph-mon[135253]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:34.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:23:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: Saving service mds.all spec with placement smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-18T12:23:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:34.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:34.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:34.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:34.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:34.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:34.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: Deploying daemon mds.l on smithi186 2023-10-18T12:23:34.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:34 smithi152 ceph-mon[134767]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:34.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 -- 172.21.15.70:0/3988080355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 msgr2=0x7fad3c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 --2- 172.21.15.70:0/3988080355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c10d5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b0a0 tx=0x7fad3400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 -- 172.21.15.70:0/3988080355 shutdown_connections 2023-10-18T12:23:34.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 --2- 172.21.15.70:0/3988080355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c10d5f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 --2- 172.21.15.70:0/3988080355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106730 0x7fad3c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 --2- 172.21.15.70:0/3988080355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 -- 172.21.15.70:0/3988080355 >> 172.21.15.70:0/3988080355 conn(0x7fad3c0fc1b0 msgr2=0x7fad3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:34.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 -- 172.21.15.70:0/3988080355 shutdown_connections 2023-10-18T12:23:34.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.726+0000 7fad431ff700 1 -- 172.21.15.70:0/3988080355 wait complete. 2023-10-18T12:23:34.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.727+0000 7fad431ff700 1 Processor -- start 2023-10-18T12:23:34.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 -- start start 2023-10-18T12:23:34.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106730 0x7fad3c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19cb40 con 0x7fad3c100720 2023-10-18T12:23:34.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19ccb0 con 0x7fad3c106730 2023-10-18T12:23:34.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad431ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19ce70 con 0x7fad3c1010e0 2023-10-18T12:23:34.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad40f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad3bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad40f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58010/0 (socket says 172.21.15.70:58010) 2023-10-18T12:23:34.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad40f9b700 1 -- 172.21.15.70:0/575093868 learned_addr learned my addr 172.21.15.70:0/575093868 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:34.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad3bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41384/0 (socket says 172.21.15.70:41384) 2023-10-18T12:23:34.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.728+0000 7fad4179c700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106730 0x7fad3c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad40f9b700 1 -- 172.21.15.70:0/575093868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 msgr2=0x7fad3c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad40f9b700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad40f9b700 1 -- 172.21.15.70:0/575093868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106730 msgr2=0x7fad3c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad40f9b700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106730 0x7fad3c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad40f9b700 1 -- 172.21.15.70:0/575093868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009520 con 0x7fad3c100720 2023-10-18T12:23:34.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad40f9b700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c198d60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad3000e9a0 tx=0x7fad3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:34.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30009790 con 0x7fad3c100720 2023-10-18T12:23:34.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad3000cbd0 con 0x7fad3c100720 2023-10-18T12:23:34.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30018680 con 0x7fad3c100720 2023-10-18T12:23:34.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad34009d00 con 0x7fad3c100720 2023-10-18T12:23:34.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.729+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c1a58d0 con 0x7fad3c100720 2023-10-18T12:23:34.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.730+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c101e60 con 0x7fad3c100720 2023-10-18T12:23:34.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.732+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fad3000cd40 con 0x7fad3c100720 2023-10-18T12:23:34.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.733+0000 7fad39ffb700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fad2c0a14d0 0x7fad2c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:34.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.733+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fad30014070 con 0x7fad3c100720 2023-10-18T12:23:34.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.733+0000 7fad3bfff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fad2c0a14d0 0x7fad2c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:34.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.734+0000 7fad3bfff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fad2c0a14d0 0x7fad2c0a3990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad3c19a220 tx=0x7fad28009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:34.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.737+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fad30052100 con 0x7fad3c100720 2023-10-18T12:23:34.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:23:34.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: Saving service mds.all spec with placement smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-18T12:23:34.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:34.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:23:34.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:34.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:23:34.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:34.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:34.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:34.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:34.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: Deploying daemon mds.l on smithi186 2023-10-18T12:23:34.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:34 smithi070 ceph-mon[98639]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:34.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.938+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad3c1106f0 con 0x7fad3c100720 2023-10-18T12:23:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.939+0000 7fad39ffb700 1 -- 172.21.15.70:0/575093868 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad300965d0 con 0x7fad3c100720 2023-10-18T12:23:34.942 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:23:34.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fad2c0a14d0 msgr2=0x7fad2c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fad2c0a14d0 0x7fad2c0a3990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad3c19a220 tx=0x7fad28009260 comp rx=0 tx=0).stop 2023-10-18T12:23:34.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 msgr2=0x7fad3c198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:34.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c198d60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad3000e9a0 tx=0x7fad3000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 shutdown_connections 2023-10-18T12:23:34.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fad2c0a14d0 0x7fad2c0a3990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106730 0x7fad3c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c100720 0x7fad3c198d60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 --2- 172.21.15.70:0/575093868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad3c1010e0 0x7fad3c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:34.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 >> 172.21.15.70:0/575093868 conn(0x7fad3c0fc1b0 msgr2=0x7fad3c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:34.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 shutdown_connections 2023-10-18T12:23:34.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:34.941+0000 7fad431ff700 1 -- 172.21.15.70:0/575093868 wait complete. 2023-10-18T12:23:34.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:35.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1513369178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f8101060 msgr2=0x7f54f810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:35.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1513369178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f8101060 0x7f54f810d570 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b0a0 tx=0x7f54f000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1513369178 shutdown_connections 2023-10-18T12:23:35.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1513369178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f8101060 0x7f54f810d570 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1513369178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f81066b0 0x7f54f8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1513369178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f81006a0 0x7f54f8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1513369178 >> 172.21.15.70:0/1513369178 conn(0x7f54f80fc170 msgr2=0x7f54f80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:35.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1513369178 shutdown_connections 2023-10-18T12:23:35.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.159+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1513369178 wait complete. 2023-10-18T12:23:35.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 Processor -- start 2023-10-18T12:23:35.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 -- start start 2023-10-18T12:23:35.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f81006a0 0x7f54f819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:35.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f8101060 0x7f54f819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:35.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f81066b0 0x7f54f8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:35.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f8199270 con 0x7f54f81066b0 2023-10-18T12:23:35.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f81993e0 con 0x7f54f8101060 2023-10-18T12:23:35.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fe9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f81995a0 con 0x7f54f81006a0 2023-10-18T12:23:35.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54fcf9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f81066b0 0x7f54f8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:35.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.160+0000 7f54f77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f8101060 0x7f54f819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:35.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54f77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f8101060 0x7f54f819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59642/0 (socket says 172.21.15.70:59642) 2023-10-18T12:23:35.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54f7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f81006a0 0x7f54f819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:35.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54f77fe700 1 -- 172.21.15.70:0/1737508959 learned_addr learned my addr 172.21.15.70:0/1737508959 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:35.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fcf9c700 1 -- 172.21.15.70:0/1737508959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f81006a0 msgr2=0x7f54f819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:35.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fcf9c700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f81006a0 0x7f54f819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fcf9c700 1 -- 172.21.15.70:0/1737508959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f8101060 msgr2=0x7f54f819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:35.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fcf9c700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f8101060 0x7f54f819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fcf9c700 1 -- 172.21.15.70:0/1737508959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8009520 con 0x7f54f81066b0 2023-10-18T12:23:35.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fcf9c700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f81066b0 0x7f54f8198aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b070 tx=0x7f54f00077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:35.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f000e040 con 0x7f54f81066b0 2023-10-18T12:23:35.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f0004e60 con 0x7f54f81066b0 2023-10-18T12:23:35.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f0009d00 con 0x7f54f81066b0 2023-10-18T12:23:35.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0005040 con 0x7f54f81066b0 2023-10-18T12:23:35.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.161+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f81a57a0 con 0x7f54f81066b0 2023-10-18T12:23:35.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.162+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f8101de0 con 0x7f54f81066b0 2023-10-18T12:23:35.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.164+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f54f001d470 con 0x7f54f81066b0 2023-10-18T12:23:35.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.165+0000 7f54f57fa700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54e00a1550 0x7f54e00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:35.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.166+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f54f00ce420 con 0x7f54f81066b0 2023-10-18T12:23:35.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.166+0000 7f54f7fff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54e00a1550 0x7f54e00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:35.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.166+0000 7f54f7fff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54e00a1550 0x7f54e00a3a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009d70 tx=0x7f54e8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:35.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.168+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f54f0097f60 con 0x7f54f81066b0 2023-10-18T12:23:35.219 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:23:34 smithi186 systemd[1]: Starting Ceph mds.l for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.417+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54f8110620 con 0x7f54f81066b0 2023-10-18T12:23:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.417+0000 7f54f57fa700 1 -- 172.21.15.70:0/1737508959 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f54f009b580 con 0x7f54f81066b0 2023-10-18T12:23:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:35.421 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-18T12:23:35.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54e00a1550 msgr2=0x7f54e00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:35.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54e00a1550 0x7f54e00a3a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009d70 tx=0x7f54e8009260 comp rx=0 tx=0).stop 2023-10-18T12:23:35.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f81066b0 msgr2=0x7f54f8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:35.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f81066b0 0x7f54f8198aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b070 tx=0x7f54f00077f0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 shutdown_connections 2023-10-18T12:23:35.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54f81066b0 0x7f54f8198aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f81006a0 0x7f54f819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54e00a1550 0x7f54e00a3a10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 --2- 172.21.15.70:0/1737508959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54f8101060 0x7f54f819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:35.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 >> 172.21.15.70:0/1737508959 conn(0x7f54f80fc170 msgr2=0x7f54f810bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:35.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 shutdown_connections 2023-10-18T12:23:35.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:35.420+0000 7f54fe9ff700 1 -- 172.21.15.70:0/1737508959 wait complete. 2023-10-18T12:23:35.425 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 1 2023-10-18T12:23:35.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1572731487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:23:35.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/575093868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:23:35.532 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1737508959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:35.533 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:23:35 smithi186 podman[173182]: 2023-10-18T12:23:35.533 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:23:35 smithi186 ceph-mds[173223]: -- 172.21.15.186:0/2522880540 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f164e11e0 con 0x555f164b6800 2023-10-18T12:23:35.533 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:23:35 smithi186 bash[173182]: 764ad2d4a69c9d0e7f84ca8b12ff6bdeb870ec611fdb795a76a6bfea48339134 2023-10-18T12:23:35.533 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:23:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-l[173198]: starting mds.l at 2023-10-18T12:23:35.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1572731487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:23:35.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/575093868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:23:35.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1737508959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:35.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1572731487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:23:35.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/575093868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:23:35.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1737508959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:35.968 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:23:35 smithi186 systemd[1]: Started Ceph mds.l for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:36.523 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.186:6832/339260253,v1:172.21.15.186:6833/339260253] up:boot 2023-10-18T12:23:36.523 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: fsmap 1 up:standby 2023-10-18T12:23:36.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T12:23:36.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:36.524 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:36.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:36.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:36.525 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:36 smithi152 ceph-mon[134767]: Deploying daemon mds.k on smithi152 2023-10-18T12:23:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.186:6832/339260253,v1:172.21.15.186:6833/339260253] up:boot 2023-10-18T12:23:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: fsmap 1 up:standby 2023-10-18T12:23:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T12:23:36.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:36.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:36.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:36.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:36 smithi070 ceph-mon[98639]: Deploying daemon mds.k on smithi152 2023-10-18T12:23:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.186:6832/339260253,v1:172.21.15.186:6833/339260253] up:boot 2023-10-18T12:23:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: fsmap 1 up:standby 2023-10-18T12:23:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T12:23:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:36.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:36.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:36 smithi186 ceph-mon[135253]: Deploying daemon mds.k on smithi152 2023-10-18T12:23:37.171 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:23:37 smithi152 systemd[1]: Starting Ceph mds.k for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:37.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:37.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.649+0000 7fd54dbff700 1 -- 172.21.15.70:0/4200365967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd548101590 msgr2=0x7fd548067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:37.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.649+0000 7fd54dbff700 1 --2- 172.21.15.70:0/4200365967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd548101590 0x7fd548067830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd538009990 tx=0x7fd538009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 -- 172.21.15.70:0/4200365967 shutdown_connections 2023-10-18T12:23:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 --2- 172.21.15.70:0/4200365967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd54810cf40 0x7fd54810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 --2- 172.21.15.70:0/4200365967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd548101590 0x7fd548067830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 --2- 172.21.15.70:0/4200365967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd5480681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 -- 172.21.15.70:0/4200365967 >> 172.21.15.70:0/4200365967 conn(0x7fd548076400 msgr2=0x7fd548076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 -- 172.21.15.70:0/4200365967 shutdown_connections 2023-10-18T12:23:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.650+0000 7fd54dbff700 1 -- 172.21.15.70:0/4200365967 wait complete. 2023-10-18T12:23:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 Processor -- start 2023-10-18T12:23:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 -- start start 2023-10-18T12:23:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd548198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:37.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548101590 0x7fd5481992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:37.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd54810cf40 0x7fd54819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:37.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54819dcc0 con 0x7fd54810cf40 2023-10-18T12:23:37.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54819de30 con 0x7fd548067d70 2023-10-18T12:23:37.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd54dbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54819dff0 con 0x7fd548101590 2023-10-18T12:23:37.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd5477fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd548198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:37.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd5477fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd548198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59670/0 (socket says 172.21.15.70:59670) 2023-10-18T12:23:37.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.651+0000 7fd5477fe700 1 -- 172.21.15.70:0/1165761605 learned_addr learned my addr 172.21.15.70:0/1165761605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:37.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd547fff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd54810cf40 0x7fd54819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:37.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd546ffd700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548101590 0x7fd5481992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:37.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd5477fe700 1 -- 172.21.15.70:0/1165761605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548101590 msgr2=0x7fd5481992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:37.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd5477fe700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548101590 0x7fd5481992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd5477fe700 1 -- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd54810cf40 msgr2=0x7fd54819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:37.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd5477fe700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd54810cf40 0x7fd54819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd5477fe700 1 -- 172.21.15.70:0/1165761605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380095f0 con 0x7fd548067d70 2023-10-18T12:23:37.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd546ffd700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548101590 0x7fd5481992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:23:37.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd5477fe700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd548198da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd538000c00 tx=0x7fd538005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:37.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53801d070 con 0x7fd548067d70 2023-10-18T12:23:37.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd538022470 con 0x7fd548067d70 2023-10-18T12:23:37.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54819e270 con 0x7fd548067d70 2023-10-18T12:23:37.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.652+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538017650 con 0x7fd548067d70 2023-10-18T12:23:37.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.653+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5481a5b10 con 0x7fd548067d70 2023-10-18T12:23:37.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.654+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5480ffc00 con 0x7fd548067d70 2023-10-18T12:23:37.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.655+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd5380225e0 con 0x7fd548067d70 2023-10-18T12:23:37.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.657+0000 7fd544ff9700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5300a1550 0x7fd5300a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:37.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.657+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fd5380ce090 con 0x7fd548067d70 2023-10-18T12:23:37.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.657+0000 7fd546ffd700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5300a1550 0x7fd5300a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:37.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.657+0000 7fd546ffd700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5300a1550 0x7fd5300a3a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd54819a2f0 tx=0x7fd53c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:37.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.659+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd538097ab0 con 0x7fd548067d70 2023-10-18T12:23:37.671 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:23:37 smithi152 podman[172919]: 2023-10-18T12:23:37.672 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:23:37 smithi152 bash[172919]: 8181e5187e610d373142749797992321edcd8d02cb23a69aa831d674ef2ed4c6 2023-10-18T12:23:37.672 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:23:37 smithi152 ceph-mds[172959]: -- 172.21.15.152:0/1150109140 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5586554011e0 con 0x5586553d7000 2023-10-18T12:23:37.672 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:23:37 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-k[172935]: starting mds.k at 2023-10-18T12:23:37.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.925+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd548067500 con 0x7fd548067d70 2023-10-18T12:23:37.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.925+0000 7fd544ff9700 1 -- 172.21.15.70:0/1165761605 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fd53809b0d0 con 0x7fd548067d70 2023-10-18T12:23:37.928 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:37.929 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-18T12:23:37.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5300a1550 msgr2=0x7fd5300a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:37.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5300a1550 0x7fd5300a3a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd54819a2f0 tx=0x7fd53c009260 comp rx=0 tx=0).stop 2023-10-18T12:23:37.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 msgr2=0x7fd548198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:37.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd548198da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd538000c00 tx=0x7fd538005740 comp rx=0 tx=0).stop 2023-10-18T12:23:37.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 shutdown_connections 2023-10-18T12:23:37.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd54810cf40 0x7fd54819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5300a1550 0x7fd5300a3a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548067d70 0x7fd548198da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 --2- 172.21.15.70:0/1165761605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548101590 0x7fd5481992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:37.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 >> 172.21.15.70:0/1165761605 conn(0x7fd548076400 msgr2=0x7fd54810de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:37.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 shutdown_connections 2023-10-18T12:23:37.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:37.928+0000 7fd54dbff700 1 -- 172.21.15.70:0/1165761605 wait complete. 2023-10-18T12:23:37.934 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 2 2023-10-18T12:23:38.171 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:23:37 smithi152 systemd[1]: Started Ceph mds.k for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: Deploying daemon mds.j on smithi070 2023-10-18T12:23:39.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1165761605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:39.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.152:6832/1678353915,v1:172.21.15.152:6833/1678353915] up:boot 2023-10-18T12:23:39.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: fsmap 2 up:standby 2023-10-18T12:23:39.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T12:23:39.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: Deploying daemon mds.j on smithi070 2023-10-18T12:23:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1165761605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:39.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.152:6832/1678353915,v1:172.21.15.152:6833/1678353915] up:boot 2023-10-18T12:23:39.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: fsmap 2 up:standby 2023-10-18T12:23:39.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T12:23:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:39.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: Deploying daemon mds.j on smithi070 2023-10-18T12:23:39.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1165761605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:39.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.152:6832/1678353915,v1:172.21.15.152:6833/1678353915] up:boot 2023-10-18T12:23:39.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: fsmap 2 up:standby 2023-10-18T12:23:39.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T12:23:39.855 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:23:39 smithi070 systemd[1]: Starting Ceph mds.j for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:39.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:40.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 -- 172.21.15.70:0/910471398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b01032b0 msgr2=0x7fc1b010f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/910471398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b01032b0 0x7fc1b010f7c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800b0a0 tx=0x7fc1a800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 -- 172.21.15.70:0/910471398 shutdown_connections 2023-10-18T12:23:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/910471398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b01032b0 0x7fc1b010f7c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/910471398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b0108900 0x7fc1b0108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/910471398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 0x7fc1b0102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 -- 172.21.15.70:0/910471398 >> 172.21.15.70:0/910471398 conn(0x7fc1b00fe3a0 msgr2=0x7fc1b01007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 -- 172.21.15.70:0/910471398 shutdown_connections 2023-10-18T12:23:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.175+0000 7fc1b65ff700 1 -- 172.21.15.70:0/910471398 wait complete. 2023-10-18T12:23:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.176+0000 7fc1b65ff700 1 Processor -- start 2023-10-18T12:23:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.176+0000 7fc1b65ff700 1 -- start start 2023-10-18T12:23:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b65ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 0x7fc1b019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b65ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b01032b0 0x7fc1b019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b65ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b0108900 0x7fc1b0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b65ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0199120 con 0x7fc1b0108900 2023-10-18T12:23:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b65ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0199290 con 0x7fc1b01032b0 2023-10-18T12:23:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b65ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0199450 con 0x7fc1b01028f0 2023-10-18T12:23:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b4b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b0108900 0x7fc1b0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1affff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 0x7fc1b019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b4b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b0108900 0x7fc1b0198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37128/0 (socket says 172.21.15.70:37128) 2023-10-18T12:23:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1b4b9c700 1 -- 172.21.15.70:0/273982927 learned_addr learned my addr 172.21.15.70:0/273982927 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.177+0000 7fc1af7fe700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b01032b0 0x7fc1b019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1affff700 1 -- 172.21.15.70:0/273982927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b01032b0 msgr2=0x7fc1b019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1affff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b01032b0 0x7fc1b019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1affff700 1 -- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b0108900 msgr2=0x7fc1b0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1affff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b0108900 0x7fc1b0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1affff700 1 -- 172.21.15.70:0/273982927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8009d00 con 0x7fc1b01028f0 2023-10-18T12:23:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1affff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 0x7fc1b019e990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9f0 tx=0x7fc1a000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000c9c0 con 0x7fc1b01028f0 2023-10-18T12:23:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b0199730 con 0x7fc1b01028f0 2023-10-18T12:23:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a0010430 con 0x7fc1b01028f0 2023-10-18T12:23:40.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0018680 con 0x7fc1b01028f0 2023-10-18T12:23:40.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.178+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b01a5980 con 0x7fc1b01028f0 2023-10-18T12:23:40.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.180+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b0104030 con 0x7fc1b01028f0 2023-10-18T12:23:40.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.181+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc1a0010860 con 0x7fc1b01028f0 2023-10-18T12:23:40.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.183+0000 7fc1ad7fa700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc1980a14d0 0x7fc1980a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:40.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.183+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fc1a0014070 con 0x7fc1b01028f0 2023-10-18T12:23:40.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.183+0000 7fc1af7fe700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc1980a14d0 0x7fc1980a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:40.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.184+0000 7fc1af7fe700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc1980a14d0 0x7fc1980a3990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0199c80 tx=0x7fc1a4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:40.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.187+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc1a0096470 con 0x7fc1b01028f0 2023-10-18T12:23:40.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.452+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1b0067530 con 0x7fc1b01028f0 2023-10-18T12:23:40.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.452+0000 7fc1ad7fa700 1 -- 172.21.15.70:0/273982927 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fc1a0099a90 con 0x7fc1b01028f0 2023-10-18T12:23:40.455 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:40.456 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T12:23:40.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc1980a14d0 msgr2=0x7fc1980a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:40.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc1980a14d0 0x7fc1980a3990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0199c80 tx=0x7fc1a4009260 comp rx=0 tx=0).stop 2023-10-18T12:23:40.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 msgr2=0x7fc1b019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:40.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 0x7fc1b019e990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9f0 tx=0x7fc1a000ed00 comp rx=0 tx=0).stop 2023-10-18T12:23:40.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 shutdown_connections 2023-10-18T12:23:40.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b0108900 0x7fc1b0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1b01028f0 0x7fc1b019e990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc1980a14d0 0x7fc1980a3990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 --2- 172.21.15.70:0/273982927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b01032b0 0x7fc1b019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:40.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 >> 172.21.15.70:0/273982927 conn(0x7fc1b00fe3a0 msgr2=0x7fc1b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:40.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 shutdown_connections 2023-10-18T12:23:40.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:40.455+0000 7fc1b65ff700 1 -- 172.21.15.70:0/273982927 wait complete. 2023-10-18T12:23:40.461 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 3 2023-10-18T12:23:40.605 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:23:40 smithi070 podman[156286]: 2023-10-18T12:23:40.605 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:23:40 smithi070 bash[156286]: dbdede8061c034a154ea49dee818a71e17d5ea7af617b5718cd7fbbf34d3ef4b 2023-10-18T12:23:40.606 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:23:40 smithi070 ceph-mds[156348]: -- 172.21.15.70:0/2225022519 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580e748b1e0 con 0x5580e7462c00 2023-10-18T12:23:40.606 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:23:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-j[156322]: starting mds.j at 2023-10-18T12:23:41.104 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:23:40 smithi070 systemd[1]: Started Ceph mds.j for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:41.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:41.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/273982927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:41.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:41.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:41.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:41.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:41.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/273982927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:41.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:41.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:41.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/273982927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:42.302 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:42 smithi070 ceph-mon[98639]: Deploying daemon mds.d on smithi070 2023-10-18T12:23:42.302 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:42 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.70:6834/104240782,v1:172.21.15.70:6835/104240782] up:boot 2023-10-18T12:23:42.302 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:42 smithi070 ceph-mon[98639]: fsmap 3 up:standby 2023-10-18T12:23:42.302 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-18T12:23:42.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:42 smithi152 ceph-mon[134767]: Deploying daemon mds.d on smithi070 2023-10-18T12:23:42.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:42 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.70:6834/104240782,v1:172.21.15.70:6835/104240782] up:boot 2023-10-18T12:23:42.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:42 smithi152 ceph-mon[134767]: fsmap 3 up:standby 2023-10-18T12:23:42.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-18T12:23:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:42 smithi186 ceph-mon[135253]: Deploying daemon mds.d on smithi070 2023-10-18T12:23:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:42 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.70:6834/104240782,v1:172.21.15.70:6835/104240782] up:boot 2023-10-18T12:23:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:42 smithi186 ceph-mon[135253]: fsmap 3 up:standby 2023-10-18T12:23:42.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-18T12:23:42.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:42.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.702+0000 7f9342bff700 1 -- 172.21.15.70:0/2955615956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 msgr2=0x7f933c10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:42.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.702+0000 7f9342bff700 1 --2- 172.21.15.70:0/2955615956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c10f350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f933400b0a0 tx=0x7f933400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.702+0000 7f9342bff700 1 -- 172.21.15.70:0/2955615956 shutdown_connections 2023-10-18T12:23:42.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.702+0000 7f9342bff700 1 --2- 172.21.15.70:0/2955615956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c10f350 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.702+0000 7f9342bff700 1 --2- 172.21.15.70:0/2955615956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 0x7f933c073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.703+0000 7f9342bff700 1 --2- 172.21.15.70:0/2955615956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f933c073540 0x7f933c10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.703+0000 7f9342bff700 1 -- 172.21.15.70:0/2955615956 >> 172.21.15.70:0/2955615956 conn(0x7f933c0fc170 msgr2=0x7f933c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:42.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.703+0000 7f9342bff700 1 -- 172.21.15.70:0/2955615956 shutdown_connections 2023-10-18T12:23:42.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.703+0000 7f9342bff700 1 -- 172.21.15.70:0/2955615956 wait complete. 2023-10-18T12:23:42.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.703+0000 7f9342bff700 1 Processor -- start 2023-10-18T12:23:42.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 -- start start 2023-10-18T12:23:42.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 0x7f933c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:42.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f933c073540 0x7f933c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:42.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:42.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c19ddb0 con 0x7f933c10cee0 2023-10-18T12:23:42.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c19df20 con 0x7f933c072c20 2023-10-18T12:23:42.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f9342bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c19e0e0 con 0x7f933c073540 2023-10-18T12:23:42.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f934119c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:42.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f934099b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 0x7f933c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:42.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f934119c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37172/0 (socket says 172.21.15.70:37172) 2023-10-18T12:23:42.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f934119c700 1 -- 172.21.15.70:0/3020441353 learned_addr learned my addr 172.21.15.70:0/3020441353 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:42.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.704+0000 7f933bfff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f933c073540 0x7f933c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:42.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934099b700 1 -- 172.21.15.70:0/3020441353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f933c073540 msgr2=0x7f933c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:42.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934099b700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f933c073540 0x7f933c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934099b700 1 -- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 msgr2=0x7f933c19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:42.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934099b700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934099b700 1 -- 172.21.15.70:0/3020441353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009d00 con 0x7f933c072c20 2023-10-18T12:23:42.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934119c700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:23:42.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f934099b700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 0x7f933c198ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f933000e9f0 tx=0x7f933000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:42.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933000c9c0 con 0x7f933c072c20 2023-10-18T12:23:42.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9330010430 con 0x7f933c072c20 2023-10-18T12:23:42.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c19e3c0 con 0x7f933c072c20 2023-10-18T12:23:42.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330018680 con 0x7f933c072c20 2023-10-18T12:23:42.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.705+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c1a5c00 con 0x7f933c072c20 2023-10-18T12:23:42.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.706+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c1080f0 con 0x7f933c072c20 2023-10-18T12:23:42.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.708+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9330010860 con 0x7f933c072c20 2023-10-18T12:23:42.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.710+0000 7f9339ffb700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f932c0a14d0 0x7f932c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:42.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.710+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f9330014070 con 0x7f933c072c20 2023-10-18T12:23:42.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.710+0000 7f933bfff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f932c0a14d0 0x7f932c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:42.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.710+0000 7f933bfff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f932c0a14d0 0x7f932c0a3990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f933c19a3e0 tx=0x7f9328009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:42.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.712+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f93300964a0 con 0x7f933c072c20 2023-10-18T12:23:42.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.973+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933c0742d0 con 0x7f933c072c20 2023-10-18T12:23:42.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.973+0000 7f9339ffb700 1 -- 172.21.15.70:0/3020441353 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7f9330099ac0 con 0x7f933c072c20 2023-10-18T12:23:42.977 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:42.977 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T12:23:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f932c0a14d0 msgr2=0x7f932c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f932c0a14d0 0x7f932c0a3990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f933c19a3e0 tx=0x7f9328009260 comp rx=0 tx=0).stop 2023-10-18T12:23:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 msgr2=0x7f933c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 0x7f933c198ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f933000e9f0 tx=0x7f933000ed00 comp rx=0 tx=0).stop 2023-10-18T12:23:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 shutdown_connections 2023-10-18T12:23:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f933c10cee0 0x7f933c19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f932c0a14d0 0x7f932c0a3990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f933c072c20 0x7f933c198ee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 --2- 172.21.15.70:0/3020441353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f933c073540 0x7f933c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 >> 172.21.15.70:0/3020441353 conn(0x7f933c0fc170 msgr2=0x7f933c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 shutdown_connections 2023-10-18T12:23:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:42.976+0000 7f9342bff700 1 -- 172.21.15.70:0/3020441353 wait complete. 2023-10-18T12:23:42.982 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 4 2023-10-18T12:23:43.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:43 smithi070 ceph-mon[98639]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:43.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:43 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3020441353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:43.356 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:23:42 smithi070 systemd[1]: Starting Ceph mds.d for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:43.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:43 smithi152 ceph-mon[134767]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:43.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:43 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3020441353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:43.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:43 smithi186 ceph-mon[135253]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:43.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:43 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3020441353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:43.654 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:23:43 smithi070 podman[156700]: 2023-10-18T12:23:43.654 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:23:43 smithi070 bash[156700]: 3e6c508fefb982d49106a44056007d0bc924ecf369f32bcb48ab285538964f6d 2023-10-18T12:23:43.654 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:23:43 smithi070 ceph-mds[156740]: -- 172.21.15.70:0/3362300663 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564bbae471e0 con 0x564bbae1ec00 2023-10-18T12:23:43.654 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:23:43 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-d[156716]: starting mds.d at 2023-10-18T12:23:44.104 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:23:43 smithi070 systemd[1]: Started Ceph mds.d for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:45.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:45.017 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:45.018 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.018 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.018 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.018 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:45.019 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:45.019 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:45.019 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: Deploying daemon mds.e on smithi152 2023-10-18T12:23:45.019 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.70:6836/3255312208,v1:172.21.15.70:6837/3255312208] up:boot 2023-10-18T12:23:45.020 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: fsmap 4 up:standby 2023-10-18T12:23:45.020 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-18T12:23:45.075 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:45.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:45.076 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:45.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:45.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: Deploying daemon mds.e on smithi152 2023-10-18T12:23:45.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.70:6836/3255312208,v1:172.21.15.70:6837/3255312208] up:boot 2023-10-18T12:23:45.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: fsmap 4 up:standby 2023-10-18T12:23:45.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-18T12:23:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 -- 172.21.15.70:0/1199366697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c1066b0 msgr2=0x7f546c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 --2- 172.21.15.70:0/1199366697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c1066b0 0x7f546c106a90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f545c009990 tx=0x7f545c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 -- 172.21.15.70:0/1199366697 shutdown_connections 2023-10-18T12:23:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 --2- 172.21.15.70:0/1199366697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c101060 0x7f546c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 --2- 172.21.15.70:0/1199366697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c1066b0 0x7f546c106a90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 --2- 172.21.15.70:0/1199366697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c1006a0 0x7f546c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 -- 172.21.15.70:0/1199366697 >> 172.21.15.70:0/1199366697 conn(0x7f546c0fc1d0 msgr2=0x7f546c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 -- 172.21.15.70:0/1199366697 shutdown_connections 2023-10-18T12:23:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.216+0000 7f54727ff700 1 -- 172.21.15.70:0/1199366697 wait complete. 2023-10-18T12:23:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:45.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:45.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:45.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:45.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: Deploying daemon mds.e on smithi152 2023-10-18T12:23:45.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.70:6836/3255312208,v1:172.21.15.70:6837/3255312208] up:boot 2023-10-18T12:23:45.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: fsmap 4 up:standby 2023-10-18T12:23:45.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-18T12:23:45.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 Processor -- start 2023-10-18T12:23:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 -- start start 2023-10-18T12:23:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c1006a0 0x7f546c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 0x7f546c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c1066b0 0x7f546c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c196fe0 con 0x7f546c101060 2023-10-18T12:23:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c197150 con 0x7f546c1006a0 2023-10-18T12:23:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f54727ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c197310 con 0x7f546c1066b0 2023-10-18T12:23:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f546b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 0x7f546c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f546b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 0x7f546c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37198/0 (socket says 172.21.15.70:37198) 2023-10-18T12:23:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f546b7fe700 1 -- 172.21.15.70:0/392126145 learned_addr learned my addr 172.21.15.70:0/392126145 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.217+0000 7f546bfff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c1006a0 0x7f546c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f5470d9c700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c1066b0 0x7f546c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546b7fe700 1 -- 172.21.15.70:0/392126145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c1066b0 msgr2=0x7f546c196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546b7fe700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c1066b0 0x7f546c196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546b7fe700 1 -- 172.21.15.70:0/392126145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c1006a0 msgr2=0x7f546c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:45.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546b7fe700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c1006a0 0x7f546c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546b7fe700 1 -- 172.21.15.70:0/392126145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5464009c40 con 0x7f546c101060 2023-10-18T12:23:45.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546bfff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c1006a0 0x7f546c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:23:45.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f546b7fe700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 0x7f546c19cd90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f546000d950 tx=0x7f546000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:45.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460009930 con 0x7f546c101060 2023-10-18T12:23:45.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5460010460 con 0x7f546c101060 2023-10-18T12:23:45.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c0095f0 con 0x7f546c101060 2023-10-18T12:23:45.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.218+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000f620 con 0x7f546c101060 2023-10-18T12:23:45.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.219+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c1a3660 con 0x7f546c101060 2023-10-18T12:23:45.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.219+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c108ca0 con 0x7f546c101060 2023-10-18T12:23:45.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.221+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f54600108d0 con 0x7f546c101060 2023-10-18T12:23:45.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.223+0000 7f54697fa700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54540a14d0 0x7f54540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:45.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.223+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f54600cc7f0 con 0x7f546c101060 2023-10-18T12:23:45.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.223+0000 7f546bfff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54540a14d0 0x7f54540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.223+0000 7f546bfff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54540a14d0 0x7f54540a3990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b2e0 tx=0x7f545c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.227+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5460096180 con 0x7f546c101060 2023-10-18T12:23:45.351 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:23:45 smithi152 systemd[1]: Starting Ceph mds.e for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:45.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.474+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f546c101de0 con 0x7f546c101060 2023-10-18T12:23:45.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.474+0000 7f54697fa700 1 -- 172.21.15.70:0/392126145 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7f5460020e90 con 0x7f546c101060 2023-10-18T12:23:45.476 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:45.477 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T12:23:45.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54540a14d0 msgr2=0x7f54540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:45.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54540a14d0 0x7f54540a3990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b2e0 tx=0x7f545c005fd0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 msgr2=0x7f546c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 0x7f546c19cd90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f546000d950 tx=0x7f546000dc60 comp rx=0 tx=0).stop 2023-10-18T12:23:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 shutdown_connections 2023-10-18T12:23:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c1066b0 0x7f546c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54540a14d0 0x7f54540a3990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f546c1006a0 0x7f546c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.476+0000 7f54727ff700 1 --2- 172.21.15.70:0/392126145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c101060 0x7f546c19cd90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.477+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 >> 172.21.15.70:0/392126145 conn(0x7f546c0fc1d0 msgr2=0x7f546c0fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.477+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 shutdown_connections 2023-10-18T12:23:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:45.477+0000 7f54727ff700 1 -- 172.21.15.70:0/392126145 wait complete. 2023-10-18T12:23:45.481 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 5 2023-10-18T12:23:45.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:45 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/392126145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:45.903 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:23:45 smithi152 podman[173255]: 2023-10-18T12:23:45.903 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:45 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/392126145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:46.171 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:23:45 smithi152 ceph-mds[173295]: -- 172.21.15.152:0/1734629271 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55657bc391e0 con 0x55657bc0e800 2023-10-18T12:23:46.172 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:23:45 smithi152 bash[173255]: 9cf2b8a6022c1341c00f6ddad4ff8c39bf92dc55738bf0be0f831fc8028be37e 2023-10-18T12:23:46.172 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:23:45 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-e[173271]: starting mds.e at 2023-10-18T12:23:46.172 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:23:45 smithi152 systemd[1]: Started Ceph mds.e for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:46.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:45 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/392126145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:47.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:47.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.323 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.323 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:47.324 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:47.324 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:47.324 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: Deploying daemon mds.f on smithi186 2023-10-18T12:23:47.324 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.152:6834/2963573440,v1:172.21.15.152:6835/2963573440] up:boot 2023-10-18T12:23:47.324 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: fsmap 5 up:standby 2023-10-18T12:23:47.325 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T12:23:47.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:47.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:47.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:47.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:47.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: Deploying daemon mds.f on smithi186 2023-10-18T12:23:47.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.152:6834/2963573440,v1:172.21.15.152:6835/2963573440] up:boot 2023-10-18T12:23:47.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: fsmap 5 up:standby 2023-10-18T12:23:47.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T12:23:47.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:47.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:47.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:47.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:47.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:47.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: Deploying daemon mds.f on smithi186 2023-10-18T12:23:47.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.152:6834/2963573440,v1:172.21.15.152:6835/2963573440] up:boot 2023-10-18T12:23:47.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: fsmap 5 up:standby 2023-10-18T12:23:47.425 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T12:23:47.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:47.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 -- 172.21.15.70:0/2650145654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d4106fe0 msgr2=0x7f25d4067870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:47.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2650145654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d4106fe0 0x7f25d4067870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009990 tx=0x7f25bc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 -- 172.21.15.70:0/2650145654 shutdown_connections 2023-10-18T12:23:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2650145654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25d4067fa0 0x7f25d410d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2650145654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d41066c0 0x7f25d4106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2650145654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d4106fe0 0x7f25d4067870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 -- 172.21.15.70:0/2650145654 >> 172.21.15.70:0/2650145654 conn(0x7f25d4076400 msgr2=0x7f25d4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:47.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 -- 172.21.15.70:0/2650145654 shutdown_connections 2023-10-18T12:23:47.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.707+0000 7f25da3ff700 1 -- 172.21.15.70:0/2650145654 wait complete. 2023-10-18T12:23:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.708+0000 7f25da3ff700 1 Processor -- start 2023-10-18T12:23:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.708+0000 7f25da3ff700 1 -- start start 2023-10-18T12:23:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25da3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d4067fa0 0x7f25d4198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25da3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 0x7f25d4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25da3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25d4106fe0 0x7f25d419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25da3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d419dce0 con 0x7f25d41066c0 2023-10-18T12:23:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25da3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d419de50 con 0x7f25d4106fe0 2023-10-18T12:23:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25da3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d419e010 con 0x7f25d4067fa0 2023-10-18T12:23:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 0x7f25d4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25d3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d4067fa0 0x7f25d4198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 0x7f25d4199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37224/0 (socket says 172.21.15.70:37224) 2023-10-18T12:23:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 -- 172.21.15.70:0/2561342668 learned_addr learned my addr 172.21.15.70:0/2561342668 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25d899c700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25d4106fe0 0x7f25d419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 -- 172.21.15.70:0/2561342668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d4067fa0 msgr2=0x7f25d4198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d4067fa0 0x7f25d4198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 -- 172.21.15.70:0/2561342668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25d4106fe0 msgr2=0x7f25d419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25d4106fe0 0x7f25d419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25cbfff700 1 -- 172.21.15.70:0/2561342668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0095f0 con 0x7f25d41066c0 2023-10-18T12:23:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.709+0000 7f25d3fff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d4067fa0 0x7f25d4198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:23:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.710+0000 7f25cbfff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 0x7f25d4199300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25bc000c00 tx=0x7f25bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.710+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc01d070 con 0x7f25d41066c0 2023-10-18T12:23:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.710+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25bc022470 con 0x7f25d41066c0 2023-10-18T12:23:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.710+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d419e290 con 0x7f25d41066c0 2023-10-18T12:23:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.710+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc017650 con 0x7f25d41066c0 2023-10-18T12:23:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.710+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d41a5b30 con 0x7f25d41066c0 2023-10-18T12:23:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.711+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d4101de0 con 0x7f25d41066c0 2023-10-18T12:23:47.719 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:23:47 smithi186 systemd[1]: Starting Ceph mds.f for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.713+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f25bc00fad0 con 0x7f25d41066c0 2023-10-18T12:23:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.715+0000 7f25d1ffb700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25c00a1550 0x7f25c00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.715+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f25bc0ce420 con 0x7f25d41066c0 2023-10-18T12:23:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.715+0000 7f25d3fff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25c00a1550 0x7f25c00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.716+0000 7f25d3fff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25c00a1550 0x7f25c00a3a10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009fd0 tx=0x7f25c4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.717+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25bc097e40 con 0x7f25d41066c0 2023-10-18T12:23:47.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.965+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25d4067530 con 0x7f25d41066c0 2023-10-18T12:23:47.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.965+0000 7f25d1ffb700 1 -- 172.21.15.70:0/2561342668 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f25bc09b460 con 0x7f25d41066c0 2023-10-18T12:23:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:47.968 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-18T12:23:47.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25c00a1550 msgr2=0x7f25c00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:47.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25c00a1550 0x7f25c00a3a10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009fd0 tx=0x7f25c4009260 comp rx=0 tx=0).stop 2023-10-18T12:23:47.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 msgr2=0x7f25d4199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:47.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 0x7f25d4199300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25bc000c00 tx=0x7f25bc005740 comp rx=0 tx=0).stop 2023-10-18T12:23:47.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 shutdown_connections 2023-10-18T12:23:47.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25c00a1550 0x7f25c00a3a10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25d4106fe0 0x7f25d419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25d4067fa0 0x7f25d4198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 --2- 172.21.15.70:0/2561342668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d41066c0 0x7f25d4199300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:47.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 >> 172.21.15.70:0/2561342668 conn(0x7f25d4076400 msgr2=0x7f25d410b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:47.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 shutdown_connections 2023-10-18T12:23:47.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:47.968+0000 7f25da3ff700 1 -- 172.21.15.70:0/2561342668 wait complete. 2023-10-18T12:23:47.973 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 6 2023-10-18T12:23:48.111 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:47 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2561342668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:48.111 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:23:47 smithi186 podman[173532]: 2023-10-18T12:23:48.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:47 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2561342668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:48.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:47 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2561342668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:48.468 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:23:48 smithi186 ceph-mds[173572]: -- 172.21.15.186:0/1489270324 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562d32a8d1e0 con 0x562d32a62800 2023-10-18T12:23:48.469 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:23:48 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-f[173548]: starting mds.f at 2023-10-18T12:23:48.469 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:23:48 smithi186 bash[173532]: bdce724402a93def7dcff33977a1caa7f089952c0f8fae128d972c8708498f89 2023-10-18T12:23:48.469 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:23:48 smithi186 systemd[1]: Started Ceph mds.f for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:49.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:49.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:49.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.186:6834/3580803773,v1:172.21.15.186:6835/3580803773] up:boot 2023-10-18T12:23:49.456 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: fsmap 6 up:standby 2023-10-18T12:23:49.456 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T12:23:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:49.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.186:6834/3580803773,v1:172.21.15.186:6835/3580803773] up:boot 2023-10-18T12:23:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: fsmap 6 up:standby 2023-10-18T12:23:49.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T12:23:49.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:49.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:49.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:49.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:49.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:49.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.186:6834/3580803773,v1:172.21.15.186:6835/3580803773] up:boot 2023-10-18T12:23:49.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: fsmap 6 up:standby 2023-10-18T12:23:49.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T12:23:49.718 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:23:49 smithi186 systemd[1]: Starting Ceph mds.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:49.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 -- 172.21.15.70:0/214542976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 msgr2=0x7f6550106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 --2- 172.21.15.70:0/214542976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550106a90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6540009990 tx=0x7f6540009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 -- 172.21.15.70:0/214542976 shutdown_connections 2023-10-18T12:23:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 --2- 172.21.15.70:0/214542976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6550101060 0x7f655010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 --2- 172.21.15.70:0/214542976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550106a90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 --2- 172.21.15.70:0/214542976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65501006a0 0x7f6550100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 -- 172.21.15.70:0/214542976 >> 172.21.15.70:0/214542976 conn(0x7f65500fc1d0 msgr2=0x7f65500fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 -- 172.21.15.70:0/214542976 shutdown_connections 2023-10-18T12:23:50.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.199+0000 7f6554fff700 1 -- 172.21.15.70:0/214542976 wait complete. 2023-10-18T12:23:50.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.200+0000 7f6554fff700 1 Processor -- start 2023-10-18T12:23:50.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.200+0000 7f6554fff700 1 -- start start 2023-10-18T12:23:50.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6554fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65501006a0 0x7f655019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:50.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6554fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6550101060 0x7f655019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:50.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6554fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:50.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6554fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550196fe0 con 0x7f65501066b0 2023-10-18T12:23:50.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6554fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550197150 con 0x7f65501006a0 2023-10-18T12:23:50.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6554fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550197310 con 0x7f6550101060 2023-10-18T12:23:50.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:50.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65501006a0 0x7f655019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:50.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45470/0 (socket says 172.21.15.70:45470) 2023-10-18T12:23:50.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65501006a0 0x7f655019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49858/0 (socket says 172.21.15.70:49858) 2023-10-18T12:23:50.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f6547fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6550101060 0x7f655019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:50.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654e59c700 1 -- 172.21.15.70:0/208172964 learned_addr learned my addr 172.21.15.70:0/208172964 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:50.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 -- 172.21.15.70:0/208172964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6550101060 msgr2=0x7f655019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:50.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6550101060 0x7f655019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 -- 172.21.15.70:0/208172964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65501006a0 msgr2=0x7f655019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:50.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65501006a0 0x7f655019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.201+0000 7f654ed9d700 1 -- 172.21.15.70:0/208172964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538009520 con 0x7f65501066b0 2023-10-18T12:23:50.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.202+0000 7f654ed9d700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550196980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f654800efc0 tx=0x7f654800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:50.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.202+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548018070 con 0x7f65501066b0 2023-10-18T12:23:50.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.202+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65400095f0 con 0x7f65501066b0 2023-10-18T12:23:50.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.202+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f654800f040 con 0x7f65501066b0 2023-10-18T12:23:50.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.202+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800cd40 con 0x7f65501066b0 2023-10-18T12:23:50.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.202+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65501a3660 con 0x7f65501066b0 2023-10-18T12:23:50.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.203+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6550108ca0 con 0x7f65501066b0 2023-10-18T12:23:50.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.205+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6548003a00 con 0x7f65501066b0 2023-10-18T12:23:50.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.206+0000 7f65477fe700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f653c0a14d0 0x7f653c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:50.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.206+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f65480cc560 con 0x7f65501066b0 2023-10-18T12:23:50.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.206+0000 7f654e59c700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f653c0a14d0 0x7f653c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:50.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.207+0000 7f654e59c700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f653c0a14d0 0x7f653c0a3990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6540006010 tx=0x7f654000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:50.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.211+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6548095f80 con 0x7f65501066b0 2023-10-18T12:23:50.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:50 smithi186 ceph-mon[135253]: Deploying daemon mds.c on smithi186 2023-10-18T12:23:50.278 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:23:50 smithi186 podman[173874]: 2023-10-18T12:23:50.278 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:23:50 smithi186 ceph-mds[173915]: -- 172.21.15.186:0/1636218716 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640a7ee51e0 con 0x5640a7ebb000 2023-10-18T12:23:50.279 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:23:50 smithi186 bash[173874]: d23577b765d5e7c575bd575271e80938dfffa5342f2910fa9ed9556ad7c3b17f 2023-10-18T12:23:50.279 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:23:50 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-c[173890]: starting mds.c at 2023-10-18T12:23:50.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:50 smithi070 ceph-mon[98639]: Deploying daemon mds.c on smithi186 2023-10-18T12:23:50.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.462+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6550101de0 con 0x7f65501066b0 2023-10-18T12:23:50.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.463+0000 7f65477fe700 1 -- 172.21.15.70:0/208172964 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f65480995a0 con 0x7f65501066b0 2023-10-18T12:23:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:50.465 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-18T12:23:50.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f653c0a14d0 msgr2=0x7f653c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:50.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f653c0a14d0 0x7f653c0a3990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6540006010 tx=0x7f654000b260 comp rx=0 tx=0).stop 2023-10-18T12:23:50.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 msgr2=0x7f6550196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:50.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550196980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f654800efc0 tx=0x7f654800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 shutdown_connections 2023-10-18T12:23:50.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65501066b0 0x7f6550196980 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f653c0a14d0 0x7f653c0a3990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65501006a0 0x7f655019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 --2- 172.21.15.70:0/208172964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6550101060 0x7f655019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:50.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 >> 172.21.15.70:0/208172964 conn(0x7f65500fc1d0 msgr2=0x7f65500fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:50.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.465+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 shutdown_connections 2023-10-18T12:23:50.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:50.466+0000 7f6554fff700 1 -- 172.21.15.70:0/208172964 wait complete. 2023-10-18T12:23:50.470 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 7 2023-10-18T12:23:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:50 smithi152 ceph-mon[134767]: Deploying daemon mds.c on smithi186 2023-10-18T12:23:50.718 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:23:50 smithi186 systemd[1]: Started Ceph mds.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:51.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:51.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:51.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:51.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:51.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:51 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/208172964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:51.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:51.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:51.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:51 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/208172964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:51.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:51.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:51.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:51 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/208172964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:51.922 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:23:51 smithi152 systemd[1]: Starting Ceph mds.h for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:52.421 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:23:52 smithi152 podman[173598]: 2023-10-18T12:23:52.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:52 smithi152 ceph-mon[134767]: Deploying daemon mds.h on smithi152 2023-10-18T12:23:52.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:52 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.186:6836/569012458,v1:172.21.15.186:6837/569012458] up:boot 2023-10-18T12:23:52.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:52 smithi152 ceph-mon[134767]: fsmap 7 up:standby 2023-10-18T12:23:52.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T12:23:52.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:52 smithi186 ceph-mon[135253]: Deploying daemon mds.h on smithi152 2023-10-18T12:23:52.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:52 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.186:6836/569012458,v1:172.21.15.186:6837/569012458] up:boot 2023-10-18T12:23:52.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:52 smithi186 ceph-mon[135253]: fsmap 7 up:standby 2023-10-18T12:23:52.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T12:23:52.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:52.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:52 smithi070 ceph-mon[98639]: Deploying daemon mds.h on smithi152 2023-10-18T12:23:52.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:52 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.186:6836/569012458,v1:172.21.15.186:6837/569012458] up:boot 2023-10-18T12:23:52.561 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:52 smithi070 ceph-mon[98639]: fsmap 7 up:standby 2023-10-18T12:23:52.561 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T12:23:52.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 -- 172.21.15.70:0/1819676553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881066b0 msgr2=0x7f3088106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:52.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 --2- 172.21.15.70:0/1819676553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881066b0 0x7f3088106a90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3078009990 tx=0x7f3078009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 -- 172.21.15.70:0/1819676553 shutdown_connections 2023-10-18T12:23:52.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 --2- 172.21.15.70:0/1819676553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3088101060 0x7f308810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 --2- 172.21.15.70:0/1819676553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881066b0 0x7f3088106a90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 --2- 172.21.15.70:0/1819676553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30881006a0 0x7f3088100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 -- 172.21.15.70:0/1819676553 >> 172.21.15.70:0/1819676553 conn(0x7f30880fc1d0 msgr2=0x7f30880fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 -- 172.21.15.70:0/1819676553 shutdown_connections 2023-10-18T12:23:52.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.701+0000 7f308d1ff700 1 -- 172.21.15.70:0/1819676553 wait complete. 2023-10-18T12:23:52.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 Processor -- start 2023-10-18T12:23:52.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 -- start start 2023-10-18T12:23:52.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 0x7f308819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:52.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3088101060 0x7f308819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:52.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30881066b0 0x7f3088196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:52.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3088196fe0 con 0x7f30881006a0 2023-10-18T12:23:52.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3088197150 con 0x7f3088101060 2023-10-18T12:23:52.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.702+0000 7f308d1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3088197310 con 0x7f30881066b0 2023-10-18T12:23:52.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 0x7f308819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:52.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f308759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30881066b0 0x7f3088196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:52.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 0x7f308819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45482/0 (socket says 172.21.15.70:45482) 2023-10-18T12:23:52.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 -- 172.21.15.70:0/3407370487 learned_addr learned my addr 172.21.15.70:0/3407370487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:52.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f308659c700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3088101060 0x7f308819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:52.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 -- 172.21.15.70:0/3407370487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30881066b0 msgr2=0x7f3088196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:52.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30881066b0 0x7f3088196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 -- 172.21.15.70:0/3407370487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3088101060 msgr2=0x7f308819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:52.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3088101060 0x7f308819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 -- 172.21.15.70:0/3407370487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780095f0 con 0x7f30881006a0 2023-10-18T12:23:52.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f3086d9d700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 0x7f308819c800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3078000c00 tx=0x7f3078005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:52.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307801d070 con 0x7f30881006a0 2023-10-18T12:23:52.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3078022470 con 0x7f30881006a0 2023-10-18T12:23:52.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3088197590 con 0x7f30881006a0 2023-10-18T12:23:52.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078017650 con 0x7f30881006a0 2023-10-18T12:23:52.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.703+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30881a37f0 con 0x7f30881006a0 2023-10-18T12:23:52.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.704+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3088108ca0 con 0x7f30881006a0 2023-10-18T12:23:52.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.706+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f30780225e0 con 0x7f30881006a0 2023-10-18T12:23:52.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.707+0000 7f306ffff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30700a14d0 0x7f30700a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:52.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.708+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f30780cdc90 con 0x7f30881006a0 2023-10-18T12:23:52.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.708+0000 7f308659c700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30700a14d0 0x7f30700a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:52.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.708+0000 7f308659c700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30700a14d0 0x7f30700a3990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3088197af0 tx=0x7f307c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:52.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.711+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f30780976b0 con 0x7f30881006a0 2023-10-18T12:23:52.922 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:23:52 smithi152 ceph-mds[173638]: -- 172.21.15.152:0/1783049925 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c367a9b1e0 con 0x55c367a70800 2023-10-18T12:23:52.922 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:23:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-h[173614]: starting mds.h at 2023-10-18T12:23:52.922 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:23:52 smithi152 bash[173598]: f62118d4fa447fcd17214a9ae939767bb9258f2500c465131cb0bde2efded214 2023-10-18T12:23:52.922 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:23:52 smithi152 systemd[1]: Started Ceph mds.h for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:52.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.955+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3088197df0 con 0x7f30881006a0 2023-10-18T12:23:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.955+0000 7f306ffff700 1 -- 172.21.15.70:0/3407370487 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f307809acd0 con 0x7f30881006a0 2023-10-18T12:23:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:52.959 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-18T12:23:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.957+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30700a14d0 msgr2=0x7f30700a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.957+0000 7f308d1ff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30700a14d0 0x7f30700a3990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3088197af0 tx=0x7f307c009260 comp rx=0 tx=0).stop 2023-10-18T12:23:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.957+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 msgr2=0x7f308819c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.957+0000 7f308d1ff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 0x7f308819c800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3078000c00 tx=0x7f3078005740 comp rx=0 tx=0).stop 2023-10-18T12:23:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 shutdown_connections 2023-10-18T12:23:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30881066b0 0x7f3088196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30881006a0 0x7f308819c800 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30700a14d0 0x7f30700a3990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 --2- 172.21.15.70:0/3407370487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3088101060 0x7f308819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:52.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 >> 172.21.15.70:0/3407370487 conn(0x7f30880fc1d0 msgr2=0x7f30880fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:52.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 shutdown_connections 2023-10-18T12:23:52.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:52.958+0000 7f308d1ff700 1 -- 172.21.15.70:0/3407370487 wait complete. 2023-10-18T12:23:52.963 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-10-18T12:23:53.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:53.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:53.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:53.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:53.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:53 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3407370487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:53.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:53.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:53.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:53.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:53 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3407370487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:53.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:53.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:53.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:53.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:53 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3407370487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:54.207 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:23:54 smithi186 systemd[1]: Starting Ceph mds.i for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:54.459 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:23:54 smithi186 podman[174217]: 2023-10-18T12:23:54.460 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-mon[135253]: Deploying daemon mds.i on smithi186 2023-10-18T12:23:54.460 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.152:6836/1543432937,v1:172.21.15.152:6837/1543432937] up:boot 2023-10-18T12:23:54.460 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-mon[135253]: fsmap 8 up:standby 2023-10-18T12:23:54.460 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T12:23:54.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-mon[135253]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:54 smithi070 ceph-mon[98639]: Deploying daemon mds.i on smithi186 2023-10-18T12:23:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:54 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.152:6836/1543432937,v1:172.21.15.152:6837/1543432937] up:boot 2023-10-18T12:23:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:54 smithi070 ceph-mon[98639]: fsmap 8 up:standby 2023-10-18T12:23:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:54 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T12:23:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:54 smithi070 ceph-mon[98639]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:54.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:54 smithi152 ceph-mon[134767]: Deploying daemon mds.i on smithi186 2023-10-18T12:23:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:54 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.152:6836/1543432937,v1:172.21.15.152:6837/1543432937] up:boot 2023-10-18T12:23:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:54 smithi152 ceph-mon[134767]: fsmap 8 up:standby 2023-10-18T12:23:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:54 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T12:23:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:54 smithi152 ceph-mon[134767]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:54.968 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-mds[174257]: -- 172.21.15.186:0/2759556875 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55619a8db1e0 con 0x55619a8b0800 2023-10-18T12:23:54.969 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:23:54 smithi186 bash[174217]: e62d2df361d542eae7485d9d35883b5d91785e2fd334de738d2cd61671b202dc 2023-10-18T12:23:54.969 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:23:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-i[174233]: starting mds.i at 2023-10-18T12:23:54.969 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:23:54 smithi186 systemd[1]: Started Ceph mds.i for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:54.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.183+0000 7f5831fff700 1 -- 172.21.15.70:0/3734746447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c073540 msgr2=0x7f582c10c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.183+0000 7f5831fff700 1 --2- 172.21.15.70:0/3734746447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c073540 0x7f582c10c9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5820009990 tx=0x7f5820009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 -- 172.21.15.70:0/3734746447 shutdown_connections 2023-10-18T12:23:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 --2- 172.21.15.70:0/3734746447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c10cee0 0x7f582c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 --2- 172.21.15.70:0/3734746447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 0x7f582c073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 --2- 172.21.15.70:0/3734746447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c073540 0x7f582c10c9a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 -- 172.21.15.70:0/3734746447 >> 172.21.15.70:0/3734746447 conn(0x7f582c0fc170 msgr2=0x7f582c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 -- 172.21.15.70:0/3734746447 shutdown_connections 2023-10-18T12:23:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.184+0000 7f5831fff700 1 -- 172.21.15.70:0/3734746447 wait complete. 2023-10-18T12:23:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 Processor -- start 2023-10-18T12:23:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 -- start start 2023-10-18T12:23:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 0x7f582c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c073540 0x7f582c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c10cee0 0x7f582c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:55.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19dcf0 con 0x7f582c10cee0 2023-10-18T12:23:55.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19de60 con 0x7f582c073540 2023-10-18T12:23:55.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f5831fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19e020 con 0x7f582c072c20 2023-10-18T12:23:55.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f582affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c073540 0x7f582c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:55.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f582bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c10cee0 0x7f582c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f582b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 0x7f582c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f582affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c073540 0x7f582c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49900/0 (socket says 172.21.15.70:49900) 2023-10-18T12:23:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.185+0000 7f582affd700 1 -- 172.21.15.70:0/2390049080 learned_addr learned my addr 172.21.15.70:0/2390049080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f582b7fe700 1 -- 172.21.15.70:0/2390049080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c073540 msgr2=0x7f582c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f582b7fe700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c073540 0x7f582c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f582b7fe700 1 -- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c10cee0 msgr2=0x7f582c19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f582b7fe700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c10cee0 0x7f582c19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f582b7fe700 1 -- 172.21.15.70:0/2390049080 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200095f0 con 0x7f582c072c20 2023-10-18T12:23:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f582b7fe700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 0x7f582c198dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f581c00e9f0 tx=0x7f581c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c004220 con 0x7f582c072c20 2023-10-18T12:23:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f581c004cf0 con 0x7f582c072c20 2023-10-18T12:23:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c19e300 con 0x7f582c072c20 2023-10-18T12:23:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00ce70 con 0x7f582c072c20 2023-10-18T12:23:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.186+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c1a5b40 con 0x7f582c072c20 2023-10-18T12:23:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.187+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c1080f0 con 0x7f582c072c20 2023-10-18T12:23:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.189+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f581c004750 con 0x7f582c072c20 2023-10-18T12:23:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.190+0000 7f5828ff9700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.191+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f581c014070 con 0x7f582c072c20 2023-10-18T12:23:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.191+0000 7f582affd700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.191+0000 7f582affd700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f582c19a320 tx=0x7f582000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.193+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f581c096620 con 0x7f582c072c20 2023-10-18T12:23:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.460+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f582c0742d0 con 0x7f582c072c20 2023-10-18T12:23:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.461+0000 7f5828ff9700 1 -- 172.21.15.70:0/2390049080 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f581c004e60 con 0x7f582c072c20 2023-10-18T12:23:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:55.464 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-18T12:23:55.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.463+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 msgr2=0x7f58140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.463+0000 7f5831fff700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f582c19a320 tx=0x7f582000b260 comp rx=0 tx=0).stop 2023-10-18T12:23:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.463+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 msgr2=0x7f582c198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.463+0000 7f5831fff700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 0x7f582c198dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f581c00e9f0 tx=0x7f581c00ed00 comp rx=0 tx=0).stop 2023-10-18T12:23:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 shutdown_connections 2023-10-18T12:23:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f582c10cee0 0x7f582c19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f582c072c20 0x7f582c198dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 --2- 172.21.15.70:0/2390049080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f582c073540 0x7f582c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 >> 172.21.15.70:0/2390049080 conn(0x7f582c0fc170 msgr2=0x7f582c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 shutdown_connections 2023-10-18T12:23:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:55.464+0000 7f5831fff700 1 -- 172.21.15.70:0/2390049080 wait complete. 2023-10-18T12:23:55.469 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 10 2023-10-18T12:23:55.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:55.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:55.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:55.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:55.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:55.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:55.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: Deploying daemon mds.g on smithi070 2023-10-18T12:23:55.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.186:6838/2873838836,v1:172.21.15.186:6839/2873838836] up:boot 2023-10-18T12:23:55.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: fsmap 9 up:standby 2023-10-18T12:23:55.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T12:23:55.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:55 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2390049080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:55.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:55.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:55.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:55.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:55.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:55.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: Deploying daemon mds.g on smithi070 2023-10-18T12:23:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.186:6838/2873838836,v1:172.21.15.186:6839/2873838836] up:boot 2023-10-18T12:23:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: fsmap 9 up:standby 2023-10-18T12:23:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T12:23:55.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:55 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2390049080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:56.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:56.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:56.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:56.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: Deploying daemon mds.g on smithi070 2023-10-18T12:23:56.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.186:6838/2873838836,v1:172.21.15.186:6839/2873838836] up:boot 2023-10-18T12:23:56.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: fsmap 9 up:standby 2023-10-18T12:23:56.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T12:23:56.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:55 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2390049080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:56.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:56 smithi186 ceph-mon[135253]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:56.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:56 smithi070 ceph-mon[98639]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:57.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:56 smithi152 ceph-mon[134767]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:57.355 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:23:56 smithi070 systemd[1]: Starting Ceph mds.g for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:23:57.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:23:57.565 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:23:57 smithi070 podman[157278]: 2023-10-18T12:23:57.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 -- 172.21.15.70:0/1632563248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 msgr2=0x7f1a10073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:57.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 --2- 172.21.15.70:0/1632563248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10073000 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009990 tx=0x7f1a0c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:23:57.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 -- 172.21.15.70:0/1632563248 shutdown_connections 2023-10-18T12:23:57.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 --2- 172.21.15.70:0/1632563248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a1010cee0 0x7f1a1010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:57.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 --2- 172.21.15.70:0/1632563248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10073000 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:57.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 --2- 172.21.15.70:0/1632563248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a10073540 0x7f1a1010c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:57.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 -- 172.21.15.70:0/1632563248 >> 172.21.15.70:0/1632563248 conn(0x7f1a100fc1d0 msgr2=0x7f1a100fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:57.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 -- 172.21.15.70:0/1632563248 shutdown_connections 2023-10-18T12:23:57.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.700+0000 7f1a177ff700 1 -- 172.21.15.70:0/1632563248 wait complete. 2023-10-18T12:23:57.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 Processor -- start 2023-10-18T12:23:57.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 -- start start 2023-10-18T12:23:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a10073540 0x7f1a101992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a1010cee0 0x7f1a1019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1019dc90 con 0x7f1a10072c20 2023-10-18T12:23:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1019de00 con 0x7f1a10073540 2023-10-18T12:23:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a177ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1019dfc0 con 0x7f1a1010cee0 2023-10-18T12:23:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.702+0000 7f1a1559b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45534/0 (socket says 172.21.15.70:45534) 2023-10-18T12:23:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 -- 172.21.15.70:0/3644604869 learned_addr learned my addr 172.21.15.70:0/3644604869 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:23:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a14d9a700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a10073540 0x7f1a101992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a15d9c700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a1010cee0 0x7f1a1019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 -- 172.21.15.70:0/3644604869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a1010cee0 msgr2=0x7f1a1019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a1010cee0 0x7f1a1019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 -- 172.21.15.70:0/3644604869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a10073540 msgr2=0x7f1a101992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a10073540 0x7f1a101992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 -- 172.21.15.70:0/3644604869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0095f0 con 0x7f1a10072c20 2023-10-18T12:23:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a1559b700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10198d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005910 tx=0x7f1a0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c01d070 con 0x7f1a10072c20 2023-10-18T12:23:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0c022470 con 0x7f1a10072c20 2023-10-18T12:23:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.703+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c017650 con 0x7f1a10072c20 2023-10-18T12:23:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.704+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1019e240 con 0x7f1a10072c20 2023-10-18T12:23:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.704+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1019e4c0 con 0x7f1a10072c20 2023-10-18T12:23:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.705+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a101080f0 con 0x7f1a10072c20 2023-10-18T12:23:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.707+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1a0c0225e0 con 0x7f1a10072c20 2023-10-18T12:23:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.708+0000 7f1a067fc700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19fc0a14d0 0x7f19fc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:23:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.709+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f1a0c0cdf60 con 0x7f1a10072c20 2023-10-18T12:23:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.709+0000 7f1a14d9a700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19fc0a14d0 0x7f19fc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:23:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.709+0000 7f1a14d9a700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19fc0a14d0 0x7f19fc0a3990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a1019a2c0 tx=0x7f1a00009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:23:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.712+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1a0c097980 con 0x7f1a10072c20 2023-10-18T12:23:57.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.994+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a1019a110 con 0x7f1a10072c20 2023-10-18T12:23:57.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.995+0000 7f1a067fc700 1 -- 172.21.15.70:0/3644604869 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f1a0c09afa0 con 0x7f1a10072c20 2023-10-18T12:23:57.998 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:23:57.998 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-18T12:23:58.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.997+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19fc0a14d0 msgr2=0x7f19fc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:58.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.997+0000 7f1a177ff700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19fc0a14d0 0x7f19fc0a3990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a1019a2c0 tx=0x7f1a00009260 comp rx=0 tx=0).stop 2023-10-18T12:23:58.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.997+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 msgr2=0x7f1a10198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:23:58.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.997+0000 7f1a177ff700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10198d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005910 tx=0x7f1a0c004730 comp rx=0 tx=0).stop 2023-10-18T12:23:58.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 shutdown_connections 2023-10-18T12:23:58.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a1010cee0 0x7f1a1019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:58.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a10072c20 0x7f1a10198d70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:58.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19fc0a14d0 0x7f19fc0a3990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 --2- 172.21.15.70:0/3644604869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a10073540 0x7f1a101992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:23:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 >> 172.21.15.70:0/3644604869 conn(0x7f1a100fc1d0 msgr2=0x7f1a100fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:23:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 shutdown_connections 2023-10-18T12:23:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:23:57.998+0000 7f1a177ff700 1 -- 172.21.15.70:0/3644604869 wait complete. 2023-10-18T12:23:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 10 2023-10-18T12:23:58.017 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:23:57 smithi070 bash[157278]: 429a5edd1f249448a7a379a1b785ef9508fcb1a5a0f475df591bbeafbb566c2a 2023-10-18T12:23:58.018 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:23:57 smithi070 ceph-mds[157361]: -- 172.21.15.70:0/4080787053 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580d3b4f1e0 con 0x5580d3b26800 2023-10-18T12:23:58.018 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:23:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g[157317]: starting mds.g at 2023-10-18T12:23:58.018 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:23:57 smithi070 systemd[1]: Started Ceph mds.g for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:23:59.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:59.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:59.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:59.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:59.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: Deploying daemon mds.a on smithi070 2023-10-18T12:23:59.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3644604869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:59.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.70:6838/3607185387,v1:172.21.15.70:6839/3607185387] up:boot 2023-10-18T12:23:59.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: fsmap 10 up:standby 2023-10-18T12:23:59.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:23:58 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T12:23:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: Deploying daemon mds.a on smithi070 2023-10-18T12:23:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3644604869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.70:6838/3607185387,v1:172.21.15.70:6839/3607185387] up:boot 2023-10-18T12:23:59.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: fsmap 10 up:standby 2023-10-18T12:23:59.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:23:58 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T12:23:59.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:23:59.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:23:59.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:23:59.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:23:59.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:23:59.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: Deploying daemon mds.a on smithi070 2023-10-18T12:23:59.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3644604869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:23:59.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.70:6838/3607185387,v1:172.21.15.70:6839/3607185387] up:boot 2023-10-18T12:23:59.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: fsmap 10 up:standby 2023-10-18T12:23:59.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:23:58 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T12:24:00.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:24:00.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 -- 172.21.15.70:0/2213290226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6414101060 msgr2=0x7f641410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:00.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 --2- 172.21.15.70:0/2213290226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6414101060 0x7f641410d570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f640c00b0a0 tx=0x7f640c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 -- 172.21.15.70:0/2213290226 shutdown_connections 2023-10-18T12:24:00.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 --2- 172.21.15.70:0/2213290226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6414101060 0x7f641410d570 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 --2- 172.21.15.70:0/2213290226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64141066b0 0x7f6414106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 --2- 172.21.15.70:0/2213290226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64141006a0 0x7f6414100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 -- 172.21.15.70:0/2213290226 >> 172.21.15.70:0/2213290226 conn(0x7f64140fc170 msgr2=0x7f64140fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 -- 172.21.15.70:0/2213290226 shutdown_connections 2023-10-18T12:24:00.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.300+0000 7f641b7ff700 1 -- 172.21.15.70:0/2213290226 wait complete. 2023-10-18T12:24:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.301+0000 7f641b7ff700 1 Processor -- start 2023-10-18T12:24:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.301+0000 7f641b7ff700 1 -- start start 2023-10-18T12:24:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641b7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 0x7f641419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641b7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6414101060 0x7f641419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64141066b0 0x7f64141968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641b7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6414197070 con 0x7f64141006a0 2023-10-18T12:24:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641b7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141971e0 con 0x7f6414101060 2023-10-18T12:24:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141973a0 con 0x7f64141066b0 2023-10-18T12:24:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 0x7f641419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 0x7f641419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34694/0 (socket says 172.21.15.70:34694) 2023-10-18T12:24:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 -- 172.21.15.70:0/1657416476 learned_addr learned my addr 172.21.15.70:0/1657416476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f6418d9a700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6414101060 0x7f641419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f6419d9c700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64141066b0 0x7f64141968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 -- 172.21.15.70:0/1657416476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64141066b0 msgr2=0x7f64141968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64141066b0 0x7f64141968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 -- 172.21.15.70:0/1657416476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6414101060 msgr2=0x7f641419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:00.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6414101060 0x7f641419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.302+0000 7f641959b700 1 -- 172.21.15.70:0/1657416476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6410009520 con 0x7f64141006a0 2023-10-18T12:24:00.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.303+0000 7f641959b700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 0x7f641419c800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f641000eb30 tx=0x7f641000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:00.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.303+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410009880 con 0x7f64141006a0 2023-10-18T12:24:00.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.303+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64100044e0 con 0x7f64141006a0 2023-10-18T12:24:00.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.303+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c009d00 con 0x7f64141006a0 2023-10-18T12:24:00.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.303+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410003d80 con 0x7f64141006a0 2023-10-18T12:24:00.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.303+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64141a3780 con 0x7f64141006a0 2023-10-18T12:24:00.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.304+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6414108cc0 con 0x7f64141006a0 2023-10-18T12:24:00.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.307+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6410003f20 con 0x7f64141006a0 2023-10-18T12:24:00.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.308+0000 7f640a7fc700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64000a1550 0x7f64000a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:00.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.308+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f6410014070 con 0x7f64141006a0 2023-10-18T12:24:00.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.308+0000 7f6418d9a700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64000a1550 0x7f64000a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:00.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.309+0000 7f6418d9a700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64000a1550 0x7f64000a3a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6414198010 tx=0x7f6404009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:00.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.312+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f64100d3080 con 0x7f64141006a0 2023-10-18T12:24:00.354 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:24:00 smithi070 systemd[1]: Starting Ceph mds.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:24:00.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.584+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6414197d80 con 0x7f64141006a0 2023-10-18T12:24:00.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.584+0000 7f640a7fc700 1 -- 172.21.15.70:0/1657416476 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f6410096800 con 0x7f64141006a0 2023-10-18T12:24:00.587 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:00.588 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24719,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-10-18T12:24:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64000a1550 msgr2=0x7f64000a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64000a1550 0x7f64000a3a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6414198010 tx=0x7f6404009260 comp rx=0 tx=0).stop 2023-10-18T12:24:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 msgr2=0x7f641419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 0x7f641419c800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f641000eb30 tx=0x7f641000ee40 comp rx=0 tx=0).stop 2023-10-18T12:24:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 shutdown_connections 2023-10-18T12:24:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64141066b0 0x7f64141968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64141006a0 0x7f641419c800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64000a1550 0x7f64000a3a10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 --2- 172.21.15.70:0/1657416476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6414101060 0x7f641419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 >> 172.21.15.70:0/1657416476 conn(0x7f64140fc170 msgr2=0x7f64140fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 shutdown_connections 2023-10-18T12:24:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:00.587+0000 7f641b7ff700 1 -- 172.21.15.70:0/1657416476 wait complete. 2023-10-18T12:24:00.593 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 11 2023-10-18T12:24:00.932 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:24:00 smithi070 podman[157706]: 2023-10-18T12:24:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1657416476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:24:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:24:01.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:01 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:01.356 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:24:00 smithi070 bash[157706]: 60e3449957ea599f6a8cbb088f4b8580dbb572e236378bfc2a24980f50af8384 2023-10-18T12:24:01.357 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:24:00 smithi070 ceph-mds[157748]: -- 172.21.15.70:0/3893752490 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b7d2151e0 con 0x562b7d1ed000 2023-10-18T12:24:01.357 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:24:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-a[157724]: starting mds.a at 2023-10-18T12:24:01.357 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:24:00 smithi070 systemd[1]: Started Ceph mds.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:24:01.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1657416476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:24:01.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:24:01.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:01 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1657416476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:01.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T12:24:01.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T12:24:01.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:01 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:02.338 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:02 smithi152 ceph-mon[134767]: Deploying daemon mds.b on smithi152 2023-10-18T12:24:02.339 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:02 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.70:6840/3718886784,v1:172.21.15.70:6841/3718886784] up:boot 2023-10-18T12:24:02.339 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:02 smithi152 ceph-mon[134767]: fsmap 11 up:standby 2023-10-18T12:24:02.339 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:02 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T12:24:02.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:02 smithi186 ceph-mon[135253]: Deploying daemon mds.b on smithi152 2023-10-18T12:24:02.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:02 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.70:6840/3718886784,v1:172.21.15.70:6841/3718886784] up:boot 2023-10-18T12:24:02.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:02 smithi186 ceph-mon[135253]: fsmap 11 up:standby 2023-10-18T12:24:02.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:02 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T12:24:02.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:02 smithi070 ceph-mon[98639]: Deploying daemon mds.b on smithi152 2023-10-18T12:24:02.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:02 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.70:6840/3718886784,v1:172.21.15.70:6841/3718886784] up:boot 2023-10-18T12:24:02.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:02 smithi070 ceph-mon[98639]: fsmap 11 up:standby 2023-10-18T12:24:02.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:02 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T12:24:02.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:24:02.671 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:02 smithi152 systemd[1]: Starting Ceph mds.b for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:24:02.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.816+0000 7f1eac5ff700 1 -- 172.21.15.70:0/910019319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4100290 msgr2=0x7f1ea410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:02.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.816+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/910019319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4100290 0x7f1ea410c420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00b0a0 tx=0x7f1e9c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:02.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 -- 172.21.15.70:0/910019319 shutdown_connections 2023-10-18T12:24:02.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/910019319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4100290 0x7f1ea410c420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:02.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/910019319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4105710 0x7f1ea4105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:02.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/910019319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea40ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:02.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 -- 172.21.15.70:0/910019319 >> 172.21.15.70:0/910019319 conn(0x7f1ea4076400 msgr2=0x7f1ea4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:02.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 -- 172.21.15.70:0/910019319 shutdown_connections 2023-10-18T12:24:02.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.817+0000 7f1eac5ff700 1 -- 172.21.15.70:0/910019319 wait complete. 2023-10-18T12:24:02.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 Processor -- start 2023-10-18T12:24:02.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 -- start start 2023-10-18T12:24:02.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:02.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4100290 0x7f1ea4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:02.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ea4105710 0x7f1ea419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:02.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419dd10 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419de80 con 0x7f1ea4100290 2023-10-18T12:24:02.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eac5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419e040 con 0x7f1ea4105710 2023-10-18T12:24:02.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eaa39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:02.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eaa39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea4198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34728/0 (socket says 172.21.15.70:34728) 2023-10-18T12:24:02.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.818+0000 7f1eaa39b700 1 -- 172.21.15.70:0/2487269708 learned_addr learned my addr 172.21.15.70:0/2487269708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:02.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1ea9b9a700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4100290 0x7f1ea4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:02.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaab9c700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ea4105710 0x7f1ea419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:02.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaa39b700 1 -- 172.21.15.70:0/2487269708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ea4105710 msgr2=0x7f1ea419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:02.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaa39b700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ea4105710 0x7f1ea419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:02.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaa39b700 1 -- 172.21.15.70:0/2487269708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4100290 msgr2=0x7f1ea4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:02.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaa39b700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4100290 0x7f1ea4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:02.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaa39b700 1 -- 172.21.15.70:0/2487269708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0009520 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eaa39b700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea4198df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000eb30 tx=0x7f1ea000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:02.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0009880 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea00044e0 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0010430 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c009d00 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.819+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea41a5980 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.820+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea4100cc0 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.822+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1ea0003ee0 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.824+0000 7f1e9b7fe700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e900a1550 0x7f1e900a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:02.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.825+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f1ea0014070 con 0x7f1ea40ff8d0 2023-10-18T12:24:02.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.825+0000 7f1ea9b9a700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e900a1550 0x7f1e900a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:02.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.825+0000 7f1ea9b9a700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e900a1550 0x7f1e900a3a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ea419a340 tx=0x7f1e94009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:02.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:02.829+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ea0096840 con 0x7f1ea40ff8d0 2023-10-18T12:24:03.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.073+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ea4067530 con 0x7f1ea40ff8d0 2023-10-18T12:24:03.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.074+0000 7f1e9b7fe700 1 -- 172.21.15.70:0/2487269708 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f1ea0004190 con 0x7f1ea40ff8d0 2023-10-18T12:24:03.077 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:03.077 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24719,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-10-18T12:24:03.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e900a1550 msgr2=0x7f1e900a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:03.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e900a1550 0x7f1e900a3a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ea419a340 tx=0x7f1e94009260 comp rx=0 tx=0).stop 2023-10-18T12:24:03.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 msgr2=0x7f1ea4198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:03.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea4198df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000eb30 tx=0x7f1ea000eef0 comp rx=0 tx=0).stop 2023-10-18T12:24:03.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 shutdown_connections 2023-10-18T12:24:03.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ea4105710 0x7f1ea419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:03.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40ff8d0 0x7f1ea4198df0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:03.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e900a1550 0x7f1e900a3a10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:03.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 --2- 172.21.15.70:0/2487269708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4100290 0x7f1ea4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:03.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 >> 172.21.15.70:0/2487269708 conn(0x7f1ea4076400 msgr2=0x7f1ea4067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:03.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 shutdown_connections 2023-10-18T12:24:03.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:03.076+0000 7f1eac5ff700 1 -- 172.21.15.70:0/2487269708 wait complete. 2023-10-18T12:24:03.082 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 12 2023-10-18T12:24:03.127 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:02 smithi152 podman[173945]: 2023-10-18T12:24:03.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:03 smithi070 ceph-mon[98639]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:03 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2487269708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:03.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:03 smithi152 ceph-mon[134767]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:03 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2487269708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:03.422 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:03 smithi152 ceph-mds[173985]: -- 172.21.15.152:0/844455458 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a73ce351e0 con 0x55a73ce0ac00 2023-10-18T12:24:03.422 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: starting mds.b at 2023-10-18T12:24:03.423 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:03 smithi152 bash[173945]: c275d358b42f44da2ad376d56109bace9842c619c6ff721a6a042d530cfeb0b6 2023-10-18T12:24:03.423 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:03 smithi152 systemd[1]: Started Ceph mds.b for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:24:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:03 smithi186 ceph-mon[135253]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:03.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:03 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2487269708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:24:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:04.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] up:boot 2023-10-18T12:24:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: fsmap 12 up:standby 2023-10-18T12:24:04.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:04 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T12:24:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:24:04.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:04.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] up:boot 2023-10-18T12:24:04.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: fsmap 12 up:standby 2023-10-18T12:24:04.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:04 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T12:24:04.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:24:04.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:04.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:04.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] up:boot 2023-10-18T12:24:04.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: fsmap 12 up:standby 2023-10-18T12:24:04.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:04 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T12:24:05.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:24:05.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/1689969171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc4067fa0 msgr2=0x7fbfc410d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/1689969171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc4067fa0 0x7fbfc410d530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b0a0 tx=0x7fbfbc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/1689969171 shutdown_connections 2023-10-18T12:24:05.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/1689969171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc4067fa0 0x7fbfc410d530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/1689969171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc41066c0 0x7fbfc4106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/1689969171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4106fe0 0x7fbfc4067870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/1689969171 >> 172.21.15.70:0/1689969171 conn(0x7fbfc4076400 msgr2=0x7fbfc4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:05.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/1689969171 shutdown_connections 2023-10-18T12:24:05.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.307+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/1689969171 wait complete. 2023-10-18T12:24:05.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.308+0000 7fbfcc1ff700 1 Processor -- start 2023-10-18T12:24:05.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 -- start start 2023-10-18T12:24:05.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 0x7fbfc4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc41066c0 0x7fbfc4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc4106fe0 0x7fbfc419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc419dde0 con 0x7fbfc4106fe0 2023-10-18T12:24:05.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc419df50 con 0x7fbfc41066c0 2023-10-18T12:24:05.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfcc1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc419e110 con 0x7fbfc4067fa0 2023-10-18T12:24:05.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfc9f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 0x7fbfc4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfc9f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 0x7fbfc4198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:45006/0 (socket says 172.21.15.70:45006) 2023-10-18T12:24:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfc9f9b700 1 -- 172.21.15.70:0/3476879906 learned_addr learned my addr 172.21.15.70:0/3476879906 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfc979a700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc41066c0 0x7fbfc4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfca79c700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc4106fe0 0x7fbfc419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.309+0000 7fbfc9f9b700 1 -- 172.21.15.70:0/3476879906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc41066c0 msgr2=0x7fbfc4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfc9f9b700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc41066c0 0x7fbfc4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfc9f9b700 1 -- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc4106fe0 msgr2=0x7fbfc419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfc9f9b700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc4106fe0 0x7fbfc419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfc9f9b700 1 -- 172.21.15.70:0/3476879906 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009d00 con 0x7fbfc4067fa0 2023-10-18T12:24:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfc979a700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc41066c0 0x7fbfc4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:05.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfca79c700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc4106fe0 0x7fbfc419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:05.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfc9f9b700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 0x7fbfc4198ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000e9a0 tx=0x7fbfc000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:05.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0009790 con 0x7fbfc4067fa0 2023-10-18T12:24:05.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc0010430 con 0x7fbfc4067fa0 2023-10-18T12:24:05.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc419e3f0 con 0x7fbfc4067fa0 2023-10-18T12:24:05.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0018680 con 0x7fbfc4067fa0 2023-10-18T12:24:05.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.310+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc41a5c30 con 0x7fbfc4067fa0 2023-10-18T12:24:05.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.311+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc4101de0 con 0x7fbfc4067fa0 2023-10-18T12:24:05.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.313+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fbfc00105a0 con 0x7fbfc4067fa0 2023-10-18T12:24:05.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.314+0000 7fbfbaffd700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbfb00a1550 0x7fbfb00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.315+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fbfc0014070 con 0x7fbfc4067fa0 2023-10-18T12:24:05.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.315+0000 7fbfc979a700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbfb00a1550 0x7fbfb00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.315+0000 7fbfc979a700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbfb00a1550 0x7fbfb00a3a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbfc419a410 tx=0x7fbfb400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:05.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.317+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbfc0096570 con 0x7fbfc4067fa0 2023-10-18T12:24:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.573+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfc4067530 con 0x7fbfc4067fa0 2023-10-18T12:24:05.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.574+0000 7fbfbaffd700 1 -- 172.21.15.70:0/3476879906 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7fbfc0099b90 con 0x7fbfc4067fa0 2023-10-18T12:24:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:05.577 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24719,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24743,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-18T12:24:05.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbfb00a1550 msgr2=0x7fbfb00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbfb00a1550 0x7fbfb00a3a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbfc419a410 tx=0x7fbfb400b410 comp rx=0 tx=0).stop 2023-10-18T12:24:05.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 msgr2=0x7fbfc4198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 0x7fbfc4198ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000e9a0 tx=0x7fbfc000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 shutdown_connections 2023-10-18T12:24:05.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc4106fe0 0x7fbfc419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc4067fa0 0x7fbfc4198ec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbfb00a1550 0x7fbfb00a3a10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 --2- 172.21.15.70:0/3476879906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfc41066c0 0x7fbfc4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.576+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 >> 172.21.15.70:0/3476879906 conn(0x7fbfc4076400 msgr2=0x7fbfc410b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:05.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.577+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 shutdown_connections 2023-10-18T12:24:05.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.577+0000 7fbfcc1ff700 1 -- 172.21.15.70:0/3476879906 wait complete. 2023-10-18T12:24:05.583 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 13 2023-10-18T12:24:05.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:05.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 -- 172.21.15.70:0/3311486337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02241006a0 msgr2=0x7f0224100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 --2- 172.21.15.70:0/3311486337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02241006a0 0x7f0224100b20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f020c009990 tx=0x7f020c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 -- 172.21.15.70:0/3311486337 shutdown_connections 2023-10-18T12:24:05.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 --2- 172.21.15.70:0/3311486337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0224101060 0x7f022410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 --2- 172.21.15.70:0/3311486337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 --2- 172.21.15.70:0/3311486337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02241006a0 0x7f0224100b20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 -- 172.21.15.70:0/3311486337 >> 172.21.15.70:0/3311486337 conn(0x7f02240fc1d0 msgr2=0x7f02240fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:05.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 -- 172.21.15.70:0/3311486337 shutdown_connections 2023-10-18T12:24:05.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.807+0000 7f0229bff700 1 -- 172.21.15.70:0/3311486337 wait complete. 2023-10-18T12:24:05.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 Processor -- start 2023-10-18T12:24:05.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 -- start start 2023-10-18T12:24:05.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02241006a0 0x7f0224072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0224101060 0x7f0224073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241a4f80 con 0x7f0224101060 2023-10-18T12:24:05.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241a50f0 con 0x7f02241066b0 2023-10-18T12:24:05.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.808+0000 7f0229bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241a52b0 con 0x7f02241006a0 2023-10-18T12:24:05.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47464/0 (socket says 172.21.15.70:47464) 2023-10-18T12:24:05.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 -- 172.21.15.70:0/1172679658 learned_addr learned my addr 172.21.15.70:0/1172679658 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:05.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f02237fe700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02241006a0 0x7f0224072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f021bfff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0224101060 0x7f0224073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 -- 172.21.15.70:0/1172679658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02241006a0 msgr2=0x7f0224072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02241006a0 0x7f0224072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 -- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0224101060 msgr2=0x7f0224073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:05.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0224101060 0x7f0224073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:05.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 -- 172.21.15.70:0/1172679658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0095f0 con 0x7f02241066b0 2023-10-18T12:24:05.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f0223fff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224073570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f021c00ef30 tx=0x7f021c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:05.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.809+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c018070 con 0x7f02241066b0 2023-10-18T12:24:05.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.810+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021c00f040 con 0x7f02241066b0 2023-10-18T12:24:05.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.810+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02241a5530 con 0x7f02241066b0 2023-10-18T12:24:05.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.810+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00cd00 con 0x7f02241066b0 2023-10-18T12:24:05.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.810+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02241a59f0 con 0x7f02241066b0 2023-10-18T12:24:05.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.811+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0224108cc0 con 0x7f02241066b0 2023-10-18T12:24:05.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.812+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f021c003a00 con 0x7f02241066b0 2023-10-18T12:24:05.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.814+0000 7f02217fa700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02100a14d0 0x7f02100a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:05.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.814+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f021c0cc5a0 con 0x7f02241066b0 2023-10-18T12:24:05.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.814+0000 7f02237fe700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02100a14d0 0x7f02100a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:05.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.814+0000 7f02237fe700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02100a14d0 0x7f02100a3990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0214009be0 tx=0x7f0214009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:05.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:05.817+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f021c095fc0 con 0x7f02241066b0 2023-10-18T12:24:05.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:05 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3476879906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:05.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:05 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3476879906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:05.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:05 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3476879906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:06.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.030+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0224101de0 con 0x7f02241066b0 2023-10-18T12:24:06.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.032+0000 7f02217fa700 1 -- 172.21.15.70:0/1172679658 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18748 (secure 0 0 0) 0x7f021c0995e0 con 0x7f02241066b0 2023-10-18T12:24:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:06.035 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":81,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:23:23.115758+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:06.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.034+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02100a14d0 msgr2=0x7f02100a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.034+0000 7f0229bff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02100a14d0 0x7f02100a3990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0214009be0 tx=0x7f0214009260 comp rx=0 tx=0).stop 2023-10-18T12:24:06.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.034+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 msgr2=0x7f0224073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.034+0000 7f0229bff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224073570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f021c00ef30 tx=0x7f021c00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 shutdown_connections 2023-10-18T12:24:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02100a14d0 0x7f02100a3990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02241066b0 0x7f0224073570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02241006a0 0x7f0224072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 --2- 172.21.15.70:0/1172679658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0224101060 0x7f0224073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 >> 172.21.15.70:0/1172679658 conn(0x7f02240fc1d0 msgr2=0x7f02240fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:06.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 shutdown_connections 2023-10-18T12:24:06.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.035+0000 7f0229bff700 1 -- 172.21.15.70:0/1172679658 wait complete. 2023-10-18T12:24:06.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:24:06.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:24:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2662591319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080734b0 msgr2=0x7fab081111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2662591319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080734b0 0x7fab081111e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faafc009990 tx=0x7faafc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2662591319 shutdown_connections 2023-10-18T12:24:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2662591319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08111720 0x7fab08113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2662591319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab08072b90 0x7fab08072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2662591319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080734b0 0x7fab081111e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2662591319 >> 172.21.15.70:0/2662591319 conn(0x7fab080fc200 msgr2=0x7fab080fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:06.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2662591319 shutdown_connections 2023-10-18T12:24:06.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.275+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2662591319 wait complete. 2023-10-18T12:24:06.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.276+0000 7fab0e9ff700 1 Processor -- start 2023-10-18T12:24:06.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.276+0000 7fab0e9ff700 1 -- start start 2023-10-18T12:24:06.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0e9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 0x7fab0819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0e9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080734b0 0x7fab0819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0e9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab08111720 0x7fab081a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0e9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081a2170 con 0x7fab080734b0 2023-10-18T12:24:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0e9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081a22e0 con 0x7fab08111720 2023-10-18T12:24:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0e9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081a24a0 con 0x7fab08072b90 2023-10-18T12:24:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 0x7fab0819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080734b0 0x7fab0819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 0x7fab0819d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:45056/0 (socket says 172.21.15.70:45056) 2023-10-18T12:24:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 -- 172.21.15.70:0/2528568027 learned_addr learned my addr 172.21.15.70:0/2528568027 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab0cf9c700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab08111720 0x7fab081a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 -- 172.21.15.70:0/2528568027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab08111720 msgr2=0x7fab081a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab08111720 0x7fab081a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 -- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080734b0 msgr2=0x7fab0819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080734b0 0x7fab0819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab07fff700 1 -- 172.21.15.70:0/2528568027 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009520 con 0x7fab08072b90 2023-10-18T12:24:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.277+0000 7fab077fe700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080734b0 0x7fab0819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.278+0000 7fab07fff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 0x7fab0819d250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faaf800eb30 tx=0x7faaf800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.278+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8009880 con 0x7fab08072b90 2023-10-18T12:24:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.278+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc0095f0 con 0x7fab08072b90 2023-10-18T12:24:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.278+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaf80044e0 con 0x7fab08072b90 2023-10-18T12:24:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.278+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8003d80 con 0x7fab08072b90 2023-10-18T12:24:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.278+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab081a9de0 con 0x7fab08072b90 2023-10-18T12:24:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.279+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab08103c00 con 0x7fab08072b90 2023-10-18T12:24:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.281+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faaf8003f20 con 0x7fab08072b90 2023-10-18T12:24:06.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.282+0000 7fab057fa700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaf00a14d0 0x7faaf00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.283+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7faaf8014070 con 0x7fab08072b90 2023-10-18T12:24:06.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.283+0000 7fab077fe700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaf00a14d0 0x7faaf00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.283+0000 7fab077fe700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaf00a14d0 0x7faaf00a3990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab0819e7a0 tx=0x7faafc00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:06.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.285+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faaf8096ce0 con 0x7fab08072b90 2023-10-18T12:24:06.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.506+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab0819e550 con 0x7fab08072b90 2023-10-18T12:24:06.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.507+0000 7fab057fa700 1 -- 172.21.15.70:0/2528568027 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7faaf809a300 con 0x7fab08072b90 2023-10-18T12:24:06.510 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:24:06.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaf00a14d0 msgr2=0x7faaf00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaf00a14d0 0x7faaf00a3990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab0819e7a0 tx=0x7faafc00b260 comp rx=0 tx=0).stop 2023-10-18T12:24:06.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 msgr2=0x7fab0819d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 0x7fab0819d250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faaf800eb30 tx=0x7faaf800ee40 comp rx=0 tx=0).stop 2023-10-18T12:24:06.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 shutdown_connections 2023-10-18T12:24:06.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaf00a14d0 0x7faaf00a3990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab08111720 0x7fab081a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab08072b90 0x7fab0819d250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 --2- 172.21.15.70:0/2528568027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080734b0 0x7fab0819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 >> 172.21.15.70:0/2528568027 conn(0x7fab080fc200 msgr2=0x7fab080fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:06.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 shutdown_connections 2023-10-18T12:24:06.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.510+0000 7fab0e9ff700 1 -- 172.21.15.70:0/2528568027 wait complete. 2023-10-18T12:24:06.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T12:24:06.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 -- 172.21.15.70:0/573243326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 msgr2=0x7fd978072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/573243326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 0x7fd978072fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9680098f0 tx=0x7fd968009c00 comp rx=0 tx=0).stop 2023-10-18T12:24:06.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 -- 172.21.15.70:0/573243326 shutdown_connections 2023-10-18T12:24:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/573243326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 0x7fd97810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/573243326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 0x7fd978072fa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/573243326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd97810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 -- 172.21.15.70:0/573243326 >> 172.21.15.70:0/573243326 conn(0x7fd9780fc180 msgr2=0x7fd9780fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 -- 172.21.15.70:0/573243326 shutdown_connections 2023-10-18T12:24:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.744+0000 7fd97d9ff700 1 -- 172.21.15.70:0/573243326 wait complete. 2023-10-18T12:24:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 Processor -- start 2023-10-18T12:24:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 -- start start 2023-10-18T12:24:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 0x7fd978198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd978199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 0x7fd97819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97819dc70 con 0x7fd97810cf30 2023-10-18T12:24:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97819dde0 con 0x7fd9780734e0 2023-10-18T12:24:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.745+0000 7fd97d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97819dfa0 con 0x7fd978072bc0 2023-10-18T12:24:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 0x7fd97819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd978199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd978199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47498/0 (socket says 172.21.15.70:47498) 2023-10-18T12:24:06.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 -- 172.21.15.70:0/1035794174 learned_addr learned my addr 172.21.15.70:0/1035794174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:06.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd976ffd700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 0x7fd978198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 -- 172.21.15.70:0/1035794174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 msgr2=0x7fd978198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 0x7fd978198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 -- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 msgr2=0x7fd97819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 0x7fd97819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 -- 172.21.15.70:0/1035794174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968009520 con 0x7fd9780734e0 2023-10-18T12:24:06.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9777fe700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 0x7fd97819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:06.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.746+0000 7fd9767fc700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd978199300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00d980 tx=0x7fd96c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:06.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.747+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00f890 con 0x7fd9780734e0 2023-10-18T12:24:06.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.747+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96c010460 con 0x7fd9780734e0 2023-10-18T12:24:06.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.747+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c009600 con 0x7fd9780734e0 2023-10-18T12:24:06.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.747+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97819e280 con 0x7fd9780734e0 2023-10-18T12:24:06.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.747+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9781a5ac0 con 0x7fd9780734e0 2023-10-18T12:24:06.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.748+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd978108140 con 0x7fd9780734e0 2023-10-18T12:24:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.749+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd96c0105d0 con 0x7fd9780734e0 2023-10-18T12:24:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.751+0000 7fd97c9fd700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9600a1550 0x7fd9600a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.751+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fd96c0cc710 con 0x7fd9780734e0 2023-10-18T12:24:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.751+0000 7fd976ffd700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9600a1550 0x7fd9600a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.752+0000 7fd976ffd700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9600a1550 0x7fd9600a3a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd968000c00 tx=0x7fd968005dc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:06.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.755+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd96c0960a0 con 0x7fd9780734e0 2023-10-18T12:24:06.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:06.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1172679658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:06.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:06 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2528568027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:06.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:06.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1172679658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:06.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:06 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2528568027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:06.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:06.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1172679658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:06.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:06.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:06 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2528568027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.971+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd97819a0c0 con 0x7fd9780734e0 2023-10-18T12:24:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.972+0000 7fd97c9fd700 1 -- 172.21.15.70:0/1035794174 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd96c0996c0 con 0x7fd9780734e0 2023-10-18T12:24:06.975 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:06.975 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-18T12:24:06.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:24:06.975 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-18T12:24:06.976 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-18T12:24:06.976 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:24:06.976 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-18T12:24:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9600a1550 msgr2=0x7fd9600a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9600a1550 0x7fd9600a3a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd968000c00 tx=0x7fd968005dc0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 msgr2=0x7fd978199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd978199300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00d980 tx=0x7fd96c00dc90 comp rx=0 tx=0).stop 2023-10-18T12:24:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 shutdown_connections 2023-10-18T12:24:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97810cf30 0x7fd97819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978072bc0 0x7fd978198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9600a1550 0x7fd9600a3a10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 --2- 172.21.15.70:0/1035794174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9780734e0 0x7fd978199300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 >> 172.21.15.70:0/1035794174 conn(0x7fd9780fc180 msgr2=0x7fd9780fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 shutdown_connections 2023-10-18T12:24:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:06.974+0000 7fd97d9ff700 1 -- 172.21.15.70:0/1035794174 wait complete. 2023-10-18T12:24:06.997 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T12:24:06.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T12:24:07.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 -- 172.21.15.70:0/1637499127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e8072c20 msgr2=0x7f02e8073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:07.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1637499127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e8072c20 0x7f02e8073000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009990 tx=0x7f02d8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 -- 172.21.15.70:0/1637499127 shutdown_connections 2023-10-18T12:24:07.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1637499127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e810cf50 0x7f02e810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1637499127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e8072c20 0x7f02e8073000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1637499127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e8073540 0x7f02e810ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 -- 172.21.15.70:0/1637499127 >> 172.21.15.70:0/1637499127 conn(0x7f02e80fc200 msgr2=0x7f02e80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:07.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 -- 172.21.15.70:0/1637499127 shutdown_connections 2023-10-18T12:24:07.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.204+0000 7f02ecdff700 1 -- 172.21.15.70:0/1637499127 wait complete. 2023-10-18T12:24:07.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.205+0000 7f02ecdff700 1 Processor -- start 2023-10-18T12:24:07.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.205+0000 7f02ecdff700 1 -- start start 2023-10-18T12:24:07.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02ecdff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 0x7f02e819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:07.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02ecdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e8073540 0x7f02e819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:07.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02ecdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e810cf50 0x7f02e8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:07.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02ecdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e8199340 con 0x7f02e810cf50 2023-10-18T12:24:07.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02ecdff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e81994b0 con 0x7f02e8072c20 2023-10-18T12:24:07.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02ecdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e8199670 con 0x7f02e8073540 2023-10-18T12:24:07.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e810cf50 0x7f02e8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:07.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 0x7f02e819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:07.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 0x7f02e819d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47518/0 (socket says 172.21.15.70:47518) 2023-10-18T12:24:07.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e659c700 1 -- 172.21.15.70:0/1348730213 learned_addr learned my addr 172.21.15.70:0/1348730213 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:07.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e5d9b700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e8073540 0x7f02e819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:07.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e6d9d700 1 -- 172.21.15.70:0/1348730213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e8073540 msgr2=0x7f02e819df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:07.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e6d9d700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e8073540 0x7f02e819df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e6d9d700 1 -- 172.21.15.70:0/1348730213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 msgr2=0x7f02e819d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:07.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e6d9d700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 0x7f02e819d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e6d9d700 1 -- 172.21.15.70:0/1348730213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80095f0 con 0x7f02e810cf50 2023-10-18T12:24:07.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.206+0000 7f02e659c700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 0x7f02e819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:07.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.207+0000 7f02e6d9d700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e810cf50 0x7f02e8198cb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02e0007f10 tx=0x7f02e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:07.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.207+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0009ea0 con 0x7f02e810cf50 2023-10-18T12:24:07.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.207+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e000f040 con 0x7f02e810cf50 2023-10-18T12:24:07.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.207+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0004350 con 0x7f02e810cf50 2023-10-18T12:24:07.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.207+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e8199950 con 0x7f02e810cf50 2023-10-18T12:24:07.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.207+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e81a5b20 con 0x7f02e810cf50 2023-10-18T12:24:07.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.208+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e8108160 con 0x7f02e810cf50 2023-10-18T12:24:07.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.212+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f02e000db20 con 0x7f02e810cf50 2023-10-18T12:24:07.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.213+0000 7f02d77fe700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02d00a14d0 0x7f02d00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:07.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.213+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f02e00cc270 con 0x7f02e810cf50 2023-10-18T12:24:07.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.213+0000 7f02e659c700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02d00a14d0 0x7f02d00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:07.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.214+0000 7f02e659c700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02d00a14d0 0x7f02d00a3990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006010 tx=0x7f02d800b280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:07.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.217+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f02e0095c00 con 0x7f02e810cf50 2023-10-18T12:24:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.431+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f02e819a0d0 con 0x7f02e810cf50 2023-10-18T12:24:07.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.947+0000 7f02d77fe700 1 -- 172.21.15.70:0/1348730213 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f02e0099220 con 0x7f02e810cf50 2023-10-18T12:24:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02d00a14d0 msgr2=0x7f02d00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02d00a14d0 0x7f02d00a3990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006010 tx=0x7f02d800b280 comp rx=0 tx=0).stop 2023-10-18T12:24:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e810cf50 msgr2=0x7f02e8198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e810cf50 0x7f02e8198cb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02e0007f10 tx=0x7f02e000d3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 shutdown_connections 2023-10-18T12:24:07.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e810cf50 0x7f02e8198cb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02d00a14d0 0x7f02d00a3990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02e8072c20 0x7f02e819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 --2- 172.21.15.70:0/1348730213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e8073540 0x7f02e819df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.950+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 >> 172.21.15.70:0/1348730213 conn(0x7f02e80fc200 msgr2=0x7f02e80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.951+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 shutdown_connections 2023-10-18T12:24:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:07.951+0000 7f02ecdff700 1 -- 172.21.15.70:0/1348730213 wait complete. 2023-10-18T12:24:07.956 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-18T12:24:07.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-18T12:24:08.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.182+0000 7f2d45dff700 1 -- 172.21.15.70:0/436197778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100a40 msgr2=0x7f2d4010cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:08.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.182+0000 7f2d45dff700 1 --2- 172.21.15.70:0/436197778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100a40 0x7f2d4010cc50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009990 tx=0x7f2d34009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.182+0000 7f2d45dff700 1 -- 172.21.15.70:0/436197778 shutdown_connections 2023-10-18T12:24:08.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.182+0000 7f2d45dff700 1 --2- 172.21.15.70:0/436197778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010d190 0x7f2d4010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.182+0000 7f2d45dff700 1 --2- 172.21.15.70:0/436197778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d40100120 0x7f2d40100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.182+0000 7f2d45dff700 1 --2- 172.21.15.70:0/436197778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100a40 0x7f2d4010cc50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.183+0000 7f2d45dff700 1 -- 172.21.15.70:0/436197778 >> 172.21.15.70:0/436197778 conn(0x7f2d400767f0 msgr2=0x7f2d40076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:08.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.183+0000 7f2d45dff700 1 -- 172.21.15.70:0/436197778 shutdown_connections 2023-10-18T12:24:08.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.183+0000 7f2d45dff700 1 -- 172.21.15.70:0/436197778 wait complete. 2023-10-18T12:24:08.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 Processor -- start 2023-10-18T12:24:08.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 -- start start 2023-10-18T12:24:08.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 0x7f2d40198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:08.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d40100a40 0x7f2d40199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:08.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010d190 0x7f2d4019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:08.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019dd60 con 0x7f2d40100120 2023-10-18T12:24:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019ded0 con 0x7f2d4010d190 2023-10-18T12:24:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d45dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019e090 con 0x7f2d40100a40 2023-10-18T12:24:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d3f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 0x7f2d40198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d3f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 0x7f2d40198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35958/0 (socket says 172.21.15.70:35958) 2023-10-18T12:24:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d3f7fe700 1 -- 172.21.15.70:0/2260904379 learned_addr learned my addr 172.21.15.70:0/2260904379 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.184+0000 7f2d3effd700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d40100a40 0x7f2d40199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3ffff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010d190 0x7f2d4019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3f7fe700 1 -- 172.21.15.70:0/2260904379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d40100a40 msgr2=0x7f2d40199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3f7fe700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d40100a40 0x7f2d40199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3f7fe700 1 -- 172.21.15.70:0/2260904379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010d190 msgr2=0x7f2d4019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3f7fe700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010d190 0x7f2d4019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3f7fe700 1 -- 172.21.15.70:0/2260904379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d340095f0 con 0x7f2d40100120 2023-10-18T12:24:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3f7fe700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 0x7f2d40198e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e9a0 tx=0x7f2d3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30009790 con 0x7f2d40100120 2023-10-18T12:24:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4019e370 con 0x7f2d40100120 2023-10-18T12:24:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d30010430 con 0x7f2d40100120 2023-10-18T12:24:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30018680 con 0x7f2d40100120 2023-10-18T12:24:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.185+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d401a5bb0 con 0x7f2d40100120 2023-10-18T12:24:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.186+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d401083b0 con 0x7f2d40100120 2023-10-18T12:24:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.189+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2d300105a0 con 0x7f2d40100120 2023-10-18T12:24:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.190+0000 7f2d3cff9700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d280a14d0 0x7f2d280a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.190+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7712+0+0 (secure 0 0 0) 0x7f2d30014070 con 0x7f2d40100120 2023-10-18T12:24:08.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.190+0000 7f2d3effd700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d280a14d0 0x7f2d280a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.191+0000 7f2d3effd700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d280a14d0 0x7f2d280a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d4019a390 tx=0x7f2d34005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.193+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d300967a0 con 0x7f2d40100120 2023-10-18T12:24:08.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:07 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:08.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:07 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:08.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:07 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1035794174' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-18T12:24:08.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:07 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:08.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:07 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:08.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:07 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1035794174' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-18T12:24:08.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.417+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f2d400ffc30 con 0x7f2d40100120 2023-10-18T12:24:08.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:07 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:08.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:07 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:08.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:07 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1035794174' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-18T12:24:08.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.950+0000 7f2d3cff9700 1 -- 172.21.15.70:0/2260904379 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f2d300260d0 con 0x7f2d40100120 2023-10-18T12:24:08.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d280a14d0 msgr2=0x7f2d280a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:08.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d280a14d0 0x7f2d280a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d4019a390 tx=0x7f2d34005fd0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 msgr2=0x7f2d40198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:08.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 0x7f2d40198e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e9a0 tx=0x7f2d3000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 shutdown_connections 2023-10-18T12:24:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d280a14d0 0x7f2d280a3990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010d190 0x7f2d4019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d40100120 0x7f2d40198e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 --2- 172.21.15.70:0/2260904379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d40100a40 0x7f2d40199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:08.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.953+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 >> 172.21.15.70:0/2260904379 conn(0x7f2d400767f0 msgr2=0x7f2d4010e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:08.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.954+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 shutdown_connections 2023-10-18T12:24:08.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:08.954+0000 7f2d45dff700 1 -- 172.21.15.70:0/2260904379 wait complete. 2023-10-18T12:24:08.963 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-18T12:24:08.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T12:24:09.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 -- 172.21.15.70:0/555771124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c106730 msgr2=0x7efd6c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:09.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 --2- 172.21.15.70:0/555771124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c106730 0x7efd6c106b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd68009990 tx=0x7efd68009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:09.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 -- 172.21.15.70:0/555771124 shutdown_connections 2023-10-18T12:24:09.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 --2- 172.21.15.70:0/555771124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c1010e0 0x7efd6c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:09.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 --2- 172.21.15.70:0/555771124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c106730 0x7efd6c106b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:09.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 --2- 172.21.15.70:0/555771124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c100720 0x7efd6c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:09.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 -- 172.21.15.70:0/555771124 >> 172.21.15.70:0/555771124 conn(0x7efd6c0fc1b0 msgr2=0x7efd6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:09.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 -- 172.21.15.70:0/555771124 shutdown_connections 2023-10-18T12:24:09.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.210+0000 7efd747ff700 1 -- 172.21.15.70:0/555771124 wait complete. 2023-10-18T12:24:09.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.211+0000 7efd747ff700 1 Processor -- start 2023-10-18T12:24:09.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.211+0000 7efd747ff700 1 -- start start 2023-10-18T12:24:09.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd747ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 0x7efd6c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:09.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd747ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c1010e0 0x7efd6c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:09.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd747ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c106730 0x7efd6c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:09.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd747ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19dcb0 con 0x7efd6c106730 2023-10-18T12:24:09.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd747ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19de20 con 0x7efd6c1010e0 2023-10-18T12:24:09.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd747ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19dfe0 con 0x7efd6c100720 2023-10-18T12:24:09.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 0x7efd6c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:09.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 0x7efd6c198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60100/0 (socket says 172.21.15.70:60100) 2023-10-18T12:24:09.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 -- 172.21.15.70:0/1466911362 learned_addr learned my addr 172.21.15.70:0/1466911362 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:09.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd71d9a700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c1010e0 0x7efd6c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:09.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd72d9c700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c106730 0x7efd6c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:09.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 -- 172.21.15.70:0/1466911362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c1010e0 msgr2=0x7efd6c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:09.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c1010e0 0x7efd6c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:09.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 -- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c106730 msgr2=0x7efd6c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:09.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c106730 0x7efd6c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:09.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd7259b700 1 -- 172.21.15.70:0/1466911362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680095f0 con 0x7efd6c100720 2023-10-18T12:24:09.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.212+0000 7efd72d9c700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c106730 0x7efd6c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:09.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.213+0000 7efd7259b700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 0x7efd6c198d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd68005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:09.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.213+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6801d070 con 0x7efd6c100720 2023-10-18T12:24:09.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.213+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c19e260 con 0x7efd6c100720 2023-10-18T12:24:09.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.213+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd68022470 con 0x7efd6c100720 2023-10-18T12:24:09.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.213+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68017650 con 0x7efd6c100720 2023-10-18T12:24:09.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.213+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1a5b00 con 0x7efd6c100720 2023-10-18T12:24:09.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.214+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c101e60 con 0x7efd6c100720 2023-10-18T12:24:09.224 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1348730213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T12:24:09.224 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:09.225 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:09.225 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:24:09.225 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1348730213' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-18T12:24:09.225 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: osdmap e82: 12 total, 12 up, 12 in 2023-10-18T12:24:09.225 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:09.226 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:24:09.226 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:08 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2260904379' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T12:24:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.216+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7efd680225e0 con 0x7efd6c100720 2023-10-18T12:24:09.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.217+0000 7efd637fe700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd580a14d0 0x7efd580a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:09.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.217+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8094+0+0 (secure 0 0 0) 0x7efd680ce6b0 con 0x7efd6c100720 2023-10-18T12:24:09.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.218+0000 7efd71d9a700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd580a14d0 0x7efd580a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:09.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.218+0000 7efd71d9a700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd580a14d0 0x7efd580a3990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd6c19a2e0 tx=0x7efd5c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:09.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.220+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd68097de0 con 0x7efd6c100720 2023-10-18T12:24:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1348730213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T12:24:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:09.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:09.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:24:09.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1348730213' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-18T12:24:09.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: osdmap e82: 12 total, 12 up, 12 in 2023-10-18T12:24:09.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:09.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:24:09.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:08 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2260904379' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T12:24:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1348730213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T12:24:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:09.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:24:09.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1348730213' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-18T12:24:09.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: osdmap e82: 12 total, 12 up, 12 in 2023-10-18T12:24:09.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:09.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:24:09.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:08 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2260904379' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T12:24:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:09.482+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efd6c0675a0 con 0x7efd6c100720 2023-10-18T12:24:10.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.119+0000 7efd637fe700 1 -- 172.21.15.70:0/1466911362 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7efd6809b400 con 0x7efd6c100720 2023-10-18T12:24:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd580a14d0 msgr2=0x7efd580a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd580a14d0 0x7efd580a3990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd6c19a2e0 tx=0x7efd5c009260 comp rx=0 tx=0).stop 2023-10-18T12:24:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 msgr2=0x7efd6c198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 0x7efd6c198d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd68005740 comp rx=0 tx=0).stop 2023-10-18T12:24:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 shutdown_connections 2023-10-18T12:24:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c106730 0x7efd6c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd6c100720 0x7efd6c198d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd580a14d0 0x7efd580a3990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 --2- 172.21.15.70:0/1466911362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c1010e0 0x7efd6c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.122+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 >> 172.21.15.70:0/1466911362 conn(0x7efd6c0fc1b0 msgr2=0x7efd6c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.123+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 shutdown_connections 2023-10-18T12:24:10.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.123+0000 7efd747ff700 1 -- 172.21.15.70:0/1466911362 wait complete. 2023-10-18T12:24:10.129 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 2 and data pool 3 2023-10-18T12:24:10.149 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-18T12:24:10.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-18T12:24:10.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:09 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2260904379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-18T12:24:10.218 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:09 smithi070 ceph-mon[98639]: osdmap e83: 12 total, 12 up, 12 in 2023-10-18T12:24:10.218 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:09 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1466911362' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T12:24:10.218 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:09 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T12:24:10.218 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:09 smithi070 ceph-mon[98639]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-18T12:24:10.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:09 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2260904379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-18T12:24:10.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:09 smithi186 ceph-mon[135253]: osdmap e83: 12 total, 12 up, 12 in 2023-10-18T12:24:10.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:09 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1466911362' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T12:24:10.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:09 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T12:24:10.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:09 smithi186 ceph-mon[135253]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-18T12:24:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 -- 172.21.15.70:0/3573836977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c00734c0 msgr2=0x7fc0c010caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/3573836977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c00734c0 0x7fc0c010caa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009990 tx=0x7fc0b0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 -- 172.21.15.70:0/3573836977 shutdown_connections 2023-10-18T12:24:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/3573836977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/3573836977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/3573836977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c00734c0 0x7fc0c010caa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 -- 172.21.15.70:0/3573836977 >> 172.21.15.70:0/3573836977 conn(0x7fc0c00fc1f0 msgr2=0x7fc0c00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:10.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 -- 172.21.15.70:0/3573836977 shutdown_connections 2023-10-18T12:24:10.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.361+0000 7fc0c75ff700 1 -- 172.21.15.70:0/3573836977 wait complete. 2023-10-18T12:24:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.362+0000 7fc0c75ff700 1 Processor -- start 2023-10-18T12:24:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 -- start start 2023-10-18T12:24:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c00734c0 0x7fc0c0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c019dd40 con 0x7fc0c0072ba0 2023-10-18T12:24:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c019deb0 con 0x7fc0c010cfe0 2023-10-18T12:24:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c75ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c019e070 con 0x7fc0c00734c0 2023-10-18T12:24:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c539b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c5b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.363+0000 7fc0c5b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40974/0 (socket says 172.21.15.70:40974) 2023-10-18T12:24:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c539b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35992/0 (socket says 172.21.15.70:35992) 2023-10-18T12:24:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 -- 172.21.15.70:0/2982690296 learned_addr learned my addr 172.21.15.70:0/2982690296 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c4b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c00734c0 0x7fc0c0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 -- 172.21.15.70:0/2982690296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c00734c0 msgr2=0x7fc0c0199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c00734c0 0x7fc0c0199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 -- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 msgr2=0x7fc0c0198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 -- 172.21.15.70:0/2982690296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b00095f0 con 0x7fc0c010cfe0 2023-10-18T12:24:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c539b700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c4b9a700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c00734c0 0x7fc0c0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0c5b9c700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c019d700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800ef30 tx=0x7fc0b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:10.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.364+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8018070 con 0x7fc0c010cfe0 2023-10-18T12:24:10.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.365+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c019e2f0 con 0x7fc0c010cfe0 2023-10-18T12:24:10.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.365+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b800f040 con 0x7fc0c010cfe0 2023-10-18T12:24:10.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.365+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800cd00 con 0x7fc0c010cfe0 2023-10-18T12:24:10.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.365+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c01a5b90 con 0x7fc0c010cfe0 2023-10-18T12:24:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.366+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c01081f0 con 0x7fc0c010cfe0 2023-10-18T12:24:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.369+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc0b8003a00 con 0x7fc0c010cfe0 2023-10-18T12:24:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.370+0000 7fc0b67fc700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0ac0a14d0 0x7fc0ac0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.370+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fc0b80cd1e0 con 0x7fc0c010cfe0 2023-10-18T12:24:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.371+0000 7fc0c539b700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0ac0a14d0 0x7fc0ac0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.371+0000 7fc0c539b700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0ac0a14d0 0x7fc0ac0a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011490 tx=0x7fc0bc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:10.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.376+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0b80968f0 con 0x7fc0c010cfe0 2023-10-18T12:24:10.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:09 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2260904379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-18T12:24:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:09 smithi152 ceph-mon[134767]: osdmap e83: 12 total, 12 up, 12 in 2023-10-18T12:24:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:09 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1466911362' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T12:24:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:09 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T12:24:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:09 smithi152 ceph-mon[134767]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-18T12:24:10.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.604+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fc0c0074250 con 0x7fc0c010cfe0 2023-10-18T12:24:10.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.956+0000 7fc0b67fc700 1 -- 172.21.15.70:0/2982690296 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc0b8099f10 con 0x7fc0c010cfe0 2023-10-18T12:24:10.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0ac0a14d0 msgr2=0x7fc0ac0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0ac0a14d0 0x7fc0ac0a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011490 tx=0x7fc0bc009260 comp rx=0 tx=0).stop 2023-10-18T12:24:10.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 msgr2=0x7fc0c019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:10.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c019d700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800ef30 tx=0x7fc0b800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 shutdown_connections 2023-10-18T12:24:10.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0ac0a14d0 0x7fc0ac0a3990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0c010cfe0 0x7fc0c019d700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0c0072ba0 0x7fc0c0198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 --2- 172.21.15.70:0/2982690296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0c00734c0 0x7fc0c0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 >> 172.21.15.70:0/2982690296 conn(0x7fc0c00fc1f0 msgr2=0x7fc0c00fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 shutdown_connections 2023-10-18T12:24:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:10.959+0000 7fc0c75ff700 1 -- 172.21.15.70:0/2982690296 wait complete. 2023-10-18T12:24:10.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T12:24:11.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 -- 172.21.15.70:0/1540313745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c20102140 msgr2=0x7f9c2010a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:11.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 --2- 172.21.15.70:0/1540313745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c20102140 0x7f9c2010a620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009990 tx=0x7f9c10009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:11.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 -- 172.21.15.70:0/1540313745 shutdown_connections 2023-10-18T12:24:11.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 --2- 172.21.15.70:0/1540313745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c2010ac40 0x7f9c2010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:11.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 --2- 172.21.15.70:0/1540313745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:11.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 --2- 172.21.15.70:0/1540313745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c20102140 0x7f9c2010a620 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:11.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 -- 172.21.15.70:0/1540313745 >> 172.21.15.70:0/1540313745 conn(0x7f9c200fb060 msgr2=0x7f9c200fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:11.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 -- 172.21.15.70:0/1540313745 shutdown_connections 2023-10-18T12:24:11.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.198+0000 7f9c27dff700 1 -- 172.21.15.70:0/1540313745 wait complete. 2023-10-18T12:24:11.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 Processor -- start 2023-10-18T12:24:11.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 -- start start 2023-10-18T12:24:11.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:11.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c20102140 0x7f9c20199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:11.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 0x7f9c2019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:11.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019cb30 con 0x7f9c20102140 2023-10-18T12:24:11.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019cca0 con 0x7f9c2010ac40 2023-10-18T12:24:11.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.199+0000 7f9c27dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019ce60 con 0x7f9c20101820 2023-10-18T12:24:11.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2539a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c20102140 0x7f9c20199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c25b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 0x7f9c2019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c25b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60140/0 (socket says 172.21.15.70:60140) 2023-10-18T12:24:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c25b9b700 1 -- 172.21.15.70:0/3852879105 learned_addr learned my addr 172.21.15.70:0/3852879105 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2539a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c20102140 0x7f9c20199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36012/0 (socket says 172.21.15.70:36012) 2023-10-18T12:24:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 0x7f9c2019c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40992/0 (socket says 172.21.15.70:40992) 2023-10-18T12:24:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 -- 172.21.15.70:0/3852879105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 msgr2=0x7f9c20198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 -- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c20102140 msgr2=0x7f9c20199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c20102140 0x7f9c20199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:11.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 -- 172.21.15.70:0/3852879105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100095f0 con 0x7f9c2010ac40 2023-10-18T12:24:11.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c25b9b700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:11.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c2639c700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 0x7f9c2019c530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800efc0 tx=0x7f9c1800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:11.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.200+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c18018070 con 0x7f9c2010ac40 2023-10-18T12:24:11.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.201+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c1800f040 con 0x7f9c2010ac40 2023-10-18T12:24:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.201+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800cd40 con 0x7f9c2010ac40 2023-10-18T12:24:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.201+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2019d0e0 con 0x7f9c2010ac40 2023-10-18T12:24:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.201+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201a5aa0 con 0x7f9c2010ac40 2023-10-18T12:24:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.202+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c20108110 con 0x7f9c2010ac40 2023-10-18T12:24:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.204+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9c18003a00 con 0x7f9c2010ac40 2023-10-18T12:24:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.205+0000 7f9c16ffd700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c0c0a1540 0x7f9c0c0a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.205+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8420+0+0 (secure 0 0 0) 0x7f9c180ccaf0 con 0x7f9c2010ac40 2023-10-18T12:24:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.206+0000 7f9c25b9b700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c0c0a1540 0x7f9c0c0a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.206+0000 7f9c25b9b700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c0c0a1540 0x7f9c0c0a3a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c011550 tx=0x7f9c1c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.207+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9c180960c0 con 0x7f9c2010ac40 2023-10-18T12:24:11.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: pgmap v249: 97 pgs: 19 creating+peering, 9 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: osdmap e84: 12 total, 12 up, 12 in 2023-10-18T12:24:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T12:24:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-18T12:24:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-18T12:24:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-18T12:24:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2982690296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T12:24:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T12:24:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:10 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: pgmap v249: 97 pgs: 19 creating+peering, 9 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: osdmap e84: 12 total, 12 up, 12 in 2023-10-18T12:24:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T12:24:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-18T12:24:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-18T12:24:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-18T12:24:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2982690296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T12:24:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T12:24:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:10 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:11.417+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9c200692b0 con 0x7f9c2010ac40 2023-10-18T12:24:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: pgmap v249: 97 pgs: 19 creating+peering, 9 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:24:11.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: osdmap e84: 12 total, 12 up, 12 in 2023-10-18T12:24:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T12:24:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-18T12:24:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-18T12:24:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-18T12:24:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2982690296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T12:24:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T12:24:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:11.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:10 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-18T12:24:12.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: osdmap e85: 12 total, 12 up, 12 in 2023-10-18T12:24:12.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] up:active 2023-10-18T12:24:12.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:12.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:12.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3852879105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T12:24:12.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T12:24:12.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:12.223 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:24:12.224 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:11 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-18T12:24:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: osdmap e85: 12 total, 12 up, 12 in 2023-10-18T12:24:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] up:active 2023-10-18T12:24:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:12.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:12.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3852879105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T12:24:12.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T12:24:12.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:12.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:24:12.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:11 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-18T12:24:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: osdmap e85: 12 total, 12 up, 12 in 2023-10-18T12:24:12.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] up:active 2023-10-18T12:24:12.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:12.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:12.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3852879105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T12:24:12.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T12:24:12.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:12.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:24:12.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:24:12.424 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:11 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:24:13.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.134+0000 7f9c16ffd700 1 -- 172.21.15.70:0/3852879105 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f9c180996e0 con 0x7f9c2010ac40 2023-10-18T12:24:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c0c0a1540 msgr2=0x7f9c0c0a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c0c0a1540 0x7f9c0c0a3a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c011550 tx=0x7f9c1c009260 comp rx=0 tx=0).stop 2023-10-18T12:24:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 msgr2=0x7f9c2019c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 0x7f9c2019c530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800efc0 tx=0x7f9c1800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 shutdown_connections 2023-10-18T12:24:13.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c0c0a1540 0x7f9c0c0a3a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c2010ac40 0x7f9c2019c530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c20101820 0x7f9c20198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 --2- 172.21.15.70:0/3852879105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c20102140 0x7f9c20199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 >> 172.21.15.70:0/3852879105 conn(0x7f9c200fb060 msgr2=0x7f9c201006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:13.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.141+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 shutdown_connections 2023-10-18T12:24:13.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.142+0000 7f9c27dff700 1 -- 172.21.15.70:0/3852879105 wait complete. 2023-10-18T12:24:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data_ec' created 2023-10-18T12:24:13.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-18T12:24:13.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:12 smithi186 ceph-mon[135253]: pgmap v252: 97 pgs: 27 creating+peering, 32 active+clean, 38 unknown; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2023-10-18T12:24:13.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:12 smithi186 ceph-mon[135253]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-18T12:24:13.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:12 smithi186 ceph-mon[135253]: Cluster is now healthy 2023-10-18T12:24:13.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:12 smithi186 ceph-mon[135253]: osdmap e86: 12 total, 12 up, 12 in 2023-10-18T12:24:13.226 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:12 smithi070 ceph-mon[98639]: pgmap v252: 97 pgs: 27 creating+peering, 32 active+clean, 38 unknown; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2023-10-18T12:24:13.226 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:12 smithi070 ceph-mon[98639]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-18T12:24:13.226 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:12 smithi070 ceph-mon[98639]: Cluster is now healthy 2023-10-18T12:24:13.226 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:12 smithi070 ceph-mon[98639]: osdmap e86: 12 total, 12 up, 12 in 2023-10-18T12:24:13.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 -- 172.21.15.70:0/231647915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8601010d0 msgr2=0x7fa86010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:13.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 --2- 172.21.15.70:0/231647915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8601010d0 0x7fa86010d5e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b0a0 tx=0x7fa85800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 -- 172.21.15.70:0/231647915 shutdown_connections 2023-10-18T12:24:13.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 --2- 172.21.15.70:0/231647915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8601010d0 0x7fa86010d5e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 --2- 172.21.15.70:0/231647915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa860106720 0x7fa860106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 --2- 172.21.15.70:0/231647915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa860100710 0x7fa860100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 -- 172.21.15.70:0/231647915 >> 172.21.15.70:0/231647915 conn(0x7fa8600fc200 msgr2=0x7fa8600fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:13.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 -- 172.21.15.70:0/231647915 shutdown_connections 2023-10-18T12:24:13.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.366+0000 7fa866fff700 1 -- 172.21.15.70:0/231647915 wait complete. 2023-10-18T12:24:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.367+0000 7fa866fff700 1 Processor -- start 2023-10-18T12:24:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.367+0000 7fa866fff700 1 -- start start 2023-10-18T12:24:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.367+0000 7fa866fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa860100710 0x7fa86019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.367+0000 7fa866fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8601010d0 0x7fa86019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa866fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 0x7fa860198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa866fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860199180 con 0x7fa860106720 2023-10-18T12:24:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa866fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601992f0 con 0x7fa860100710 2023-10-18T12:24:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa866fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601994b0 con 0x7fa8601010d0 2023-10-18T12:24:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 0x7fa860198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 0x7fa860198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36036/0 (socket says 172.21.15.70:36036) 2023-10-18T12:24:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 -- 172.21.15.70:0/4017744694 learned_addr learned my addr 172.21.15.70:0/4017744694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa864d9b700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa860100710 0x7fa86019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa85ffff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8601010d0 0x7fa86019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 -- 172.21.15.70:0/4017744694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8601010d0 msgr2=0x7fa86019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8601010d0 0x7fa86019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 -- 172.21.15.70:0/4017744694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa860100710 msgr2=0x7fa86019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa860100710 0x7fa86019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 -- 172.21.15.70:0/4017744694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858009d00 con 0x7fa860106720 2023-10-18T12:24:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.368+0000 7fa86559c700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 0x7fa860198b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b070 tx=0x7fa8580077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.369+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800e040 con 0x7fa860106720 2023-10-18T12:24:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.369+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8580094c0 con 0x7fa860106720 2023-10-18T12:24:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.369+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860199730 con 0x7fa860106720 2023-10-18T12:24:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.369+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580043a0 con 0x7fa860106720 2023-10-18T12:24:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.369+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601a59e0 con 0x7fa860106720 2023-10-18T12:24:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.370+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa860101e50 con 0x7fa860106720 2023-10-18T12:24:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.372+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa858004500 con 0x7fa860106720 2023-10-18T12:24:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.373+0000 7fa85dffb700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8500a14d0 0x7fa8500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:13.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.373+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8913+0+0 (secure 0 0 0) 0x7fa8580ce510 con 0x7fa860106720 2023-10-18T12:24:13.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.373+0000 7fa864d9b700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8500a14d0 0x7fa8500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:13.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.374+0000 7fa864d9b700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8500a14d0 0x7fa8500a3990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa854009be0 tx=0x7fa854009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:13.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.376+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa8580978f0 con 0x7fa860106720 2023-10-18T12:24:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:12 smithi152 ceph-mon[134767]: pgmap v252: 97 pgs: 27 creating+peering, 32 active+clean, 38 unknown; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2023-10-18T12:24:13.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:12 smithi152 ceph-mon[134767]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-18T12:24:13.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:12 smithi152 ceph-mon[134767]: Cluster is now healthy 2023-10-18T12:24:13.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:12 smithi152 ceph-mon[134767]: osdmap e86: 12 total, 12 up, 12 in 2023-10-18T12:24:13.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:13.576+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fa8600675a0 con 0x7fa860106720 2023-10-18T12:24:14.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.135+0000 7fa85dffb700 1 -- 172.21.15.70:0/4017744694 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa85809af10 con 0x7fa860106720 2023-10-18T12:24:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.138+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8500a14d0 msgr2=0x7fa8500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.138+0000 7fa866fff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8500a14d0 0x7fa8500a3990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa854009be0 tx=0x7fa854009260 comp rx=0 tx=0).stop 2023-10-18T12:24:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.138+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 msgr2=0x7fa860198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.138+0000 7fa866fff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 0x7fa860198b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa85800b070 tx=0x7fa8580077f0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 shutdown_connections 2023-10-18T12:24:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa860106720 0x7fa860198b20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8500a14d0 0x7fa8500a3990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa860100710 0x7fa86019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 --2- 172.21.15.70:0/4017744694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8601010d0 0x7fa86019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 >> 172.21.15.70:0/4017744694 conn(0x7fa8600fc200 msgr2=0x7fa86010c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 shutdown_connections 2023-10-18T12:24:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.139+0000 7fa866fff700 1 -- 172.21.15.70:0/4017744694 wait complete. 2023-10-18T12:24:14.145 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 4 allow_ec_overwrites to true 2023-10-18T12:24:14.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-18T12:24:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 -- 172.21.15.70:0/1861849220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc41010e0 msgr2=0x7fddc410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 --2- 172.21.15.70:0/1861849220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc41010e0 0x7fddc410d5f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b0a0 tx=0x7fddbc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 -- 172.21.15.70:0/1861849220 shutdown_connections 2023-10-18T12:24:14.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 --2- 172.21.15.70:0/1861849220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc41010e0 0x7fddc410d5f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 --2- 172.21.15.70:0/1861849220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc4106730 0x7fddc4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 --2- 172.21.15.70:0/1861849220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc4100720 0x7fddc4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 -- 172.21.15.70:0/1861849220 >> 172.21.15.70:0/1861849220 conn(0x7fddc40fc1b0 msgr2=0x7fddc40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:14.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 -- 172.21.15.70:0/1861849220 shutdown_connections 2023-10-18T12:24:14.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.375+0000 7fddc9dff700 1 -- 172.21.15.70:0/1861849220 wait complete. 2023-10-18T12:24:14.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.376+0000 7fddc9dff700 1 Processor -- start 2023-10-18T12:24:14.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.376+0000 7fddc9dff700 1 -- start start 2023-10-18T12:24:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 0x7fddc419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc41010e0 0x7fddc419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc9dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc4106730 0x7fddc4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc9dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4199240 con 0x7fddc4100720 2023-10-18T12:24:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc9dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41993b0 con 0x7fddc4106730 2023-10-18T12:24:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc9dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4199570 con 0x7fddc41010e0 2023-10-18T12:24:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 0x7fddc419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 0x7fddc419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36060/0 (socket says 172.21.15.70:36060) 2023-10-18T12:24:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 -- 172.21.15.70:0/527122754 learned_addr learned my addr 172.21.15.70:0/527122754 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddbbfff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc41010e0 0x7fddc419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc3fff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc4106730 0x7fddc4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 -- 172.21.15.70:0/527122754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc41010e0 msgr2=0x7fddc419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc41010e0 0x7fddc419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 -- 172.21.15.70:0/527122754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc4106730 msgr2=0x7fddc4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc4106730 0x7fddc4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:14.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.377+0000 7fddc37fe700 1 -- 172.21.15.70:0/527122754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc009d00 con 0x7fddc4100720 2023-10-18T12:24:14.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.378+0000 7fddc37fe700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 0x7fddc419ea60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e9a0 tx=0x7fddb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:14.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.378+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4009790 con 0x7fddc4100720 2023-10-18T12:24:14.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.378+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddb4010430 con 0x7fddc4100720 2023-10-18T12:24:14.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.378+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4199850 con 0x7fddc4100720 2023-10-18T12:24:14.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.378+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4018680 con 0x7fddc4100720 2023-10-18T12:24:14.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.378+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc41a5a50 con 0x7fddc4100720 2023-10-18T12:24:14.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.379+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4101e60 con 0x7fddc4100720 2023-10-18T12:24:14.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.381+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fddb40105a0 con 0x7fddc4100720 2023-10-18T12:24:14.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.382+0000 7fddc17fa700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb00a14d0 0x7fddb00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:14.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.382+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8913+0+0 (secure 0 0 0) 0x7fddb4014070 con 0x7fddc4100720 2023-10-18T12:24:14.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.383+0000 7fddbbfff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb00a14d0 0x7fddb00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:14.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.383+0000 7fddbbfff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb00a14d0 0x7fddb00a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fddc419a230 tx=0x7fddac009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:14.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.385+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fddb40966e0 con 0x7fddc4100720 2023-10-18T12:24:14.422 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:24:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: 2023-10-18T12:24:14.131+0000 7f7721fd4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:24:14.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:14 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-18T12:24:14.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:14 smithi152 ceph-mon[134767]: osdmap e87: 12 total, 12 up, 12 in 2023-10-18T12:24:14.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:14 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4017744694' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T12:24:14.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:14 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-18T12:24:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:14 smithi186 ceph-mon[135253]: osdmap e87: 12 total, 12 up, 12 in 2023-10-18T12:24:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:14 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4017744694' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T12:24:14.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:14 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-18T12:24:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:14 smithi070 ceph-mon[98639]: osdmap e87: 12 total, 12 up, 12 in 2023-10-18T12:24:14.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4017744694' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T12:24:14.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:14.632+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fddc40675a0 con 0x7fddc4100720 2023-10-18T12:24:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.145+0000 7fddc17fa700 1 -- 172.21.15.70:0/527122754 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fddb4099d00 con 0x7fddc4100720 2023-10-18T12:24:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb00a14d0 msgr2=0x7fddb00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb00a14d0 0x7fddb00a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fddc419a230 tx=0x7fddac009260 comp rx=0 tx=0).stop 2023-10-18T12:24:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 msgr2=0x7fddc419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 0x7fddc419ea60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e9a0 tx=0x7fddb400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 shutdown_connections 2023-10-18T12:24:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb00a14d0 0x7fddb00a3990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc4106730 0x7fddc4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc4100720 0x7fddc419ea60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 --2- 172.21.15.70:0/527122754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc41010e0 0x7fddc419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 >> 172.21.15.70:0/527122754 conn(0x7fddc40fc1b0 msgr2=0x7fddc40fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.149+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 shutdown_connections 2023-10-18T12:24:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.150+0000 7fddc9dff700 1 -- 172.21.15.70:0/527122754 wait complete. 2023-10-18T12:24:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:added data pool 4 to fsmap 2023-10-18T12:24:15.172 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:24:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 -- 172.21.15.70:0/3113227112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c1066b0 msgr2=0x7f6f2c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 --2- 172.21.15.70:0/3113227112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c1066b0 0x7f6f2c106a90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009990 tx=0x7f6f28009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 -- 172.21.15.70:0/3113227112 shutdown_connections 2023-10-18T12:24:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 --2- 172.21.15.70:0/3113227112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c101060 0x7f6f2c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 --2- 172.21.15.70:0/3113227112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c1066b0 0x7f6f2c106a90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 --2- 172.21.15.70:0/3113227112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 -- 172.21.15.70:0/3113227112 >> 172.21.15.70:0/3113227112 conn(0x7f6f2c0fc170 msgr2=0x7f6f2c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:15.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 -- 172.21.15.70:0/3113227112 shutdown_connections 2023-10-18T12:24:15.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.366+0000 7f6f335ff700 1 -- 172.21.15.70:0/3113227112 wait complete. 2023-10-18T12:24:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 Processor -- start 2023-10-18T12:24:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 -- start start 2023-10-18T12:24:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c101060 0x7f6f2c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f2c1066b0 0x7f6f2c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c19cb30 con 0x7f6f2c101060 2023-10-18T12:24:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c19cca0 con 0x7f6f2c1066b0 2023-10-18T12:24:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f335ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c19ce60 con 0x7f6f2c1006a0 2023-10-18T12:24:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f3139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f3139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60192/0 (socket says 172.21.15.70:60192) 2023-10-18T12:24:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f30b9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c101060 0x7f6f2c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.367+0000 7f6f3139b700 1 -- 172.21.15.70:0/1492486027 learned_addr learned my addr 172.21.15.70:0/1492486027 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f31b9c700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f2c1066b0 0x7f6f2c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f30b9a700 1 -- 172.21.15.70:0/1492486027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 msgr2=0x7f6f2c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f30b9a700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f30b9a700 1 -- 172.21.15.70:0/1492486027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f2c1066b0 msgr2=0x7f6f2c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f30b9a700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f2c1066b0 0x7f6f2c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f30b9a700 1 -- 172.21.15.70:0/1492486027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280095f0 con 0x7f6f2c101060 2023-10-18T12:24:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f3139b700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f30b9a700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c101060 0x7f6f2c199240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00d950 tx=0x7f6f1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c009930 con 0x7f6f2c101060 2023-10-18T12:24:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f1c010460 con 0x7f6f2c101060 2023-10-18T12:24:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c19d140 con 0x7f6f2c101060 2023-10-18T12:24:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00b5d0 con 0x7f6f2c101060 2023-10-18T12:24:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.368+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c1a5aa0 con 0x7f6f2c101060 2023-10-18T12:24:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.370+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c101de0 con 0x7f6f2c101060 2023-10-18T12:24:15.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.372+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6f1c010820 con 0x7f6f2c101060 2023-10-18T12:24:15.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.373+0000 7f6f227fc700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6f180a14d0 0x7f6f180a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.373+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f6f1c0ccbb0 con 0x7f6f2c101060 2023-10-18T12:24:15.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.373+0000 7f6f3139b700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6f180a14d0 0x7f6f180a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.374+0000 7f6f3139b700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6f180a14d0 0x7f6f180a3990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800bc60 tx=0x7f6f2801a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:15.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.375+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6f1c095f40 con 0x7f6f2c101060 2023-10-18T12:24:15.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:15 smithi152 ceph-mon[134767]: pgmap v255: 129 pgs: 25 creating+peering, 77 active+clean, 27 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-10-18T12:24:15.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:15 smithi152 ceph-mon[134767]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T12:24:15.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:15 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4017744694' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-18T12:24:15.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:15 smithi152 ceph-mon[134767]: osdmap e88: 12 total, 12 up, 12 in 2023-10-18T12:24:15.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:15 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/527122754' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-18T12:24:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:15 smithi186 ceph-mon[135253]: pgmap v255: 129 pgs: 25 creating+peering, 77 active+clean, 27 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-10-18T12:24:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:15 smithi186 ceph-mon[135253]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T12:24:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:15 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4017744694' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-18T12:24:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:15 smithi186 ceph-mon[135253]: osdmap e88: 12 total, 12 up, 12 in 2023-10-18T12:24:15.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:15 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/527122754' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-18T12:24:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:15 smithi070 ceph-mon[98639]: pgmap v255: 129 pgs: 25 creating+peering, 77 active+clean, 27 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-10-18T12:24:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:15 smithi070 ceph-mon[98639]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T12:24:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:15 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4017744694' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-18T12:24:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:15 smithi070 ceph-mon[98639]: osdmap e88: 12 total, 12 up, 12 in 2023-10-18T12:24:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:15 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/527122754' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-18T12:24:15.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.620+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f2c067530 con 0x7f6f2c101060 2023-10-18T12:24:15.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.621+0000 7f6f227fc700 1 -- 172.21.15.70:0/1492486027 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11264 (secure 0 0 0) 0x7f6f1c021020 con 0x7f6f2c101060 2023-10-18T12:24:15.624 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:15.624 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24719,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:14.634476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T12:24:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6f180a14d0 msgr2=0x7f6f180a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6f180a14d0 0x7f6f180a3990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800bc60 tx=0x7f6f2801a040 comp rx=0 tx=0).stop 2023-10-18T12:24:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c101060 msgr2=0x7f6f2c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c101060 0x7f6f2c199240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00d950 tx=0x7f6f1c00dc60 comp rx=0 tx=0).stop 2023-10-18T12:24:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 shutdown_connections 2023-10-18T12:24:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6f180a14d0 0x7f6f180a3990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f2c1066b0 0x7f6f2c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f2c1006a0 0x7f6f2c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 --2- 172.21.15.70:0/1492486027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f2c101060 0x7f6f2c199240 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.623+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 >> 172.21.15.70:0/1492486027 conn(0x7f6f2c0fc170 msgr2=0x7f6f2c10bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.624+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 shutdown_connections 2023-10-18T12:24:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.624+0000 7f6f335ff700 1 -- 172.21.15.70:0/1492486027 wait complete. 2023-10-18T12:24:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2023-10-18T12:24:15.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:15.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 -- 172.21.15.70:0/3266189444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85081066b0 msgr2=0x7f8508106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 --2- 172.21.15.70:0/3266189444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85081066b0 0x7f8508106a90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8504009990 tx=0x7f8504009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 -- 172.21.15.70:0/3266189444 shutdown_connections 2023-10-18T12:24:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 --2- 172.21.15.70:0/3266189444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8508101060 0x7f850810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 --2- 172.21.15.70:0/3266189444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85081066b0 0x7f8508106a90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 --2- 172.21.15.70:0/3266189444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 0x7f8508100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 -- 172.21.15.70:0/3266189444 >> 172.21.15.70:0/3266189444 conn(0x7f85080fc170 msgr2=0x7f85080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 -- 172.21.15.70:0/3266189444 shutdown_connections 2023-10-18T12:24:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.843+0000 7f850f9ff700 1 -- 172.21.15.70:0/3266189444 wait complete. 2023-10-18T12:24:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.844+0000 7f850f9ff700 1 Processor -- start 2023-10-18T12:24:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.844+0000 7f850f9ff700 1 -- start start 2023-10-18T12:24:15.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850f9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 0x7f850819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850f9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 0x7f850819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85081066b0 0x7f8508198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850f9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8508199170 con 0x7f8508101060 2023-10-18T12:24:15.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850f9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081992e0 con 0x7f85081006a0 2023-10-18T12:24:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081994a0 con 0x7f85081066b0 2023-10-18T12:24:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 0x7f850819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 0x7f850819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36098/0 (socket says 172.21.15.70:36098) 2023-10-18T12:24:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 -- 172.21.15.70:0/1920707621 learned_addr learned my addr 172.21.15.70:0/1920707621 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850d79b700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 0x7f850819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850df9c700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85081066b0 0x7f8508198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 -- 172.21.15.70:0/1920707621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85081066b0 msgr2=0x7f8508198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85081066b0 0x7f8508198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 -- 172.21.15.70:0/1920707621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 msgr2=0x7f850819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 0x7f850819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:15.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 -- 172.21.15.70:0/1920707621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040095f0 con 0x7f8508101060 2023-10-18T12:24:15.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850d79b700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 0x7f850819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:15.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850df9c700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85081066b0 0x7f8508198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:15.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.845+0000 7f850cf9a700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 0x7f850819ef20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e9a0 tx=0x7f84f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:15.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.846+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8009790 con 0x7f8508101060 2023-10-18T12:24:15.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.846+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f800cbd0 con 0x7f8508101060 2023-10-18T12:24:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.846+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8508199780 con 0x7f8508101060 2023-10-18T12:24:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.846+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8018680 con 0x7f8508101060 2023-10-18T12:24:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.846+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85081a59d0 con 0x7f8508101060 2023-10-18T12:24:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.847+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8508101de0 con 0x7f8508101060 2023-10-18T12:24:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.849+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f84f800cd40 con 0x7f8508101060 2023-10-18T12:24:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.850+0000 7f84fe7fc700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84f40a1550 0x7f84f40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.850+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f84f8014070 con 0x7f8508101060 2023-10-18T12:24:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.850+0000 7f850d79b700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84f40a1550 0x7f84f40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.851+0000 7f850d79b700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84f40a1550 0x7f84f40a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8504006010 tx=0x7f850400b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:15.852+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f84f8096b50 con 0x7f8508101060 2023-10-18T12:24:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.054+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85080674e0 con 0x7f8508101060 2023-10-18T12:24:16.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.055+0000 7f84fe7fc700 1 -- 172.21.15.70:0/1920707621 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f84f809a170 con 0x7f8508101060 2023-10-18T12:24:16.059 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:16.059 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:16.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84f40a1550 msgr2=0x7f84f40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84f40a1550 0x7f84f40a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8504006010 tx=0x7f850400b260 comp rx=0 tx=0).stop 2023-10-18T12:24:16.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 msgr2=0x7f850819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 0x7f850819ef20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e9a0 tx=0x7f84f800ed60 comp rx=0 tx=0).stop 2023-10-18T12:24:16.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 shutdown_connections 2023-10-18T12:24:16.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85081066b0 0x7f8508198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84f40a1550 0x7f84f40a3a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85081006a0 0x7f850819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 --2- 172.21.15.70:0/1920707621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8508101060 0x7f850819ef20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 >> 172.21.15.70:0/1920707621 conn(0x7f85080fc170 msgr2=0x7f850810bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:16.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 shutdown_connections 2023-10-18T12:24:16.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.058+0000 7f850f9ff700 1 -- 172.21.15.70:0/1920707621 wait complete. 2023-10-18T12:24:16.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 -- 172.21.15.70:0/2851138884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 msgr2=0x7fc8b0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/2851138884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b0100b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc898009990 tx=0x7fc898009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 -- 172.21.15.70:0/2851138884 shutdown_connections 2023-10-18T12:24:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/2851138884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b0101060 0x7fc8b010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/2851138884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b01066b0 0x7fc8b0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/2851138884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b0100b20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 -- 172.21.15.70:0/2851138884 >> 172.21.15.70:0/2851138884 conn(0x7fc8b00fc170 msgr2=0x7fc8b00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 -- 172.21.15.70:0/2851138884 shutdown_connections 2023-10-18T12:24:16.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.279+0000 7fc8b5bff700 1 -- 172.21.15.70:0/2851138884 wait complete. 2023-10-18T12:24:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.280+0000 7fc8b5bff700 1 Processor -- start 2023-10-18T12:24:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 -- start start 2023-10-18T12:24:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b0101060 0x7fc8b019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b01066b0 0x7fc8b0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0199120 con 0x7fc8b01006a0 2023-10-18T12:24:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0199290 con 0x7fc8b0101060 2023-10-18T12:24:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0199450 con 0x7fc8b01066b0 2023-10-18T12:24:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8af7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8af7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36126/0 (socket says 172.21.15.70:36126) 2023-10-18T12:24:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8af7fe700 1 -- 172.21.15.70:0/1146703730 learned_addr learned my addr 172.21.15.70:0/1146703730 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8affff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b01066b0 0x7fc8b0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.281+0000 7fc8a7fff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b0101060 0x7fc8b019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8af7fe700 1 -- 172.21.15.70:0/1146703730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b01066b0 msgr2=0x7fc8b0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8af7fe700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b01066b0 0x7fc8b0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8af7fe700 1 -- 172.21.15.70:0/1146703730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b0101060 msgr2=0x7fc8b019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8af7fe700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b0101060 0x7fc8b019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8af7fe700 1 -- 172.21.15.70:0/1146703730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8980095f0 con 0x7fc8b01006a0 2023-10-18T12:24:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8af7fe700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b019e990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9a0 tx=0x7fc8a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0009790 con 0x7fc8b01006a0 2023-10-18T12:24:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a0010430 con 0x7fc8b01006a0 2023-10-18T12:24:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b0199730 con 0x7fc8b01006a0 2023-10-18T12:24:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0018680 con 0x7fc8b01006a0 2023-10-18T12:24:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.282+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b01a5980 con 0x7fc8b01006a0 2023-10-18T12:24:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.283+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b0101de0 con 0x7fc8b01006a0 2023-10-18T12:24:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.285+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc8a00105a0 con 0x7fc8b01006a0 2023-10-18T12:24:16.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.287+0000 7fc8ad7fa700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc89c0a1550 0x7fc89c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.287+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fc8a0014070 con 0x7fc8b01006a0 2023-10-18T12:24:16.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.287+0000 7fc8a7fff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc89c0a1550 0x7fc89c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.288+0000 7fc8a7fff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc89c0a1550 0x7fc89c0a3a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc89800b2e0 tx=0x7fc898005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.291+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc8a0096480 con 0x7fc8b01006a0 2023-10-18T12:24:16.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:16 smithi070 ceph-mon[98639]: osdmap e89: 12 total, 12 up, 12 in 2023-10-18T12:24:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:16 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/527122754' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-18T12:24:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:16 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:16 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1492486027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:16 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1920707621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:16.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:16 smithi152 ceph-mon[134767]: osdmap e89: 12 total, 12 up, 12 in 2023-10-18T12:24:16.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:16 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/527122754' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-18T12:24:16.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:16 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:16.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:16 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1492486027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:16.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:16 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1920707621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:16 smithi186 ceph-mon[135253]: osdmap e89: 12 total, 12 up, 12 in 2023-10-18T12:24:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:16 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/527122754' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-18T12:24:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:16 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:16 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1492486027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:16 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1920707621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:16.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.496+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8b0110620 con 0x7fc8b01006a0 2023-10-18T12:24:16.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.497+0000 7fc8ad7fa700 1 -- 172.21.15.70:0/1146703730 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7fc8a0099aa0 con 0x7fc8b01006a0 2023-10-18T12:24:16.500 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:16.501 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc89c0a1550 msgr2=0x7fc89c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc89c0a1550 0x7fc89c0a3a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc89800b2e0 tx=0x7fc898005960 comp rx=0 tx=0).stop 2023-10-18T12:24:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 msgr2=0x7fc8b019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b019e990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9a0 tx=0x7fc8a000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 shutdown_connections 2023-10-18T12:24:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b01066b0 0x7fc8b0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8b01006a0 0x7fc8b019e990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc89c0a1550 0x7fc89c0a3a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 --2- 172.21.15.70:0/1146703730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b0101060 0x7fc8b019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 >> 172.21.15.70:0/1146703730 conn(0x7fc8b00fc170 msgr2=0x7fc8b0072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:16.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 shutdown_connections 2023-10-18T12:24:16.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.500+0000 7fc8b5bff700 1 -- 172.21.15.70:0/1146703730 wait complete. 2023-10-18T12:24:16.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:24:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.732+0000 7f61f25ff700 1 -- 172.21.15.70:0/2354216810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec072bc0 msgr2=0x7f61ec072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.732+0000 7f61f25ff700 1 --2- 172.21.15.70:0/2354216810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec072bc0 0x7f61ec072fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009990 tx=0x7f61dc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 -- 172.21.15.70:0/2354216810 shutdown_connections 2023-10-18T12:24:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 --2- 172.21.15.70:0/2354216810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61ec10cf50 0x7f61ec10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 --2- 172.21.15.70:0/2354216810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec072bc0 0x7f61ec072fa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 --2- 172.21.15.70:0/2354216810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec0734e0 0x7f61ec10ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 -- 172.21.15.70:0/2354216810 >> 172.21.15.70:0/2354216810 conn(0x7f61ec0fc1c0 msgr2=0x7f61ec0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 -- 172.21.15.70:0/2354216810 shutdown_connections 2023-10-18T12:24:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.733+0000 7f61f25ff700 1 -- 172.21.15.70:0/2354216810 wait complete. 2023-10-18T12:24:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 Processor -- start 2023-10-18T12:24:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 -- start start 2023-10-18T12:24:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec072bc0 0x7f61ec198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61ec0734e0 0x7f61ec199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 0x7f61ec19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec19dcd0 con 0x7f61ec10cf50 2023-10-18T12:24:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec19de40 con 0x7f61ec0734e0 2023-10-18T12:24:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f25ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec19e000 con 0x7f61ec072bc0 2023-10-18T12:24:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f0b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 0x7f61ec19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f0b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 0x7f61ec19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36144/0 (socket says 172.21.15.70:36144) 2023-10-18T12:24:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61ebfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec072bc0 0x7f61ec198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.734+0000 7f61f0b9c700 1 -- 172.21.15.70:0/4134667916 learned_addr learned my addr 172.21.15.70:0/4134667916 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61e3fff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61ec0734e0 0x7f61ec199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61ebfff700 1 -- 172.21.15.70:0/4134667916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61ec0734e0 msgr2=0x7f61ec199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61ebfff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61ec0734e0 0x7f61ec199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61ebfff700 1 -- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 msgr2=0x7f61ec19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61ebfff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 0x7f61ec19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61ebfff700 1 -- 172.21.15.70:0/4134667916 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0095f0 con 0x7f61ec072bc0 2023-10-18T12:24:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61f0b9c700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 0x7f61ec19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61ebfff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec072bc0 0x7f61ec198e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006010 tx=0x7f61dc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc01d070 con 0x7f61ec072bc0 2023-10-18T12:24:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec19e280 con 0x7f61ec072bc0 2023-10-18T12:24:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61dc005050 con 0x7f61ec072bc0 2023-10-18T12:24:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.735+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc004a70 con 0x7f61ec072bc0 2023-10-18T12:24:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.736+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec1a5b20 con 0x7f61ec072bc0 2023-10-18T12:24:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.736+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec108160 con 0x7f61ec072bc0 2023-10-18T12:24:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.739+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f61dc005450 con 0x7f61ec072bc0 2023-10-18T12:24:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.740+0000 7f61e9ffb700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61d80a14d0 0x7f61d80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.740+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f61dc0ce1b0 con 0x7f61ec072bc0 2023-10-18T12:24:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.740+0000 7f61e3fff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61d80a14d0 0x7f61d80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.741+0000 7f61e3fff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61d80a14d0 0x7f61d80a3990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61ec19a300 tx=0x7f61d400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.742+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f61dc097570 con 0x7f61ec072bc0 2023-10-18T12:24:16.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.989+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61ec074270 con 0x7f61ec072bc0 2023-10-18T12:24:16.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.990+0000 7f61e9ffb700 1 -- 172.21.15.70:0/4134667916 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11264 (secure 0 0 0) 0x7f61dc09ab90 con 0x7f61ec072bc0 2023-10-18T12:24:16.993 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:16.993 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24719,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:14.634476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T12:24:16.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61d80a14d0 msgr2=0x7f61d80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61d80a14d0 0x7f61d80a3990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61ec19a300 tx=0x7f61d400b410 comp rx=0 tx=0).stop 2023-10-18T12:24:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec072bc0 msgr2=0x7f61ec198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec072bc0 0x7f61ec198e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006010 tx=0x7f61dc00f690 comp rx=0 tx=0).stop 2023-10-18T12:24:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 shutdown_connections 2023-10-18T12:24:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61ec10cf50 0x7f61ec19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec072bc0 0x7f61ec198e00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61d80a14d0 0x7f61d80a3990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 --2- 172.21.15.70:0/4134667916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61ec0734e0 0x7f61ec199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 >> 172.21.15.70:0/4134667916 conn(0x7f61ec0fc1c0 msgr2=0x7f61ec0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 shutdown_connections 2023-10-18T12:24:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:16.992+0000 7f61f25ff700 1 -- 172.21.15.70:0/4134667916 wait complete. 2023-10-18T12:24:16.998 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2023-10-18T12:24:17.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 -- 172.21.15.70:0/1478165468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4073540 msgr2=0x7f19d410c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 --2- 172.21.15.70:0/1478165468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4073540 0x7f19d410c9a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19bc009990 tx=0x7f19bc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 -- 172.21.15.70:0/1478165468 shutdown_connections 2023-10-18T12:24:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 --2- 172.21.15.70:0/1478165468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19d410cee0 0x7f19d410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 --2- 172.21.15.70:0/1478165468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4072c20 0x7f19d4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 --2- 172.21.15.70:0/1478165468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4073540 0x7f19d410c9a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 -- 172.21.15.70:0/1478165468 >> 172.21.15.70:0/1478165468 conn(0x7f19d40fc170 msgr2=0x7f19d40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 -- 172.21.15.70:0/1478165468 shutdown_connections 2023-10-18T12:24:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.212+0000 7f19d99ff700 1 -- 172.21.15.70:0/1478165468 wait complete. 2023-10-18T12:24:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 Processor -- start 2023-10-18T12:24:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 -- start start 2023-10-18T12:24:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 0x7f19d419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19d4073540 0x7f19d419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d410cee0 0x7f19d4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4199200 con 0x7f19d4072c20 2023-10-18T12:24:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4199370 con 0x7f19d4073540 2023-10-18T12:24:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4199530 con 0x7f19d410cee0 2023-10-18T12:24:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 0x7f19d419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 0x7f19d419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36160/0 (socket says 172.21.15.70:36160) 2023-10-18T12:24:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.213+0000 7f19d2ffd700 1 -- 172.21.15.70:0/3495722855 learned_addr learned my addr 172.21.15.70:0/3495722855 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19cbfff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19d4073540 0x7f19d419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d37fe700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d410cee0 0x7f19d4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d2ffd700 1 -- 172.21.15.70:0/3495722855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d410cee0 msgr2=0x7f19d4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d2ffd700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d410cee0 0x7f19d4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d2ffd700 1 -- 172.21.15.70:0/3495722855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19d4073540 msgr2=0x7f19d419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d2ffd700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19d4073540 0x7f19d419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d2ffd700 1 -- 172.21.15.70:0/3495722855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0095f0 con 0x7f19d4072c20 2023-10-18T12:24:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d37fe700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d410cee0 0x7f19d4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:17.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d2ffd700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 0x7f19d419ea70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19c400e9a0 tx=0x7f19c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4009790 con 0x7f19d4072c20 2023-10-18T12:24:17.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c400cbd0 con 0x7f19d4072c20 2023-10-18T12:24:17.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d4199810 con 0x7f19d4072c20 2023-10-18T12:24:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4018680 con 0x7f19d4072c20 2023-10-18T12:24:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.214+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d41a5a60 con 0x7f19d4072c20 2023-10-18T12:24:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.215+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d41080f0 con 0x7f19d4072c20 2023-10-18T12:24:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.217+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f19c400cd40 con 0x7f19d4072c20 2023-10-18T12:24:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.219+0000 7f19d89fd700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19c00a14d0 0x7f19c00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.219+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f19c4014070 con 0x7f19d4072c20 2023-10-18T12:24:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.219+0000 7f19cbfff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19c00a14d0 0x7f19c00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.220+0000 7f19cbfff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19c00a14d0 0x7f19c00a3990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0098e0 tx=0x7f19bc005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.222+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f19c4052100 con 0x7f19d4072c20 2023-10-18T12:24:17.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:17 smithi070 ceph-mon[98639]: pgmap v258: 129 pgs: 7 creating+peering, 101 active+clean, 21 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-10-18T12:24:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:17 smithi070 ceph-mon[98639]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-18T12:24:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:17 smithi070 ceph-mon[98639]: Cluster is now healthy 2023-10-18T12:24:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:17 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1146703730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:17 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4134667916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:17.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:17 smithi152 ceph-mon[134767]: pgmap v258: 129 pgs: 7 creating+peering, 101 active+clean, 21 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-10-18T12:24:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:17 smithi152 ceph-mon[134767]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-18T12:24:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:17 smithi152 ceph-mon[134767]: Cluster is now healthy 2023-10-18T12:24:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:17 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1146703730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:17.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:17 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4134667916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:17.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.425+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19d4199ff0 con 0x7f19d4072c20 2023-10-18T12:24:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.427+0000 7f19d89fd700 1 -- 172.21.15.70:0/3495722855 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f19c4096660 con 0x7f19d4072c20 2023-10-18T12:24:17.430 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:17.430 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.429+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19c00a14d0 msgr2=0x7f19c00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.429+0000 7f19d99ff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19c00a14d0 0x7f19c00a3990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0098e0 tx=0x7f19bc005960 comp rx=0 tx=0).stop 2023-10-18T12:24:17.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.429+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 msgr2=0x7f19d419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.429+0000 7f19d99ff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 0x7f19d419ea70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19c400e9a0 tx=0x7f19c400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 shutdown_connections 2023-10-18T12:24:17.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d410cee0 0x7f19d4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19d4072c20 0x7f19d419ea70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19c00a14d0 0x7f19c00a3990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 --2- 172.21.15.70:0/3495722855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19d4073540 0x7f19d419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 >> 172.21.15.70:0/3495722855 conn(0x7f19d40fc170 msgr2=0x7f19d40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 shutdown_connections 2023-10-18T12:24:17.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.430+0000 7f19d99ff700 1 -- 172.21.15.70:0/3495722855 wait complete. 2023-10-18T12:24:17.452 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T12:24:09.485042+0000', 'modified': '2023-10-18T12:24:14.634476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6839/1810263953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1810263953}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1810263953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-18T12:24:17.452 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:24:17.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T12:24:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:17 smithi186 ceph-mon[135253]: pgmap v258: 129 pgs: 7 creating+peering, 101 active+clean, 21 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-10-18T12:24:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:17 smithi186 ceph-mon[135253]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-18T12:24:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:17 smithi186 ceph-mon[135253]: Cluster is now healthy 2023-10-18T12:24:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:17 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1146703730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:17 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4134667916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 -- 172.21.15.70:0/2931789439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc810cee0 msgr2=0x7f2dc810f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 --2- 172.21.15.70:0/2931789439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc810cee0 0x7f2dc810f350 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b0a0 tx=0x7f2dc000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 -- 172.21.15.70:0/2931789439 shutdown_connections 2023-10-18T12:24:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 --2- 172.21.15.70:0/2931789439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc810cee0 0x7f2dc810f350 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 --2- 172.21.15.70:0/2931789439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 0x7f2dc8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 --2- 172.21.15.70:0/2931789439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc8073540 0x7f2dc810c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 -- 172.21.15.70:0/2931789439 >> 172.21.15.70:0/2931789439 conn(0x7f2dc80fc170 msgr2=0x7f2dc80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 -- 172.21.15.70:0/2931789439 shutdown_connections 2023-10-18T12:24:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.649+0000 7f2dccdff700 1 -- 172.21.15.70:0/2931789439 wait complete. 2023-10-18T12:24:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.650+0000 7f2dccdff700 1 Processor -- start 2023-10-18T12:24:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.650+0000 7f2dccdff700 1 -- start start 2023-10-18T12:24:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dccdff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 0x7f2dc8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dccdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 0x7f2dc8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dccdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc810cee0 0x7f2dc819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dccdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc819dd90 con 0x7f2dc8073540 2023-10-18T12:24:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dccdff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc819df00 con 0x7f2dc8072c20 2023-10-18T12:24:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dccdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc819e0c0 con 0x7f2dc810cee0 2023-10-18T12:24:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 0x7f2dc8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 0x7f2dc8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36174/0 (socket says 172.21.15.70:36174) 2023-10-18T12:24:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/852067294 learned_addr learned my addr 172.21.15.70:0/852067294 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc659c700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 0x7f2dc8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc6d9d700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc810cee0 0x7f2dc819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/852067294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc810cee0 msgr2=0x7f2dc819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc810cee0 0x7f2dc819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/852067294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 msgr2=0x7f2dc8198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 0x7f2dc8198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.651+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/852067294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009520 con 0x7f2dc8073540 2023-10-18T12:24:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2dc659c700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 0x7f2dc8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 0x7f2dc8199400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00e9a0 tx=0x7f2dbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2db77fe700 1 -- 172.21.15.70:0/852067294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc009790 con 0x7f2dc8073540 2023-10-18T12:24:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2db77fe700 1 -- 172.21.15.70:0/852067294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dbc010430 con 0x7f2dc8073540 2023-10-18T12:24:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2db77fe700 1 -- 172.21.15.70:0/852067294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc018680 con 0x7f2dc8073540 2023-10-18T12:24:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc0009d00 con 0x7f2dc8073540 2023-10-18T12:24:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.652+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc81a5a00 con 0x7f2dc8073540 2023-10-18T12:24:17.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.655+0000 7f2db77fe700 1 -- 172.21.15.70:0/852067294 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2dbc0105a0 con 0x7f2dc8073540 2023-10-18T12:24:17.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.657+0000 7f2de07b9b80 1 Processor -- start 2023-10-18T12:24:17.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.657+0000 7f2de07b9b80 1 -- start start 2023-10-18T12:24:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.657+0000 7f2db77fe700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2db00a1550 0x7f2db00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.657+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e705cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.657+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e7057040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e7057a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2db77fe700 1 -- 172.21.15.70:0/852067294 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2dbc014070 con 0x7f2dc8073540 2023-10-18T12:24:17.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6fca690 con 0x5555e7057580 2023-10-18T12:24:17.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6fb0c10 con 0x5555e705d4b0 2023-10-18T12:24:17.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6fae530 con 0x5555e705cb90 2023-10-18T12:24:17.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc6d9d700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2db00a1550 0x7f2db00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e7057040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e705cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e7057040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:41142/0 (socket says 172.21.15.70:41142) 2023-10-18T12:24:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/3945635987 learned_addr learned my addr 172.21.15.70:0/3945635987 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc6d9d700 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e7057a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/3945635987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 msgr2=0x5555e705cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e705cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/3945635987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 msgr2=0x5555e7057a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e7057a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/3945635987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555e6f673a0 con 0x5555e705d4b0 2023-10-18T12:24:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc659c700 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e705cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.658+0000 7f2dc6d9d700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2db00a1550 0x7f2db00a3a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b780 tx=0x7f2dc0007d90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e7057040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009230 tx=0x7f2dbc099b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3555a4303f0540 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2db4ff9700 1 -- 172.21.15.70:0/3945635987 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc09a140 con 0x5555e705d4b0 2023-10-18T12:24:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2db4ff9700 1 -- 172.21.15.70:0/3945635987 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dbc09e290 con 0x5555e705d4b0 2023-10-18T12:24:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2db4ff9700 1 -- 172.21.15.70:0/3945635987 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc099890 con 0x5555e705d4b0 2023-10-18T12:24:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3945635987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 msgr2=0x5555e7057040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e7057040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009230 tx=0x7f2dbc099b20 comp rx=0 tx=0).stop 2023-10-18T12:24:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3945635987 shutdown_connections 2023-10-18T12:24:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e7057a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.659+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e705cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.660+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3945635987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e7057040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.660+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3945635987 >> 172.21.15.70:0/3945635987 conn(0x5555e6fcb6d0 msgr2=0x5555e7051b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.660+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3945635987 shutdown_connections 2023-10-18T12:24:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.660+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3945635987 wait complete. 2023-10-18T12:24:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.660+0000 7f2de07b9b80 1 -- start start 2023-10-18T12:24:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.660+0000 7f2de07b9b80 1 Processor -- start 2023-10-18T12:24:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e7057580 0x5555e70634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 0x5555e70607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6fa9e20 con 0x5555e705d4b0 2023-10-18T12:24:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6f466c0 con 0x5555e705cb90 2023-10-18T12:24:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6f46360 con 0x5555e7057580 2023-10-18T12:24:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2de07b9b80 10 client.0 _ll_register_callbacks cb 0x5555e6e90a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:24:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 0x5555e70607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e7057580 0x5555e70634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e7057580 0x5555e70634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60324/0 (socket says 172.21.15.70:60324) 2023-10-18T12:24:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc659c700 1 -- 172.21.15.70:0/4197550935 learned_addr learned my addr 172.21.15.70:0/4197550935 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 0x5555e70607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:41158/0 (socket says 172.21.15.70:41158) 2023-10-18T12:24:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/4197550935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e7057580 msgr2=0x5555e70634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e7057580 0x5555e70634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/4197550935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 msgr2=0x5555e7060ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.661+0000 7f2dc5d9b700 1 -- 172.21.15.70:0/4197550935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db8009e20 con 0x5555e705cb90 2023-10-18T12:24:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 0x5555e70607b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5555e7053780 tx=0x7f2dbc0a24d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d62e4edd9bd7e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dadffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:24:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dadffb700 1 -- 172.21.15.70:0/4197550935 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc09a140 con 0x5555e705cb90 2023-10-18T12:24:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555e6faeea0 con 0x5555e705cb90 2023-10-18T12:24:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dadffb700 1 -- 172.21.15.70:0/4197550935 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2dbc010880 con 0x5555e705cb90 2023-10-18T12:24:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dadffb700 1 -- 172.21.15.70:0/4197550935 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dbc0b82c0 con 0x5555e705cb90 2023-10-18T12:24:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2de07b9b80 10 client.24851 fetch_fsmap learned FSMap version 17 2023-10-18T12:24:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f2dc81a5a00 con 0x5555e705cb90 2023-10-18T12:24:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dadffb700 1 -- 172.21.15.70:0/4197550935 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555e6faeea0 con 0x5555e705cb90 2023-10-18T12:24:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.662+0000 7f2dadffb700 1 -- 172.21.15.70:0/4197550935 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2dbc099130 con 0x5555e705cb90 2023-10-18T12:24:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2de07b9b80 10 client.24851 fetch_fsmap finished waiting for FSMap version 17 2023-10-18T12:24:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2de07b9b80 10 client.24851 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T12:24:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2de07b9b80 20 client.24851 populate_metadata read hostname 'smithi070' 2023-10-18T12:24:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e7070b40 0x5555e706f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2de07b9b80 4 client.24851 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:24:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555e7026860 con 0x5555e7070b40 2023-10-18T12:24:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.663+0000 7f2dc659c700 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e7070b40 0x5555e706f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.664+0000 7f2dadffb700 10 client.24851 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:24:17.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.664+0000 7f2dc659c700 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e7070b40 0x5555e706f7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a45a89c9d8d7be server_cookie=d65e2cc7a826b631 in_seq=0 out_seq=0 2023-10-18T12:24:17.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.664+0000 7f2dadffb700 1 -- 172.21.15.70:0/4197550935 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555e7026860 con 0x5555e7070b40 2023-10-18T12:24:17.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.664+0000 7f2dadffb700 10 client.24851 handle_command_reply: tid=0 2023-10-18T12:24:17.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.664+0000 7f2de07b9b80 1 client.24851 shutdown 2023-10-18T12:24:17.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 msgr2=0x5555e70607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 0x5555e70607b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5555e7053780 tx=0x7f2dbc0a24d0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 shutdown_connections 2023-10-18T12:24:17.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e7070b40 0x5555e706f7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e7057580 0x5555e70634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/4197550935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705cb90 0x5555e70607b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2dadffb700 0 client.24851 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:24:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 >> 172.21.15.70:0/4197550935 conn(0x5555e6fcb6d0 msgr2=0x5555e7051a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 shutdown_connections 2023-10-18T12:24:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 1 -- 172.21.15.70:0/4197550935 wait complete. 2023-10-18T12:24:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.665+0000 7f2de07b9b80 20 client.24851 trim_cache size 0 max 16384 2023-10-18T12:24:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 Processor -- start 2023-10-18T12:24:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 -- start start 2023-10-18T12:24:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 0x5555e7057040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e706ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e70520e0 con 0x5555e705d4b0 2023-10-18T12:24:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6e2a8d0 con 0x5555e7057580 2023-10-18T12:24:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e6f61720 con 0x5555e705cb90 2023-10-18T12:24:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 0x5555e7057040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 0x5555e7057040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:41160/0 (socket says 172.21.15.70:41160) 2023-10-18T12:24:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 -- 172.21.15.70:0/1295950787 learned_addr learned my addr 172.21.15.70:0/1295950787 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e706ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 -- 172.21.15.70:0/1295950787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 msgr2=0x5555e706ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e706ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 -- 172.21.15.70:0/1295950787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 msgr2=0x5555e7060090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 -- 172.21.15.70:0/1295950787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555e6dd4d90 con 0x5555e7057580 2023-10-18T12:24:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.684+0000 7f2dc659c700 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 0x5555e7057040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5555e70616c0 tx=0x7f2db8005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cfba787277b303d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2db57fa700 1 -- 172.21.15.70:0/1295950787 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db801e070 con 0x5555e7057580 2023-10-18T12:24:17.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2db57fa700 1 -- 172.21.15.70:0/1295950787 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db8018040 con 0x5555e7057580 2023-10-18T12:24:17.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2db57fa700 1 -- 172.21.15.70:0/1295950787 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db80139d0 con 0x5555e7057580 2023-10-18T12:24:17.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 -- 172.21.15.70:0/1295950787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 msgr2=0x5555e7057040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 0x5555e7057040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5555e70616c0 tx=0x7f2db8005e40 comp rx=0 tx=0).stop 2023-10-18T12:24:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 -- 172.21.15.70:0/1295950787 shutdown_connections 2023-10-18T12:24:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e705d4b0 0x5555e7060090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e7057580 0x5555e7057040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/1295950787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e706ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 -- 172.21.15.70:0/1295950787 >> 172.21.15.70:0/1295950787 conn(0x5555e6fcb6d0 msgr2=0x5555e706f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 -- 172.21.15.70:0/1295950787 shutdown_connections 2023-10-18T12:24:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.685+0000 7f2de07b9b80 1 -- 172.21.15.70:0/1295950787 wait complete. 2023-10-18T12:24:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.686+0000 7f2de07b9b80 1 -- start start 2023-10-18T12:24:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.686+0000 7f2de07b9b80 1 Processor -- start 2023-10-18T12:24:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e705fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e7060210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e70c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e70c4a70 con 0x5555e7057580 2023-10-18T12:24:17.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e70c4be0 con 0x5555e705d4b0 2023-10-18T12:24:17.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e70c4da0 con 0x5555e705cb90 2023-10-18T12:24:17.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2de07b9b80 10 client.0 _ll_register_callbacks cb 0x7f2db8009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:24:17.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e705fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e705fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36220/0 (socket says 172.21.15.70:36220) 2023-10-18T12:24:17.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 -- 172.21.15.70:0/3588025608 learned_addr learned my addr 172.21.15.70:0/3588025608 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:17.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc6d9d700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e70c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e7060210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 -- 172.21.15.70:0/3588025608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 msgr2=0x5555e7060210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e7060210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 -- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 msgr2=0x5555e70c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:24:17.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e70c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.687+0000 7f2dc659c700 1 -- 172.21.15.70:0/3588025608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db8013040 con 0x5555e7057580 2023-10-18T12:24:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2dc659c700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e705fcd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2db801ec60 tx=0x7f2db80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e8b754e1194aec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2dadffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:24:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2dadffb700 1 -- 172.21.15.70:0/3588025608 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db801e440 con 0x5555e7057580 2023-10-18T12:24:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2dadffb700 1 -- 172.21.15.70:0/3588025608 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2db80136f0 con 0x5555e7057580 2023-10-18T12:24:17.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dbc09e290 con 0x5555e7057580 2023-10-18T12:24:17.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2dadffb700 1 -- 172.21.15.70:0/3588025608 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db801e5a0 con 0x5555e7057580 2023-10-18T12:24:17.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2de07b9b80 10 client.14823 fetch_fsmap learned FSMap version 17 2023-10-18T12:24:17.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5555e6dbdc10 con 0x5555e7057580 2023-10-18T12:24:17.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.688+0000 7f2dadffb700 1 -- 172.21.15.70:0/3588025608 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dbc09e290 con 0x5555e7057580 2023-10-18T12:24:17.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2dadffb700 1 -- 172.21.15.70:0/3588025608 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2db8030060 con 0x5555e7057580 2023-10-18T12:24:17.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2de07b9b80 10 client.14823 fetch_fsmap finished waiting for FSMap version 17 2023-10-18T12:24:17.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2de07b9b80 10 client.14823 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T12:24:17.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2de07b9b80 20 client.14823 populate_metadata read hostname 'smithi070' 2023-10-18T12:24:17.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e70ca0c0 0x5555e70c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2de07b9b80 4 client.14823 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:24:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5555e6fa2e00 con 0x5555e70ca0c0 2023-10-18T12:24:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.689+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e70ca0c0 0x5555e70c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2dc5d9b700 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e70ca0c0 0x5555e70c6510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23de6a49aeaefdd6 server_cookie=472f37d9bd3e0fe in_seq=0 out_seq=0 2023-10-18T12:24:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2dadffb700 10 client.14823 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:24:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2dadffb700 1 -- 172.21.15.70:0/3588025608 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5555e6fa2e00 con 0x5555e70ca0c0 2023-10-18T12:24:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2dadffb700 10 client.14823 handle_command_reply: tid=0 2023-10-18T12:24:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 client.14823 shutdown 2023-10-18T12:24:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 msgr2=0x5555e705fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e705fcd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2db801ec60 tx=0x7f2db80046a0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 shutdown_connections 2023-10-18T12:24:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555e70ca0c0 0x5555e70c6510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555e705d4b0 0x5555e70c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5555e7057580 0x5555e705fcd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2dadffb700 0 client.14823 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:24:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 --2- 172.21.15.70:0/3588025608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555e705cb90 0x5555e7060210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 >> 172.21.15.70:0/3588025608 conn(0x5555e6fcb6d0 msgr2=0x5555e704b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 shutdown_connections 2023-10-18T12:24:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 1 -- 172.21.15.70:0/3588025608 wait complete. 2023-10-18T12:24:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.690+0000 7f2de07b9b80 20 client.14823 trim_cache size 0 max 16384 2023-10-18T12:24:17.708 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:24:17.708 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "f94fde2a-6daf-11ee-8db6-212e2dc638e7", 2023-10-18T12:24:17.708 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-18T12:24:17.709 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 24743, 2023-10-18T12:24:17.709 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-18T12:24:17.709 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-18T12:24:17.709 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-18T12:24:17.709 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 7.5713136509999996, 2023-10-18T12:24:17.709 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 17, 2023-10-18T12:24:17.710 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 89, 2023-10-18T12:24:17.710 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 89, 2023-10-18T12:24:17.710 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 14.547617467 2023-10-18T12:24:17.710 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:24:17.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2db00a1550 msgr2=0x7f2db00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2db00a1550 0x7f2db00a3a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b780 tx=0x7f2dc0007d90 comp rx=0 tx=0).stop 2023-10-18T12:24:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 msgr2=0x7f2dc8199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 0x7f2dc8199400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00e9a0 tx=0x7f2dbc00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 shutdown_connections 2023-10-18T12:24:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dc810cee0 0x7f2dc819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2db00a1550 0x7f2db00a3a10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dc8072c20 0x7f2dc8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 --2- 172.21.15.70:0/852067294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dc8073540 0x7f2dc8199400 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 >> 172.21.15.70:0/852067294 conn(0x7f2dc80fc170 msgr2=0x7f2dc80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 shutdown_connections 2023-10-18T12:24:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:17.691+0000 7f2dccdff700 1 -- 172.21.15.70:0/852067294 wait complete. 2023-10-18T12:24:17.714 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T12:24:17.715 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:17.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-18T12:24:18.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:18 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3495722855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:18.445 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:18.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:18 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3495722855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:18.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:18 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3495722855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:19.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:19 smithi070 ceph-mon[98639]: pgmap v259: 129 pgs: 7 creating+peering, 116 active+clean, 6 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2023-10-18T12:24:19.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:19 smithi152 ceph-mon[134767]: pgmap v259: 129 pgs: 7 creating+peering, 116 active+clean, 6 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2023-10-18T12:24:19.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:19 smithi186 ceph-mon[135253]: pgmap v259: 129 pgs: 7 creating+peering, 116 active+clean, 6 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2023-10-18T12:24:19.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 -- 172.21.15.70:0/4249764357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb581057c0 msgr2=0x7feb58105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:19.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 --2- 172.21.15.70:0/4249764357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb581057c0 0x7feb58105ba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feb48009990 tx=0x7feb48009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 -- 172.21.15.70:0/4249764357 shutdown_connections 2023-10-18T12:24:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 --2- 172.21.15.70:0/4249764357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb58101260 0x7feb5810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 --2- 172.21.15.70:0/4249764357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb581057c0 0x7feb58105ba0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 --2- 172.21.15.70:0/4249764357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 0x7feb58100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 -- 172.21.15.70:0/4249764357 >> 172.21.15.70:0/4249764357 conn(0x7feb580779e0 msgr2=0x7feb58077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 -- 172.21.15.70:0/4249764357 shutdown_connections 2023-10-18T12:24:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.914+0000 7feb5d65e700 1 -- 172.21.15.70:0/4249764357 wait complete. 2023-10-18T12:24:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.915+0000 7feb5d65e700 1 Processor -- start 2023-10-18T12:24:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.915+0000 7feb5d65e700 1 -- start start 2023-10-18T12:24:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb5d65e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 0x7feb58198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb5d65e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 0x7feb581993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb5d65e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb581057c0 0x7feb5819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb5d65e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5819dda0 con 0x7feb58101260 2023-10-18T12:24:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb5d65e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5819df10 con 0x7feb581008a0 2023-10-18T12:24:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb5d65e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5819e0d0 con 0x7feb581057c0 2023-10-18T12:24:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 0x7feb581993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 0x7feb581993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48276/0 (socket says 172.21.15.70:48276) 2023-10-18T12:24:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 -- 172.21.15.70:0/288338457 learned_addr learned my addr 172.21.15.70:0/288338457 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb56ffd700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 0x7feb58198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb577fe700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb581057c0 0x7feb5819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 -- 172.21.15.70:0/288338457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb581057c0 msgr2=0x7feb5819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb581057c0 0x7feb5819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 -- 172.21.15.70:0/288338457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 msgr2=0x7feb58198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.916+0000 7feb567fc700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 0x7feb58198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb567fc700 1 -- 172.21.15.70:0/288338457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480095f0 con 0x7feb58101260 2023-10-18T12:24:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb56ffd700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 0x7feb58198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb567fc700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 0x7feb581993e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00d950 tx=0x7feb4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00f890 con 0x7feb58101260 2023-10-18T12:24:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5819e3b0 con 0x7feb58101260 2023-10-18T12:24:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4c010460 con 0x7feb58101260 2023-10-18T12:24:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c01b650 con 0x7feb58101260 2023-10-18T12:24:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.917+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb581a5c10 con 0x7feb58101260 2023-10-18T12:24:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.918+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb58101f30 con 0x7feb58101260 2023-10-18T12:24:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.920+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7feb4c0108d0 con 0x7feb58101260 2023-10-18T12:24:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.921+0000 7feb3ffff700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feb400a14d0 0x7feb400a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.922+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7feb4c0cd140 con 0x7feb58101260 2023-10-18T12:24:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.922+0000 7feb56ffd700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feb400a14d0 0x7feb400a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.922+0000 7feb56ffd700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feb400a14d0 0x7feb400a3990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b5c0 tx=0x7feb4801a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:19.926+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feb4c0964d0 con 0x7feb58101260 2023-10-18T12:24:20.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.165+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb5804e4c0 con 0x7feb58101260 2023-10-18T12:24:20.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.166+0000 7feb3ffff700 1 -- 172.21.15.70:0/288338457 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7feb4c099af0 con 0x7feb58101260 2023-10-18T12:24:20.169 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T12:24:20.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feb400a14d0 msgr2=0x7feb400a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:20.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feb400a14d0 0x7feb400a3990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b5c0 tx=0x7feb4801a040 comp rx=0 tx=0).stop 2023-10-18T12:24:20.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 msgr2=0x7feb581993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:20.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 0x7feb581993e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00d950 tx=0x7feb4c00dc60 comp rx=0 tx=0).stop 2023-10-18T12:24:20.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 shutdown_connections 2023-10-18T12:24:20.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb581057c0 0x7feb5819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:20.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feb400a14d0 0x7feb400a3990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:20.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb581008a0 0x7feb58198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:20.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 --2- 172.21.15.70:0/288338457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb58101260 0x7feb581993e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:20.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.168+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 >> 172.21.15.70:0/288338457 conn(0x7feb580779e0 msgr2=0x7feb5810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:20.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.169+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 shutdown_connections 2023-10-18T12:24:20.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:20.169+0000 7feb5d65e700 1 -- 172.21.15.70:0/288338457 wait complete. 2023-10-18T12:24:20.630 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T12:24:20.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T12:24:20.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-18T12:24:20.631 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-18T12:24:20.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:20.632 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T12:24:20.632 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T12:24:20.633 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T12:24:20.633 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T12:24:20.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:20.634 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-18T12:24:20.679 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T12:24:20.679 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T12:24:20.679 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T12:24:20.680 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:24:20.680 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-18T12:24:20.680 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:24:20.681 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:24:20.681 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:24:20.681 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-18T12:24:20.681 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:24:20.681 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:24:20.682 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-18T12:24:20.682 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:24:20.682 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:24:20.683 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T12:24:20.683 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:24:20.683 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T12:24:20.683 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41676sec preferred_lft 41676sec 2023-10-18T12:24:20.683 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::6f76:96b5:cb68:c2e6/64 scope link noprefixroute 2023-10-18T12:24:20.684 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:24:20.685 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T12:24:20.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:20.686 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:24:20.686 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:24:20.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2023-10-18T12:24:20.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2023-10-18T12:24:20.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2023-10-18T12:24:20.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T12:24:20.688 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:24:21.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:21.268 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T12:24:21.341 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-18T12:24:21.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:21.345 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-18T12:24:21.401 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-18T12:24:21.401 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T12:24:21.401 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T12:24:21.401 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T12:24:21.402 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T12:24:21.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:21.403 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:24:21.404 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:24:21.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T12:24:21.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T12:24:21.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T12:24:21.405 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:24:21.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:21 smithi152 ceph-mon[134767]: pgmap v260: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-10-18T12:24:21.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:21 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/288338457' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T12:24:21.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:21 smithi186 ceph-mon[135253]: pgmap v260: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-10-18T12:24:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:21 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/288338457' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T12:24:21.506 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:21 smithi070 ceph-mon[98639]: pgmap v260: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2023-10-18T12:24:21.507 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:21 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/288338457' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T12:24:22.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:22.139 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-18T12:24:22.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:22.187 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-10-18T12:24:22.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:22.272 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:24:22.272 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:24:22.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:22.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-18T12:24:22.273 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:24:22.457 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-18T12:24:22.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:22.458 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:24:22.459 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:24:22.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-18T12:24:22.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T12:24:22.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-18T12:24:22.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-18T12:24:22.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-18T12:24:22.461 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:24:22.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:22.733 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:24:22.733 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:24:22.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-18T12:24:22.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T12:24:22.734 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:24:22.907 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-18T12:24:22.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:24:22.908 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:22.952 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T12:24:22.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:24:22.953 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:23.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-10-18T12:24:23.149 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:24:23.195 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2023-10-18T12:24:23.195 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2023-10-18T12:24:23.196 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2023-10-18T12:24:23.196 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2023-10-18T12:24:23.196 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2023-10-18T12:24:23.196 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2023-10-18T12:24:23.197 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2023-10-18T12:24:23.197 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2023-10-18T12:24:23.197 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/systemd 2023-10-18T12:24:23.197 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2023-10-18T12:24:23.197 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-18T12:24:23.198 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/freezer 2023-10-18T12:24:23.198 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/devices 2023-10-18T12:24:23.198 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/cpuset 2023-10-18T12:24:23.198 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/memory 2023-10-18T12:24:23.198 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-18T12:24:23.199 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/blkio 2023-10-18T12:24:23.199 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/hugetlb 2023-10-18T12:24:23.199 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/pids 2023-10-18T12:24:23.199 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/perf_event 2023-10-18T12:24:23.199 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2023-10-18T12:24:23.200 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2023-10-18T12:24:23.200 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2023-10-18T12:24:23.200 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2023-10-18T12:24:23.200 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2023-10-18T12:24:23.200 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2023-10-18T12:24:23.200 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2023-10-18T12:24:23.201 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:24:23.201 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2023-10-18T12:24:23.201 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:24:23.201 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2023-10-18T12:24:23.201 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2023-10-18T12:24:23.202 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2023-10-18T12:24:23.202 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/f22f449cc4b92fc71bff98c56cad318cff0cf7afa376dc14c64cfd8cef38f7e7/merged 2023-10-18T12:24:23.202 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/a83e24bf087a159585f5d320d41dc7bc7ef64335104ed874428c0ed69c785efe/merged 2023-10-18T12:24:23.202 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2023-10-18T12:24:23.202 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:24:23.203 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/75ea71b1d2fe11f3b0c3d4e2ddafda1218b1478a2c48edf406668ca70d1d2acd/merged 2023-10-18T12:24:23.203 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ae83f4effaae895e288fb7889f72df0baca61141c11cc7295b51936bc8ed5122/merged 2023-10-18T12:24:23.203 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/df2e843e934ef225eae95d50b61b08a7bca25f0bc59325d20028bdaafaa910ed/merged 2023-10-18T12:24:23.203 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/fb922db7e043614e88315037e41ebe854b407f7f4287716170e358faf386d6a0/merged 2023-10-18T12:24:23.203 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/5a98d7bb0d0799b1ed18811d4e8f64f5fc42bf4ade8a2ac831ddcacae4e9b1a2/merged 2023-10-18T12:24:23.204 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/f615ab8840aa227ae80c86ad020399d087606f4e475b825eb7c7fcd3cc7f33cc/merged 2023-10-18T12:24:23.204 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ea288eef17fe76e44c20c8f45dafd3ec9eb885261a6cef5730262f33911332e1/merged 2023-10-18T12:24:23.204 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/e702a614de1c9d5d549f1ba35fe387b5f6503fd22c1041436f8d198b68542241/merged 2023-10-18T12:24:23.204 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2023-10-18T12:24:23.204 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:23.205 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:23.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:23.206 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-10-18T12:24:23.289 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T12:24:23.290 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-18T12:24:23.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-10-18T12:24:23.392 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:23 smithi070 ceph-mon[98639]: pgmap v261: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-10-18T12:24:23.404 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:24:23.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:23 smithi152 ceph-mon[134767]: pgmap v261: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-10-18T12:24:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2023-10-18T12:24:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2023-10-18T12:24:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2023-10-18T12:24:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2023-10-18T12:24:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2023-10-18T12:24:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2023-10-18T12:24:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2023-10-18T12:24:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2023-10-18T12:24:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/systemd 2023-10-18T12:24:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2023-10-18T12:24:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-18T12:24:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/freezer 2023-10-18T12:24:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/devices 2023-10-18T12:24:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/cpuset 2023-10-18T12:24:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/memory 2023-10-18T12:24:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-18T12:24:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/blkio 2023-10-18T12:24:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/hugetlb 2023-10-18T12:24:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/pids 2023-10-18T12:24:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/perf_event 2023-10-18T12:24:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2023-10-18T12:24:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2023-10-18T12:24:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2023-10-18T12:24:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2023-10-18T12:24:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2023-10-18T12:24:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2023-10-18T12:24:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2023-10-18T12:24:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:24:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2023-10-18T12:24:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:24:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2023-10-18T12:24:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2023-10-18T12:24:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2023-10-18T12:24:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/f22f449cc4b92fc71bff98c56cad318cff0cf7afa376dc14c64cfd8cef38f7e7/merged 2023-10-18T12:24:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/a83e24bf087a159585f5d320d41dc7bc7ef64335104ed874428c0ed69c785efe/merged 2023-10-18T12:24:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2023-10-18T12:24:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:24:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/75ea71b1d2fe11f3b0c3d4e2ddafda1218b1478a2c48edf406668ca70d1d2acd/merged 2023-10-18T12:24:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ae83f4effaae895e288fb7889f72df0baca61141c11cc7295b51936bc8ed5122/merged 2023-10-18T12:24:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/df2e843e934ef225eae95d50b61b08a7bca25f0bc59325d20028bdaafaa910ed/merged 2023-10-18T12:24:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/fb922db7e043614e88315037e41ebe854b407f7f4287716170e358faf386d6a0/merged 2023-10-18T12:24:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/5a98d7bb0d0799b1ed18811d4e8f64f5fc42bf4ade8a2ac831ddcacae4e9b1a2/merged 2023-10-18T12:24:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/f615ab8840aa227ae80c86ad020399d087606f4e475b825eb7c7fcd3cc7f33cc/merged 2023-10-18T12:24:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ea288eef17fe76e44c20c8f45dafd3ec9eb885261a6cef5730262f33911332e1/merged 2023-10-18T12:24:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/e702a614de1c9d5d549f1ba35fe387b5f6503fd22c1041436f8d198b68542241/merged 2023-10-18T12:24:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2023-10-18T12:24:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:23.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:23.460 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-10-18T12:24:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:23 smithi186 ceph-mon[135253]: pgmap v261: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-10-18T12:24:23.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:2023-10-18T12:24:23.558+0000 7f459d56b5c0 -1 init, newargv = 0x5652885b9d10 newargc=15 2023-10-18T12:24:23.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:ceph-fuse[160301]: starting ceph client 2023-10-18T12:24:23.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:ceph-fuse[160301]: starting fuse 2023-10-18T12:24:24.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-10-18T12:24:24.722 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:24:24.769 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2023-10-18T12:24:24.769 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2023-10-18T12:24:24.770 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2023-10-18T12:24:24.770 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2023-10-18T12:24:24.770 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2023-10-18T12:24:24.770 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2023-10-18T12:24:24.770 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2023-10-18T12:24:24.771 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2023-10-18T12:24:24.771 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/systemd 2023-10-18T12:24:24.771 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2023-10-18T12:24:24.771 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-18T12:24:24.772 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/freezer 2023-10-18T12:24:24.772 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/devices 2023-10-18T12:24:24.772 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/cpuset 2023-10-18T12:24:24.772 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/memory 2023-10-18T12:24:24.772 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-18T12:24:24.772 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/blkio 2023-10-18T12:24:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/hugetlb 2023-10-18T12:24:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/pids 2023-10-18T12:24:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup/perf_event 2023-10-18T12:24:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2023-10-18T12:24:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2023-10-18T12:24:24.774 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2023-10-18T12:24:24.774 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2023-10-18T12:24:24.774 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2023-10-18T12:24:24.774 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2023-10-18T12:24:24.775 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2023-10-18T12:24:24.775 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:24:24.775 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2023-10-18T12:24:24.775 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:24:24.775 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2023-10-18T12:24:24.776 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2023-10-18T12:24:24.776 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2023-10-18T12:24:24.776 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/f22f449cc4b92fc71bff98c56cad318cff0cf7afa376dc14c64cfd8cef38f7e7/merged 2023-10-18T12:24:24.776 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/a83e24bf087a159585f5d320d41dc7bc7ef64335104ed874428c0ed69c785efe/merged 2023-10-18T12:24:24.776 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2023-10-18T12:24:24.777 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:24:24.777 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/75ea71b1d2fe11f3b0c3d4e2ddafda1218b1478a2c48edf406668ca70d1d2acd/merged 2023-10-18T12:24:24.777 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ae83f4effaae895e288fb7889f72df0baca61141c11cc7295b51936bc8ed5122/merged 2023-10-18T12:24:24.777 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/df2e843e934ef225eae95d50b61b08a7bca25f0bc59325d20028bdaafaa910ed/merged 2023-10-18T12:24:24.777 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/fb922db7e043614e88315037e41ebe854b407f7f4287716170e358faf386d6a0/merged 2023-10-18T12:24:24.778 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/5a98d7bb0d0799b1ed18811d4e8f64f5fc42bf4ade8a2ac831ddcacae4e9b1a2/merged 2023-10-18T12:24:24.778 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/f615ab8840aa227ae80c86ad020399d087606f4e475b825eb7c7fcd3cc7f33cc/merged 2023-10-18T12:24:24.778 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ea288eef17fe76e44c20c8f45dafd3ec9eb885261a6cef5730262f33911332e1/merged 2023-10-18T12:24:24.778 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/e702a614de1c9d5d549f1ba35fe387b5f6503fd22c1041436f8d198b68542241/merged 2023-10-18T12:24:24.778 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2023-10-18T12:24:24.779 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:24.779 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:24:24.779 INFO:teuthology.orchestra.run.smithi070.stdout:/home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:24.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:24.781 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-10-18T12:24:24.865 INFO:teuthology.orchestra.run.smithi070.stdout:121 2023-10-18T12:24:24.865 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-10-18T12:24:24.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-10-18T12:24:24.866 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-10-18T12:24:24.866 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-10-18T12:24:24.867 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-10-18T12:24:24.867 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-10-18T12:24:24.867 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-18T12:24:24.868 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-10-18T12:24:24.868 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-18T12:24:24.868 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-10-18T12:24:24.869 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-18T12:24:24.869 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-18T12:24:24.870 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-10-18T12:24:24.870 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-10-18T12:24:24.870 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-10-18T12:24:24.871 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-10-18T12:24:24.871 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-18T12:24:24.871 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-10-18T12:24:24.872 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T12:24:24.872 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T12:24:24.873 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T12:24:24.873 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-10-18T12:24:24.873 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-10-18T12:24:24.874 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-10-18T12:24:24.874 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T12:24:24.874 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-18T12:24:24.875 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.admin")) 2023-10-18T12:24:24.875 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-10-18T12:24:25.048 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.admin.160301.asok 2023-10-18T12:24:25.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T12:24:25.062 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.160301.asok 2023-10-18T12:24:25.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:25.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.160301.asok status 2023-10-18T12:24:25.289 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:24:25.289 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-10-18T12:24:25.289 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T12:24:25.290 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:24:25.290 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "admin", 2023-10-18T12:24:25.290 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-10-18T12:24:25.290 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-18T12:24:25.290 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "160301", 2023-10-18T12:24:25.291 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-10-18T12:24:25.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:24:25.291 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-10-18T12:24:25.291 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-10-18T12:24:25.291 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 14835, 2023-10-18T12:24:25.292 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-10-18T12:24:25.292 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-10-18T12:24:25.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-10-18T12:24:25.292 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 14835 2023-10-18T12:24:25.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:24:25.293 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-10-18T12:24:25.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-10-18T12:24:25.293 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:24:25.293 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1001498069 2023-10-18T12:24:25.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:24:25.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:24:25.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-10-18T12:24:25.294 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-10-18T12:24:25.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:24:25.295 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1001498069 2023-10-18T12:24:25.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:24:25.295 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.14835 192.168.0.1:0/1001498069", 2023-10-18T12:24:25.295 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/1001498069", 2023-10-18T12:24:25.295 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-10-18T12:24:25.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 17, 2023-10-18T12:24:25.296 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 89, 2023-10-18T12:24:25.296 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-10-18T12:24:25.296 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-10-18T12:24:25.296 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-10-18T12:24:25.297 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:24:25.301 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:25 smithi070 ceph-mon[98639]: pgmap v262: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 108 B/s wr, 2 op/s 2023-10-18T12:24:25.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:25.304 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-10-18T12:24:25.304 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:25.348 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-10-18T12:24:25.349 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:25.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:25.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:25.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:25 smithi152 ceph-mon[134767]: pgmap v262: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 108 B/s wr, 2 op/s 2023-10-18T12:24:25.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:25.460 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-18T12:24:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:25 smithi186 ceph-mon[135253]: pgmap v262: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 108 B/s wr, 2 op/s 2023-10-18T12:24:25.588 INFO:teuthology.orchestra.run.smithi070.stdout:# file: . 2023-10-18T12:24:25.589 INFO:teuthology.orchestra.run.smithi070.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-18T12:24:25.589 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:25.590 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:24:25.591 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:24:25.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:25.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 -- 172.21.15.70:0/579514733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc641006a0 msgr2=0x7efc64100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:25.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 --2- 172.21.15.70:0/579514733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc641006a0 0x7efc64100b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009990 tx=0x7efc4c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 -- 172.21.15.70:0/579514733 shutdown_connections 2023-10-18T12:24:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 --2- 172.21.15.70:0/579514733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc64101060 0x7efc6410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 --2- 172.21.15.70:0/579514733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641066b0 0x7efc64106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 --2- 172.21.15.70:0/579514733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc641006a0 0x7efc64100b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 -- 172.21.15.70:0/579514733 >> 172.21.15.70:0/579514733 conn(0x7efc640fc170 msgr2=0x7efc640fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 -- 172.21.15.70:0/579514733 shutdown_connections 2023-10-18T12:24:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.871+0000 7efc689ff700 1 -- 172.21.15.70:0/579514733 wait complete. 2023-10-18T12:24:25.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.872+0000 7efc689ff700 1 Processor -- start 2023-10-18T12:24:25.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.872+0000 7efc689ff700 1 -- start start 2023-10-18T12:24:25.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.872+0000 7efc689ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641006a0 0x7efc64198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:25.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.872+0000 7efc689ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 0x7efc64199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:25.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.872+0000 7efc689ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc641066b0 0x7efc6419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:25.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc689ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419cb50 con 0x7efc64101060 2023-10-18T12:24:25.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc689ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419ccc0 con 0x7efc641006a0 2023-10-18T12:24:25.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc689ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419ce80 con 0x7efc641066b0 2023-10-18T12:24:25.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 0x7efc64199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:25.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 0x7efc64199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48292/0 (socket says 172.21.15.70:48292) 2023-10-18T12:24:25.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 -- 172.21.15.70:0/4271732462 learned_addr learned my addr 172.21.15.70:0/4271732462 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:25.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc6259c700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641006a0 0x7efc64198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:25.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc62d9d700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc641066b0 0x7efc6419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:25.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 -- 172.21.15.70:0/4271732462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc641066b0 msgr2=0x7efc6419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:25.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc641066b0 0x7efc6419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:25.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 -- 172.21.15.70:0/4271732462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641006a0 msgr2=0x7efc64198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:25.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641006a0 0x7efc64198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:25.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 -- 172.21.15.70:0/4271732462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c0095f0 con 0x7efc64101060 2023-10-18T12:24:25.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc62d9d700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc641066b0 0x7efc6419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:25.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.873+0000 7efc5bfff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 0x7efc64199260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc4c000c00 tx=0x7efc4c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:25.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.874+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c01d070 con 0x7efc64101060 2023-10-18T12:24:25.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.874+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4c022470 con 0x7efc64101060 2023-10-18T12:24:25.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.874+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6419d100 con 0x7efc64101060 2023-10-18T12:24:25.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.874+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c017650 con 0x7efc64101060 2023-10-18T12:24:25.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.874+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc641a5ac0 con 0x7efc64101060 2023-10-18T12:24:25.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.875+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc64101de0 con 0x7efc64101060 2023-10-18T12:24:25.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.877+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7efc4c0225e0 con 0x7efc64101060 2023-10-18T12:24:25.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.878+0000 7efc5b7fe700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc500a1550 0x7efc500a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:25.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.878+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7efc4c0ce330 con 0x7efc64101060 2023-10-18T12:24:25.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.878+0000 7efc6259c700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc500a1550 0x7efc500a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:25.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.879+0000 7efc6259c700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc500a1550 0x7efc500a3a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc54009be0 tx=0x7efc54009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:25.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:25.882+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efc4c0976f0 con 0x7efc64101060 2023-10-18T12:24:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.092+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc64067530 con 0x7efc64101060 2023-10-18T12:24:26.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.094+0000 7efc5b7fe700 1 -- 172.21.15.70:0/4271732462 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7efc4c09ad10 con 0x7efc64101060 2023-10-18T12:24:26.097 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:26.097 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:26.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc500a1550 msgr2=0x7efc500a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc500a1550 0x7efc500a3a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc54009be0 tx=0x7efc54009260 comp rx=0 tx=0).stop 2023-10-18T12:24:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 msgr2=0x7efc64199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 0x7efc64199260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc4c000c00 tx=0x7efc4c005740 comp rx=0 tx=0).stop 2023-10-18T12:24:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 shutdown_connections 2023-10-18T12:24:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc641066b0 0x7efc6419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc500a1550 0x7efc500a3a10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc641006a0 0x7efc64198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 --2- 172.21.15.70:0/4271732462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc64101060 0x7efc64199260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.096+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 >> 172.21.15.70:0/4271732462 conn(0x7efc640fc170 msgr2=0x7efc640fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.097+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 shutdown_connections 2023-10-18T12:24:26.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.097+0000 7efc689ff700 1 -- 172.21.15.70:0/4271732462 wait complete. 2023-10-18T12:24:26.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:24:26.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:24:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 -- 172.21.15.70:0/690309522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c100720 msgr2=0x7f280c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 --2- 172.21.15.70:0/690309522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c100720 0x7f280c100ba0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2800009990 tx=0x7f2800009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 -- 172.21.15.70:0/690309522 shutdown_connections 2023-10-18T12:24:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 --2- 172.21.15.70:0/690309522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f280c1010e0 0x7f280c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 --2- 172.21.15.70:0/690309522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c106730 0x7f280c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 --2- 172.21.15.70:0/690309522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c100720 0x7f280c100ba0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.317+0000 7f2811bff700 1 -- 172.21.15.70:0/690309522 >> 172.21.15.70:0/690309522 conn(0x7f280c0fc1b0 msgr2=0x7f280c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:26.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.318+0000 7f2811bff700 1 -- 172.21.15.70:0/690309522 shutdown_connections 2023-10-18T12:24:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.318+0000 7f2811bff700 1 -- 172.21.15.70:0/690309522 wait complete. 2023-10-18T12:24:26.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 Processor -- start 2023-10-18T12:24:26.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 -- start start 2023-10-18T12:24:26.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f280c100720 0x7f280c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 0x7f280c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c106730 0x7f280c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c19cb60 con 0x7f280c1010e0 2023-10-18T12:24:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c19ccd0 con 0x7f280c100720 2023-10-18T12:24:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f2811bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c19ce90 con 0x7f280c106730 2023-10-18T12:24:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f280affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 0x7f280c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f280affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 0x7f280c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48312/0 (socket says 172.21.15.70:48312) 2023-10-18T12:24:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f280affd700 1 -- 172.21.15.70:0/1064877277 learned_addr learned my addr 172.21.15.70:0/1064877277 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.319+0000 7f280bfff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c106730 0x7f280c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280b7fe700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f280c100720 0x7f280c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280affd700 1 -- 172.21.15.70:0/1064877277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c106730 msgr2=0x7f280c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280affd700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c106730 0x7f280c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280affd700 1 -- 172.21.15.70:0/1064877277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f280c100720 msgr2=0x7f280c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280affd700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f280c100720 0x7f280c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280affd700 1 -- 172.21.15.70:0/1064877277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000095f0 con 0x7f280c1010e0 2023-10-18T12:24:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f280affd700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 0x7f280c199270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f2800004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280001d070 con 0x7f280c1010e0 2023-10-18T12:24:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2800022470 con 0x7f280c1010e0 2023-10-18T12:24:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800017650 con 0x7f280c1010e0 2023-10-18T12:24:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.320+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c19d110 con 0x7f280c1010e0 2023-10-18T12:24:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.321+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c19d3c0 con 0x7f280c1010e0 2023-10-18T12:24:26.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.321+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c101e60 con 0x7f280c1010e0 2023-10-18T12:24:26.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.323+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f28000225e0 con 0x7f280c1010e0 2023-10-18T12:24:26.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.325+0000 7f2808ff9700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f27f40a1550 0x7f27f40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.325+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f28000ce6c0 con 0x7f280c1010e0 2023-10-18T12:24:26.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.325+0000 7f280b7fe700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f27f40a1550 0x7f27f40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.325+0000 7f280b7fe700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f27f40a1550 0x7f27f40a3a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009be0 tx=0x7f27fc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.327+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2800097b00 con 0x7f280c1010e0 2023-10-18T12:24:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:26 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4271732462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:26.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:26 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4271732462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:26.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.527+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f280c110650 con 0x7f280c1010e0 2023-10-18T12:24:26.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.527+0000 7f2808ff9700 1 -- 172.21.15.70:0/1064877277 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28000270a0 con 0x7f280c1010e0 2023-10-18T12:24:26.531 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:24:26.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f27f40a1550 msgr2=0x7f27f40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f27f40a1550 0x7f27f40a3a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009be0 tx=0x7f27fc009260 comp rx=0 tx=0).stop 2023-10-18T12:24:26.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 msgr2=0x7f280c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 0x7f280c199270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f2800004730 comp rx=0 tx=0).stop 2023-10-18T12:24:26.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 shutdown_connections 2023-10-18T12:24:26.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280c106730 0x7f280c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f27f40a1550 0x7f27f40a3a10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f280c100720 0x7f280c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 --2- 172.21.15.70:0/1064877277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f280c1010e0 0x7f280c199270 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 >> 172.21.15.70:0/1064877277 conn(0x7f280c0fc1b0 msgr2=0x7f280c10bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:26.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 shutdown_connections 2023-10-18T12:24:26.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.530+0000 7f2811bff700 1 -- 172.21.15.70:0/1064877277 wait complete. 2023-10-18T12:24:26.548 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:26 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4271732462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:26.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:24:26.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.750+0000 7fe40b7ff700 1 -- 172.21.15.70:0/3059591234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe404100720 msgr2=0x7fe404100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.750+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/3059591234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe404100720 0x7fe404100ba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009990 tx=0x7fe3f4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 -- 172.21.15.70:0/3059591234 shutdown_connections 2023-10-18T12:24:26.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/3059591234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4041010e0 0x7fe40410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/3059591234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 0x7fe404106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/3059591234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe404100720 0x7fe404100ba0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 -- 172.21.15.70:0/3059591234 >> 172.21.15.70:0/3059591234 conn(0x7fe4040fc1b0 msgr2=0x7fe4040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:26.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 -- 172.21.15.70:0/3059591234 shutdown_connections 2023-10-18T12:24:26.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.751+0000 7fe40b7ff700 1 -- 172.21.15.70:0/3059591234 wait complete. 2023-10-18T12:24:26.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 Processor -- start 2023-10-18T12:24:26.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 -- start start 2023-10-18T12:24:26.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 0x7fe404198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4041010e0 0x7fe404199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 0x7fe40419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419cb30 con 0x7fe404100720 2023-10-18T12:24:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419cca0 con 0x7fe4041010e0 2023-10-18T12:24:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419ce60 con 0x7fe404106730 2023-10-18T12:24:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40959b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 0x7fe404198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40959b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 0x7fe404198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48334/0 (socket says 172.21.15.70:48334) 2023-10-18T12:24:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe40959b700 1 -- 172.21.15.70:0/1900208066 learned_addr learned my addr 172.21.15.70:0/1900208066 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.752+0000 7fe409d9c700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 0x7fe40419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe408d9a700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4041010e0 0x7fe404199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40959b700 1 -- 172.21.15.70:0/1900208066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 msgr2=0x7fe40419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40959b700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 0x7fe40419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40959b700 1 -- 172.21.15.70:0/1900208066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4041010e0 msgr2=0x7fe404199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40959b700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4041010e0 0x7fe404199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40959b700 1 -- 172.21.15.70:0/1900208066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400009520 con 0x7fe404100720 2023-10-18T12:24:26.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe409d9c700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 0x7fe40419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:26.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40959b700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 0x7fe404198d00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe40000eb30 tx=0x7fe40000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:26.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400009880 con 0x7fe404100720 2023-10-18T12:24:26.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f40095f0 con 0x7fe404100720 2023-10-18T12:24:26.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.753+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4000044e0 con 0x7fe404100720 2023-10-18T12:24:26.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.754+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400003d80 con 0x7fe404100720 2023-10-18T12:24:26.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.754+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4041a58c0 con 0x7fe404100720 2023-10-18T12:24:26.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.755+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe404101e60 con 0x7fe404100720 2023-10-18T12:24:26.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.756+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe400003f20 con 0x7fe404100720 2023-10-18T12:24:26.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.758+0000 7fe3fa7fc700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3f00a14d0 0x7fe3f00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:26.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.758+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fe400014070 con 0x7fe404100720 2023-10-18T12:24:26.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.758+0000 7fe408d9a700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3f00a14d0 0x7fe3f00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:26.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.758+0000 7fe408d9a700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3f00a14d0 0x7fe3f00a3990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40098e0 tx=0x7fe3f4005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:26.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.761+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe400096880 con 0x7fe404100720 2023-10-18T12:24:26.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.963+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe40419a060 con 0x7fe404100720 2023-10-18T12:24:26.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.963+0000 7fe3fa7fc700 1 -- 172.21.15.70:0/1900208066 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe400099ea0 con 0x7fe404100720 2023-10-18T12:24:26.967 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:24:26.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3f00a14d0 msgr2=0x7fe3f00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3f00a14d0 0x7fe3f00a3990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40098e0 tx=0x7fe3f4005960 comp rx=0 tx=0).stop 2023-10-18T12:24:26.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 msgr2=0x7fe404198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:26.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 0x7fe404198d00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe40000eb30 tx=0x7fe40000ee40 comp rx=0 tx=0).stop 2023-10-18T12:24:26.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 shutdown_connections 2023-10-18T12:24:26.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404106730 0x7fe40419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe404100720 0x7fe404198d00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe3f00a14d0 0x7fe3f00a3990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 --2- 172.21.15.70:0/1900208066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4041010e0 0x7fe404199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:26.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 >> 172.21.15.70:0/1900208066 conn(0x7fe4040fc1b0 msgr2=0x7fe40410bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:26.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 shutdown_connections 2023-10-18T12:24:26.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:26.966+0000 7fe40b7ff700 1 -- 172.21.15.70:0/1900208066 wait complete. 2023-10-18T12:24:26.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:24:27.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 -- 172.21.15.70:0/1082746889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234100710 msgr2=0x7f0234100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 --2- 172.21.15.70:0/1082746889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234100710 0x7f0234100b90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0220009990 tx=0x7f0220009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 -- 172.21.15.70:0/1082746889 shutdown_connections 2023-10-18T12:24:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 --2- 172.21.15.70:0/1082746889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02341010d0 0x7f023410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 --2- 172.21.15.70:0/1082746889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0234106720 0x7f0234106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 --2- 172.21.15.70:0/1082746889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234100710 0x7f0234100b90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 -- 172.21.15.70:0/1082746889 >> 172.21.15.70:0/1082746889 conn(0x7f02340fc200 msgr2=0x7f02340fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 -- 172.21.15.70:0/1082746889 shutdown_connections 2023-10-18T12:24:27.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.187+0000 7f023adff700 1 -- 172.21.15.70:0/1082746889 wait complete. 2023-10-18T12:24:27.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.188+0000 7f023adff700 1 Processor -- start 2023-10-18T12:24:27.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 -- start start 2023-10-18T12:24:27.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0234100710 0x7f0234198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 0x7f0234199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234106720 0x7f023419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419dcc0 con 0x7f02341010d0 2023-10-18T12:24:27.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419de30 con 0x7f0234100710 2023-10-18T12:24:27.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023adff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419dff0 con 0x7f0234106720 2023-10-18T12:24:27.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f0233fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 0x7f0234199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f0233fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 0x7f0234199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48354/0 (socket says 172.21.15.70:48354) 2023-10-18T12:24:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f0233fff700 1 -- 172.21.15.70:0/2877714790 learned_addr learned my addr 172.21.15.70:0/2877714790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f023939c700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234106720 0x7f023419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.189+0000 7f0238b9b700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0234100710 0x7f0234198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0233fff700 1 -- 172.21.15.70:0/2877714790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234106720 msgr2=0x7f023419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0233fff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234106720 0x7f023419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0233fff700 1 -- 172.21.15.70:0/2877714790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0234100710 msgr2=0x7f0234198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0233fff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0234100710 0x7f0234198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0233fff700 1 -- 172.21.15.70:0/2877714790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228009520 con 0x7f02341010d0 2023-10-18T12:24:27.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0233fff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 0x7f0234199380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0220005910 tx=0x7f0220004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:27.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022001d070 con 0x7f02341010d0 2023-10-18T12:24:27.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0220022470 con 0x7f02341010d0 2023-10-18T12:24:27.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200095f0 con 0x7f02341010d0 2023-10-18T12:24:27.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220017650 con 0x7f02341010d0 2023-10-18T12:24:27.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.190+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02341a5930 con 0x7f02341010d0 2023-10-18T12:24:27.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.191+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0234101e50 con 0x7f02341010d0 2023-10-18T12:24:27.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.193+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f02200225e0 con 0x7f02341010d0 2023-10-18T12:24:27.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.194+0000 7f0231ffb700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02240a14d0 0x7f02240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.195+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f02200ce2a0 con 0x7f02341010d0 2023-10-18T12:24:27.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.195+0000 7f0238b9b700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02240a14d0 0x7f02240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.195+0000 7f0238b9b700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02240a14d0 0x7f02240a3990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d70 tx=0x7f0228009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:27.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.198+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0220097780 con 0x7f02341010d0 2023-10-18T12:24:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:27 smithi070 ceph-mon[98639]: pgmap v263: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-18T12:24:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:27 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1064877277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:27 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1900208066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:24:27.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.400+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0234110680 con 0x7f02341010d0 2023-10-18T12:24:27.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.401+0000 7f0231ffb700 1 -- 172.21.15.70:0/2877714790 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02200270a0 con 0x7f02341010d0 2023-10-18T12:24:27.405 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:24:27.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02240a14d0 msgr2=0x7f02240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02240a14d0 0x7f02240a3990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d70 tx=0x7f0228009260 comp rx=0 tx=0).stop 2023-10-18T12:24:27.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 msgr2=0x7f0234199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 0x7f0234199380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0220005910 tx=0x7f0220004730 comp rx=0 tx=0).stop 2023-10-18T12:24:27.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 shutdown_connections 2023-10-18T12:24:27.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0234106720 0x7f023419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f02240a14d0 0x7f02240a3990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0234100710 0x7f0234198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 --2- 172.21.15.70:0/2877714790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02341010d0 0x7f0234199380 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 >> 172.21.15.70:0/2877714790 conn(0x7f02340fc200 msgr2=0x7f02340fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:27.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 shutdown_connections 2023-10-18T12:24:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.403+0000 7f023adff700 1 -- 172.21.15.70:0/2877714790 wait complete. 2023-10-18T12:24:27.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:27 smithi152 ceph-mon[134767]: pgmap v263: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-18T12:24:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:27 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1064877277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:27 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1900208066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:24:27.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:24:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:27 smithi186 ceph-mon[135253]: pgmap v263: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-18T12:24:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:27 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1064877277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:27 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1900208066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:24:27.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 -- 172.21.15.70:0/1365039270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef4100710 msgr2=0x7feef4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 --2- 172.21.15.70:0/1365039270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef4100710 0x7feef4100b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feee0009990 tx=0x7feee0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 -- 172.21.15.70:0/1365039270 shutdown_connections 2023-10-18T12:24:27.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 --2- 172.21.15.70:0/1365039270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef41010d0 0x7feef410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 --2- 172.21.15.70:0/1365039270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feef4106720 0x7feef4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 --2- 172.21.15.70:0/1365039270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef4100710 0x7feef4100b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 -- 172.21.15.70:0/1365039270 >> 172.21.15.70:0/1365039270 conn(0x7feef40fc200 msgr2=0x7feef40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:27.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 -- 172.21.15.70:0/1365039270 shutdown_connections 2023-10-18T12:24:27.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.624+0000 7feefb1ff700 1 -- 172.21.15.70:0/1365039270 wait complete. 2023-10-18T12:24:27.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.625+0000 7feefb1ff700 1 Processor -- start 2023-10-18T12:24:27.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.625+0000 7feefb1ff700 1 -- start start 2023-10-18T12:24:27.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.625+0000 7feefb1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feef4100710 0x7feef419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feefb1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 0x7feef419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feefb1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef4106720 0x7feef41969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feefb1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef41971b0 con 0x7feef41010d0 2023-10-18T12:24:27.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feefb1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef4197320 con 0x7feef4100710 2023-10-18T12:24:27.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feefb1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef41974e0 con 0x7feef4106720 2023-10-18T12:24:27.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 0x7feef419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef8f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feef4100710 0x7feef419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 0x7feef419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48368/0 (socket says 172.21.15.70:48368) 2023-10-18T12:24:27.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 -- 172.21.15.70:0/3804245142 learned_addr learned my addr 172.21.15.70:0/3804245142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:27.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef979c700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef4106720 0x7feef41969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 -- 172.21.15.70:0/3804245142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef4106720 msgr2=0x7feef41969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef4106720 0x7feef41969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 -- 172.21.15.70:0/3804245142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feef4100710 msgr2=0x7feef419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feef4100710 0x7feef419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 -- 172.21.15.70:0/3804245142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00095f0 con 0x7feef41010d0 2023-10-18T12:24:27.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.626+0000 7feef3fff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 0x7feef419ce30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feee0004730 tx=0x7feee0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:27.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.627+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee001d070 con 0x7feef41010d0 2023-10-18T12:24:27.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.627+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee0022470 con 0x7feef41010d0 2023-10-18T12:24:27.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.627+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0017650 con 0x7feef41010d0 2023-10-18T12:24:27.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.627+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4197760 con 0x7feef41010d0 2023-10-18T12:24:27.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.627+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef41a3870 con 0x7feef41010d0 2023-10-18T12:24:27.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.628+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef4108d30 con 0x7feef41010d0 2023-10-18T12:24:27.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.630+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7feee00225e0 con 0x7feef41010d0 2023-10-18T12:24:27.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.631+0000 7feef1ffb700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feee40a14d0 0x7feee40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:27.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.631+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7feee00ce9a0 con 0x7feef41010d0 2023-10-18T12:24:27.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.631+0000 7feef8f9b700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feee40a14d0 0x7feee40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:27.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.632+0000 7feef8f9b700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feee40a14d0 0x7feee40a3990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feee8009be0 tx=0x7feee8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:27.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.633+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feee0097d60 con 0x7feef41010d0 2023-10-18T12:24:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.831+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feef4101e50 con 0x7feef41010d0 2023-10-18T12:24:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.832+0000 7feef1ffb700 1 -- 172.21.15.70:0/3804245142 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7feee009b380 con 0x7feef41010d0 2023-10-18T12:24:27.835 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:24:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feee40a14d0 msgr2=0x7feee40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feee40a14d0 0x7feee40a3990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feee8009be0 tx=0x7feee8009260 comp rx=0 tx=0).stop 2023-10-18T12:24:27.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 msgr2=0x7feef419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:27.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 0x7feef419ce30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feee0004730 tx=0x7feee0004760 comp rx=0 tx=0).stop 2023-10-18T12:24:27.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 shutdown_connections 2023-10-18T12:24:27.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef4106720 0x7feef41969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feee40a14d0 0x7feee40a3990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feef4100710 0x7feef419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 --2- 172.21.15.70:0/3804245142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef41010d0 0x7feef419ce30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:27.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.834+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 >> 172.21.15.70:0/3804245142 conn(0x7feef40fc200 msgr2=0x7feef40fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:27.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.835+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 shutdown_connections 2023-10-18T12:24:27.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:27.835+0000 7feefb1ff700 1 -- 172.21.15.70:0/3804245142 wait complete. 2023-10-18T12:24:27.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:24:28.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 -- 172.21.15.70:0/1390867125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541066b0 msgr2=0x7f5454106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 --2- 172.21.15.70:0/1390867125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541066b0 0x7f5454106a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5444009990 tx=0x7f5444009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 -- 172.21.15.70:0/1390867125 shutdown_connections 2023-10-18T12:24:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 --2- 172.21.15.70:0/1390867125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454101060 0x7f545410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 --2- 172.21.15.70:0/1390867125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541066b0 0x7f5454106a90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 --2- 172.21.15.70:0/1390867125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54541006a0 0x7f5454100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 -- 172.21.15.70:0/1390867125 >> 172.21.15.70:0/1390867125 conn(0x7f54540fc170 msgr2=0x7f54540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 -- 172.21.15.70:0/1390867125 shutdown_connections 2023-10-18T12:24:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.053+0000 7f5459fff700 1 -- 172.21.15.70:0/1390867125 wait complete. 2023-10-18T12:24:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.054+0000 7f5459fff700 1 Processor -- start 2023-10-18T12:24:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 -- start start 2023-10-18T12:24:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541006a0 0x7f545419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5454101060 0x7f545419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 0x7f5454196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5454196f60 con 0x7f54541066b0 2023-10-18T12:24:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54541970d0 con 0x7f5454101060 2023-10-18T12:24:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5459fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5454197290 con 0x7f54541006a0 2023-10-18T12:24:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f5453fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 0x7f5454196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.055+0000 7f54537fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541006a0 0x7f545419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f54537fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541006a0 0x7f545419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:33520/0 (socket says 172.21.15.70:33520) 2023-10-18T12:24:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 0x7f5454196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46122/0 (socket says 172.21.15.70:46122) 2023-10-18T12:24:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f54537fe700 1 -- 172.21.15.70:0/3826868950 learned_addr learned my addr 172.21.15.70:0/3826868950 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f544bfff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5454101060 0x7f545419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 -- 172.21.15.70:0/3826868950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541006a0 msgr2=0x7f545419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541006a0 0x7f545419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 -- 172.21.15.70:0/3826868950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5454101060 msgr2=0x7f545419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5454101060 0x7f545419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 -- 172.21.15.70:0/3826868950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440095f0 con 0x7f54541066b0 2023-10-18T12:24:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5453fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 0x7f5454196900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f544c00efc0 tx=0x7f544c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c018070 con 0x7f54541066b0 2023-10-18T12:24:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f544c00f040 con 0x7f54541066b0 2023-10-18T12:24:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00cd40 con 0x7f54541066b0 2023-10-18T12:24:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.056+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5454197510 con 0x7f54541066b0 2023-10-18T12:24:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.057+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54541a37c0 con 0x7f54541066b0 2023-10-18T12:24:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.057+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5454108c30 con 0x7f54541066b0 2023-10-18T12:24:28.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.059+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f544c003a00 con 0x7f54541066b0 2023-10-18T12:24:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.061+0000 7f54517fa700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54400a14d0 0x7f54400a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.061+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f544c0cd510 con 0x7f54541066b0 2023-10-18T12:24:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.061+0000 7f54537fe700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54400a14d0 0x7f54400a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.062+0000 7f54537fe700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54400a14d0 0x7f54400a3990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5444006010 tx=0x7f544400b280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.063+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f544c096930 con 0x7f54541066b0 2023-10-18T12:24:28.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.282+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5454067560 con 0x7f54541066b0 2023-10-18T12:24:28.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.282+0000 7f54517fa700 1 -- 172.21.15.70:0/3826868950 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v89) v1 ==== 81+0+884 (secure 0 0 0) 0x7f544c099f50 con 0x7f54541066b0 2023-10-18T12:24:28.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1564492170 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:28.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2836082256 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:28.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/392897085 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:28.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1990317867 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:28.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2942646472 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:28.287 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1990317867 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:28.287 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/771435827 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:28.287 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1853570886 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:28.287 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4154455091 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:28.287 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1853570886 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:28.288 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4120653611 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:28.288 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/678282316 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:28.288 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3661620656 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:28.288 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2836082256 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:28.288 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/477422870 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:28.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54400a14d0 msgr2=0x7f54400a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54400a14d0 0x7f54400a3990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5444006010 tx=0x7f544400b280 comp rx=0 tx=0).stop 2023-10-18T12:24:28.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 msgr2=0x7f5454196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 0x7f5454196900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f544c00efc0 tx=0x7f544c00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 shutdown_connections 2023-10-18T12:24:28.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54541066b0 0x7f5454196900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54541006a0 0x7f545419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f54400a14d0 0x7f54400a3990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 --2- 172.21.15.70:0/3826868950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5454101060 0x7f545419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 >> 172.21.15.70:0/3826868950 conn(0x7f54540fc170 msgr2=0x7f54540fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:28.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 shutdown_connections 2023-10-18T12:24:28.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.285+0000 7f5459fff700 1 -- 172.21.15.70:0/3826868950 wait complete. 2023-10-18T12:24:28.292 INFO:teuthology.orchestra.run.smithi070.stderr:listed 15 entries 2023-10-18T12:24:28.306 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:24:28.306 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:24:28.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:28.450 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:28 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2877714790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:24:28.451 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:28 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3804245142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:24:28.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:28 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2877714790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:24:28.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:28 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3804245142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:24:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2752186253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe381006a0 msgr2=0x7fbe38100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2752186253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe381006a0 0x7fbe38100b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0098f0 tx=0x7fbe2c009c00 comp rx=0 tx=0).stop 2023-10-18T12:24:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2752186253 shutdown_connections 2023-10-18T12:24:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2752186253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe38101060 0x7fbe3810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2752186253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe381066b0 0x7fbe38106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2752186253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe381006a0 0x7fbe38100b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2752186253 >> 172.21.15.70:0/2752186253 conn(0x7fbe380fc170 msgr2=0x7fbe380fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2752186253 shutdown_connections 2023-10-18T12:24:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.585+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2752186253 wait complete. 2023-10-18T12:24:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.586+0000 7fbe3dbff700 1 Processor -- start 2023-10-18T12:24:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.586+0000 7fbe3dbff700 1 -- start start 2023-10-18T12:24:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe3dbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe381006a0 0x7fbe38072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe3dbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe38101060 0x7fbe38073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe3dbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 0x7fbe38073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe3dbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe381a4ed0 con 0x7fbe381066b0 2023-10-18T12:24:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe3dbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe381a5040 con 0x7fbe381006a0 2023-10-18T12:24:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe3dbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe381a5200 con 0x7fbe38101060 2023-10-18T12:24:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 0x7fbe38073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 0x7fbe38073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46140/0 (socket says 172.21.15.70:46140) 2023-10-18T12:24:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 -- 172.21.15.70:0/2420531236 learned_addr learned my addr 172.21.15.70:0/2420531236 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe377fe700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe381006a0 0x7fbe38072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe36ffd700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe38101060 0x7fbe38073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 -- 172.21.15.70:0/2420531236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe38101060 msgr2=0x7fbe38073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe38101060 0x7fbe38073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 -- 172.21.15.70:0/2420531236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe381006a0 msgr2=0x7fbe38072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe381006a0 0x7fbe38072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe37fff700 1 -- 172.21.15.70:0/2420531236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c009520 con 0x7fbe381066b0 2023-10-18T12:24:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.587+0000 7fbe36ffd700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe38101060 0x7fbe38073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.588+0000 7fbe37fff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 0x7fbe38073570 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c110 tx=0x7fbe3000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.588+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30017070 con 0x7fbe381066b0 2023-10-18T12:24:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.588+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3000e050 con 0x7fbe381066b0 2023-10-18T12:24:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.588+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe381a5480 con 0x7fbe381066b0 2023-10-18T12:24:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.588+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30007940 con 0x7fbe381066b0 2023-10-18T12:24:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.588+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe381a5940 con 0x7fbe381066b0 2023-10-18T12:24:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.589+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe38108cc0 con 0x7fbe381066b0 2023-10-18T12:24:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.591+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fbe30004ad0 con 0x7fbe381066b0 2023-10-18T12:24:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.592+0000 7fbe34ff9700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe200a14d0 0x7fbe200a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.592+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fbe300ccbb0 con 0x7fbe381066b0 2023-10-18T12:24:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.593+0000 7fbe377fe700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe200a14d0 0x7fbe200a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:28.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.593+0000 7fbe377fe700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe200a14d0 0x7fbe200a3990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009be0 tx=0x7fbe28009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:28.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.596+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbe30095ee0 con 0x7fbe381066b0 2023-10-18T12:24:28.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:28 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2877714790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:24:28.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:28 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3804245142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:24:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.801+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe38073f80 con 0x7fbe381066b0 2023-10-18T12:24:28.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.802+0000 7fbe34ff9700 1 -- 172.21.15.70:0/2420531236 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7fbe30099500 con 0x7fbe381066b0 2023-10-18T12:24:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:28.806 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:28.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe200a14d0 msgr2=0x7fbe200a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe200a14d0 0x7fbe200a3990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009be0 tx=0x7fbe28009260 comp rx=0 tx=0).stop 2023-10-18T12:24:28.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 msgr2=0x7fbe38073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:28.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 0x7fbe38073570 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c110 tx=0x7fbe3000c420 comp rx=0 tx=0).stop 2023-10-18T12:24:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 shutdown_connections 2023-10-18T12:24:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe381066b0 0x7fbe38073570 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe200a14d0 0x7fbe200a3990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe381006a0 0x7fbe38072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 --2- 172.21.15.70:0/2420531236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe38101060 0x7fbe38073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 >> 172.21.15.70:0/2420531236 conn(0x7fbe380fc170 msgr2=0x7fbe380fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 shutdown_connections 2023-10-18T12:24:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:28.805+0000 7fbe3dbff700 1 -- 172.21.15.70:0/2420531236 wait complete. 2023-10-18T12:24:28.827 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:24:28.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:24:29.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 -- 172.21.15.70:0/3122593656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb144100720 msgr2=0x7fb144100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 --2- 172.21.15.70:0/3122593656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb144100720 0x7fb144100ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb138009990 tx=0x7fb138009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 -- 172.21.15.70:0/3122593656 shutdown_connections 2023-10-18T12:24:29.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 --2- 172.21.15.70:0/3122593656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1441010e0 0x7fb14410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 --2- 172.21.15.70:0/3122593656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb144106730 0x7fb144106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 --2- 172.21.15.70:0/3122593656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb144100720 0x7fb144100ba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 -- 172.21.15.70:0/3122593656 >> 172.21.15.70:0/3122593656 conn(0x7fb1440fc1b0 msgr2=0x7fb1440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:29.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.025+0000 7fb1489ff700 1 -- 172.21.15.70:0/3122593656 shutdown_connections 2023-10-18T12:24:29.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.026+0000 7fb1489ff700 1 -- 172.21.15.70:0/3122593656 wait complete. 2023-10-18T12:24:29.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.026+0000 7fb1489ff700 1 Processor -- start 2023-10-18T12:24:29.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 -- start start 2023-10-18T12:24:29.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb144100720 0x7fb144198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1441010e0 0x7fb1441992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 0x7fb14419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14419dcd0 con 0x7fb144106730 2023-10-18T12:24:29.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14419de40 con 0x7fb1441010e0 2023-10-18T12:24:29.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb1489ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14419e000 con 0x7fb144100720 2023-10-18T12:24:29.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb142d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 0x7fb14419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb14259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb144100720 0x7fb144198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb142d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 0x7fb14419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46158/0 (socket says 172.21.15.70:46158) 2023-10-18T12:24:29.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb142d9d700 1 -- 172.21.15.70:0/412245825 learned_addr learned my addr 172.21.15.70:0/412245825 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:29.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.027+0000 7fb141d9b700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1441010e0 0x7fb1441992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb142d9d700 1 -- 172.21.15.70:0/412245825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb144100720 msgr2=0x7fb144198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb142d9d700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb144100720 0x7fb144198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb142d9d700 1 -- 172.21.15.70:0/412245825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1441010e0 msgr2=0x7fb1441992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb142d9d700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1441010e0 0x7fb1441992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb142d9d700 1 -- 172.21.15.70:0/412245825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380095f0 con 0x7fb144106730 2023-10-18T12:24:29.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb142d9d700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 0x7fb14419d690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00ef30 tx=0x7fb13c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:29.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c018070 con 0x7fb144106730 2023-10-18T12:24:29.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13c00f040 con 0x7fb144106730 2023-10-18T12:24:29.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00cd00 con 0x7fb144106730 2023-10-18T12:24:29.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14419e280 con 0x7fb144106730 2023-10-18T12:24:29.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.028+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1441a5b20 con 0x7fb144106730 2023-10-18T12:24:29.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.029+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144101e60 con 0x7fb144106730 2023-10-18T12:24:29.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.031+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb13c003a00 con 0x7fb144106730 2023-10-18T12:24:29.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.033+0000 7fb1337fe700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb12c0a14d0 0x7fb12c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.034+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fb13c0ccc00 con 0x7fb144106730 2023-10-18T12:24:29.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.034+0000 7fb14259c700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb12c0a14d0 0x7fb12c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.034+0000 7fb14259c700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb12c0a14d0 0x7fb12c0a3990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb134009be0 tx=0x7fb134009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:29.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.038+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb13c095fc0 con 0x7fb144106730 2023-10-18T12:24:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.242+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb14419a150 con 0x7fb144106730 2023-10-18T12:24:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.242+0000 7fb1337fe700 1 -- 172.21.15.70:0/412245825 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb13c0995e0 con 0x7fb144106730 2023-10-18T12:24:29.246 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:24:29.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb12c0a14d0 msgr2=0x7fb12c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb12c0a14d0 0x7fb12c0a3990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb134009be0 tx=0x7fb134009260 comp rx=0 tx=0).stop 2023-10-18T12:24:29.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 msgr2=0x7fb14419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 0x7fb14419d690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00ef30 tx=0x7fb13c00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 shutdown_connections 2023-10-18T12:24:29.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb144106730 0x7fb14419d690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb144100720 0x7fb144198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb12c0a14d0 0x7fb12c0a3990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 --2- 172.21.15.70:0/412245825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1441010e0 0x7fb1441992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 >> 172.21.15.70:0/412245825 conn(0x7fb1440fc1b0 msgr2=0x7fb144072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:29.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 shutdown_connections 2023-10-18T12:24:29.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.245+0000 7fb1489ff700 1 -- 172.21.15.70:0/412245825 wait complete. 2023-10-18T12:24:29.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:24:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 -- 172.21.15.70:0/4049307022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 msgr2=0x7f533c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 --2- 172.21.15.70:0/4049307022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c100ba0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5324009990 tx=0x7f5324009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 -- 172.21.15.70:0/4049307022 shutdown_connections 2023-10-18T12:24:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 --2- 172.21.15.70:0/4049307022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1010e0 0x7f533c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 --2- 172.21.15.70:0/4049307022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c106730 0x7f533c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 --2- 172.21.15.70:0/4049307022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c100ba0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 -- 172.21.15.70:0/4049307022 >> 172.21.15.70:0/4049307022 conn(0x7f533c0fc1b0 msgr2=0x7f533c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 -- 172.21.15.70:0/4049307022 shutdown_connections 2023-10-18T12:24:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.463+0000 7f5340dff700 1 -- 172.21.15.70:0/4049307022 wait complete. 2023-10-18T12:24:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:29 smithi186 ceph-mon[135253]: pgmap v264: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-10-18T12:24:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:29 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3826868950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:24:29.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:29 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2420531236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 Processor -- start 2023-10-18T12:24:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 -- start start 2023-10-18T12:24:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1010e0 0x7f533c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c106730 0x7f533c1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c197030 con 0x7f533c100720 2023-10-18T12:24:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1971a0 con 0x7f533c1010e0 2023-10-18T12:24:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f5340dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c197360 con 0x7f533c106730 2023-10-18T12:24:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f533a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f533a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46170/0 (socket says 172.21.15.70:46170) 2023-10-18T12:24:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.464+0000 7f533a59c700 1 -- 172.21.15.70:0/1579867745 learned_addr learned my addr 172.21.15.70:0/1579867745 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f5333fff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1010e0 0x7f533c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533ad9d700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c106730 0x7f533c1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533a59c700 1 -- 172.21.15.70:0/1579867745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c106730 msgr2=0x7f533c1969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533a59c700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c106730 0x7f533c1969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533a59c700 1 -- 172.21.15.70:0/1579867745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1010e0 msgr2=0x7f533c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533a59c700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1010e0 0x7f533c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533a59c700 1 -- 172.21.15.70:0/1579867745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240095f0 con 0x7f533c100720 2023-10-18T12:24:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f533a59c700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c19c8a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e9a0 tx=0x7f532c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c009790 con 0x7f533c100720 2023-10-18T12:24:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f532c00cbd0 con 0x7f533c100720 2023-10-18T12:24:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c197640 con 0x7f533c100720 2023-10-18T12:24:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c018680 con 0x7f533c100720 2023-10-18T12:24:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.465+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1a3890 con 0x7f533c100720 2023-10-18T12:24:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.466+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c108cb0 con 0x7f533c100720 2023-10-18T12:24:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.468+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f532c00cd40 con 0x7f533c100720 2023-10-18T12:24:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.470+0000 7f53337fe700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53280a1550 0x7f53280a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.471+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f532c014070 con 0x7f533c100720 2023-10-18T12:24:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.471+0000 7f5333fff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53280a1550 0x7f53280a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.471+0000 7f5333fff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53280a1550 0x7f53280a3a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f533c197b90 tx=0x7f532400b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.475+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f532c052100 con 0x7f533c100720 2023-10-18T12:24:29.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:29 smithi070 ceph-mon[98639]: pgmap v264: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-10-18T12:24:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:29 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3826868950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:24:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:29 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2420531236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:29.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:29 smithi152 ceph-mon[134767]: pgmap v264: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-10-18T12:24:29.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:29 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3826868950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:24:29.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:29 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2420531236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:29.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.675+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f533c197e90 con 0x7f533c100720 2023-10-18T12:24:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.676+0000 7f53337fe700 1 -- 172.21.15.70:0/1579867745 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f532c096530 con 0x7f533c100720 2023-10-18T12:24:29.679 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:24:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53280a1550 msgr2=0x7f53280a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53280a1550 0x7f53280a3a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f533c197b90 tx=0x7f532400b260 comp rx=0 tx=0).stop 2023-10-18T12:24:29.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 msgr2=0x7f533c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c19c8a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e9a0 tx=0x7f532c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 shutdown_connections 2023-10-18T12:24:29.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f533c106730 0x7f533c1969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c100720 0x7f533c19c8a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53280a1550 0x7f53280a3a10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 --2- 172.21.15.70:0/1579867745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f533c1010e0 0x7f533c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.678+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 >> 172.21.15.70:0/1579867745 conn(0x7f533c0fc1b0 msgr2=0x7f533c0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.679+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 shutdown_connections 2023-10-18T12:24:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.679+0000 7f5340dff700 1 -- 172.21.15.70:0/1579867745 wait complete. 2023-10-18T12:24:29.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:24:29.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 -- 172.21.15.70:0/3672456004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4481010d0 msgr2=0x7fa44810d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 --2- 172.21.15.70:0/3672456004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4481010d0 0x7fa44810d5e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b0a0 tx=0x7fa44000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 -- 172.21.15.70:0/3672456004 shutdown_connections 2023-10-18T12:24:29.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 --2- 172.21.15.70:0/3672456004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4481010d0 0x7fa44810d5e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 --2- 172.21.15.70:0/3672456004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa448106720 0x7fa448106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 --2- 172.21.15.70:0/3672456004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa448100710 0x7fa448100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 -- 172.21.15.70:0/3672456004 >> 172.21.15.70:0/3672456004 conn(0x7fa4480fc200 msgr2=0x7fa4480fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:29.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 -- 172.21.15.70:0/3672456004 shutdown_connections 2023-10-18T12:24:29.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.899+0000 7fa4503ff700 1 -- 172.21.15.70:0/3672456004 wait complete. 2023-10-18T12:24:29.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 Processor -- start 2023-10-18T12:24:29.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 -- start start 2023-10-18T12:24:29.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 0x7fa448072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4481010d0 0x7fa448073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa448106720 0x7fa448073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4481a4ed0 con 0x7fa448100710 2023-10-18T12:24:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4481a5040 con 0x7fa448106720 2023-10-18T12:24:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.900+0000 7fa4503ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4481a5200 con 0x7fa4481010d0 2023-10-18T12:24:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 0x7fa448072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 0x7fa448072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46184/0 (socket says 172.21.15.70:46184) 2023-10-18T12:24:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 -- 172.21.15.70:0/176609851 learned_addr learned my addr 172.21.15.70:0/176609851 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e99c700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa448106720 0x7fa448073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44d99a700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4481010d0 0x7fa448073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 -- 172.21.15.70:0/176609851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4481010d0 msgr2=0x7fa448073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4481010d0 0x7fa448073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 -- 172.21.15.70:0/176609851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa448106720 msgr2=0x7fa448073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa448106720 0x7fa448073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 -- 172.21.15.70:0/176609851 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440009d00 con 0x7fa448100710 2023-10-18T12:24:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa44e19b700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 0x7fa448072af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e9a0 tx=0x7fa44400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:29.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444009790 con 0x7fa448100710 2023-10-18T12:24:29.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa444010430 con 0x7fa448100710 2023-10-18T12:24:29.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4481a54e0 con 0x7fa448100710 2023-10-18T12:24:29.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.901+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444018680 con 0x7fa448100710 2023-10-18T12:24:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.902+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4481a59f0 con 0x7fa448100710 2023-10-18T12:24:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.902+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa448108d10 con 0x7fa448100710 2023-10-18T12:24:29.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.904+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa4440105a0 con 0x7fa448100710 2023-10-18T12:24:29.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.906+0000 7fa43f7fe700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4340a14d0 0x7fa4340a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:29.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.906+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fa444014070 con 0x7fa448100710 2023-10-18T12:24:29.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.906+0000 7fa44d99a700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4340a14d0 0x7fa4340a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.907+0000 7fa44d99a700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4340a14d0 0x7fa4340a3990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa4480741d0 tx=0x7fa43800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:29.908+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa4440966e0 con 0x7fa448100710 2023-10-18T12:24:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.110+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa448073f80 con 0x7fa448100710 2023-10-18T12:24:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.110+0000 7fa43f7fe700 1 -- 172.21.15.70:0/176609851 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa444099d00 con 0x7fa448100710 2023-10-18T12:24:30.114 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:24:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4340a14d0 msgr2=0x7fa4340a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4340a14d0 0x7fa4340a3990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa4480741d0 tx=0x7fa43800b410 comp rx=0 tx=0).stop 2023-10-18T12:24:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 msgr2=0x7fa448072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 0x7fa448072af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e9a0 tx=0x7fa44400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 shutdown_connections 2023-10-18T12:24:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4340a14d0 0x7fa4340a3990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa448106720 0x7fa448073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa448100710 0x7fa448072af0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 --2- 172.21.15.70:0/176609851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4481010d0 0x7fa448073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 >> 172.21.15.70:0/176609851 conn(0x7fa4480fc200 msgr2=0x7fa4480fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:30.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 shutdown_connections 2023-10-18T12:24:30.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.113+0000 7fa4503ff700 1 -- 172.21.15.70:0/176609851 wait complete. 2023-10-18T12:24:30.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:24:30.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.337+0000 7f6222fff700 1 -- 172.21.15.70:0/3250375456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c1010e0 msgr2=0x7f621c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.337+0000 7f6222fff700 1 --2- 172.21.15.70:0/3250375456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c1010e0 0x7f621c10d5f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f621400b0a0 tx=0x7f621400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 -- 172.21.15.70:0/3250375456 shutdown_connections 2023-10-18T12:24:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 --2- 172.21.15.70:0/3250375456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c1010e0 0x7f621c10d5f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 --2- 172.21.15.70:0/3250375456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c106730 0x7f621c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 --2- 172.21.15.70:0/3250375456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621c100720 0x7f621c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 -- 172.21.15.70:0/3250375456 >> 172.21.15.70:0/3250375456 conn(0x7f621c0fc1b0 msgr2=0x7f621c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 -- 172.21.15.70:0/3250375456 shutdown_connections 2023-10-18T12:24:30.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.338+0000 7f6222fff700 1 -- 172.21.15.70:0/3250375456 wait complete. 2023-10-18T12:24:30.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.339+0000 7f6222fff700 1 Processor -- start 2023-10-18T12:24:30.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.339+0000 7f6222fff700 1 -- start start 2023-10-18T12:24:30.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6222fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 0x7f621c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6222fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c1010e0 0x7f621c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6222fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621c106730 0x7f621c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6222fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c197200 con 0x7f621c100720 2023-10-18T12:24:30.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6222fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c197370 con 0x7f621c1010e0 2023-10-18T12:24:30.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6222fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c197530 con 0x7f621c106730 2023-10-18T12:24:30.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 0x7f621c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 0x7f621c19c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46190/0 (socket says 172.21.15.70:46190) 2023-10-18T12:24:30.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 -- 172.21.15.70:0/2998898203 learned_addr learned my addr 172.21.15.70:0/2998898203 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:30.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f622159c700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621c106730 0x7f621c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f621bfff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c1010e0 0x7f621c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 -- 172.21.15.70:0/2998898203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621c106730 msgr2=0x7f621c196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621c106730 0x7f621c196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 -- 172.21.15.70:0/2998898203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c1010e0 msgr2=0x7f621c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c1010e0 0x7f621c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.340+0000 7f6220d9b700 1 -- 172.21.15.70:0/2998898203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009d00 con 0x7f621c100720 2023-10-18T12:24:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.341+0000 7f6220d9b700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 0x7f621c19c990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f621000e9a0 tx=0x7f621000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.341+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210009790 con 0x7f621c100720 2023-10-18T12:24:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.341+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621000cbd0 con 0x7f621c100720 2023-10-18T12:24:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.341+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c197810 con 0x7f621c100720 2023-10-18T12:24:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.341+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210018680 con 0x7f621c100720 2023-10-18T12:24:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.341+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c1a3960 con 0x7f621c100720 2023-10-18T12:24:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.342+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c108d40 con 0x7f621c100720 2023-10-18T12:24:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.344+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f621000cd40 con 0x7f621c100720 2023-10-18T12:24:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.346+0000 7f6219ffb700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f620c0a1550 0x7f620c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.347+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f6210014070 con 0x7f621c100720 2023-10-18T12:24:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.347+0000 7f621bfff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f620c0a1550 0x7f620c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.347+0000 7f621bfff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f620c0a1550 0x7f620c0a3a10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f621c1981a0 tx=0x7f6208009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.350+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6210052100 con 0x7f621c100720 2023-10-18T12:24:30.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:30 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/412245825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:30 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1579867745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:30 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/176609851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.552+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f621c197f10 con 0x7f621c100720 2023-10-18T12:24:30.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.552+0000 7f6219ffb700 1 -- 172.21.15.70:0/2998898203 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6210096510 con 0x7f621c100720 2023-10-18T12:24:30.556 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:24:30.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f620c0a1550 msgr2=0x7f620c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f620c0a1550 0x7f620c0a3a10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f621c1981a0 tx=0x7f6208009260 comp rx=0 tx=0).stop 2023-10-18T12:24:30.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 msgr2=0x7f621c19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 0x7f621c19c990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f621000e9a0 tx=0x7f621000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 shutdown_connections 2023-10-18T12:24:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621c106730 0x7f621c196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f621c100720 0x7f621c19c990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f620c0a1550 0x7f620c0a3a10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 --2- 172.21.15.70:0/2998898203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c1010e0 0x7f621c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 >> 172.21.15.70:0/2998898203 conn(0x7f621c0fc1b0 msgr2=0x7f621c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 shutdown_connections 2023-10-18T12:24:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.555+0000 7f6222fff700 1 -- 172.21.15.70:0/2998898203 wait complete. 2023-10-18T12:24:30.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:30 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/412245825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:30 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1579867745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:30 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/176609851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:24:30.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:30 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/412245825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:30 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1579867745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:30 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/176609851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:24:30.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.773+0000 7efc8efff700 1 -- 172.21.15.70:0/328390931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88073540 msgr2=0x7efc8810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 --2- 172.21.15.70:0/328390931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88073540 0x7efc8810c9a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc74009990 tx=0x7efc74009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 -- 172.21.15.70:0/328390931 shutdown_connections 2023-10-18T12:24:30.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 --2- 172.21.15.70:0/328390931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc8810cee0 0x7efc8810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 --2- 172.21.15.70:0/328390931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc88072c20 0x7efc88073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 --2- 172.21.15.70:0/328390931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88073540 0x7efc8810c9a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 -- 172.21.15.70:0/328390931 >> 172.21.15.70:0/328390931 conn(0x7efc880fc170 msgr2=0x7efc880fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:30.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 -- 172.21.15.70:0/328390931 shutdown_connections 2023-10-18T12:24:30.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.774+0000 7efc8efff700 1 -- 172.21.15.70:0/328390931 wait complete. 2023-10-18T12:24:30.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 Processor -- start 2023-10-18T12:24:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 -- start start 2023-10-18T12:24:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 0x7efc88198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc88073540 0x7efc881992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc8810cee0 0x7efc8819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8819dcd0 con 0x7efc88072c20 2023-10-18T12:24:30.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8819de40 con 0x7efc8810cee0 2023-10-18T12:24:30.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8efff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8819e000 con 0x7efc88073540 2023-10-18T12:24:30.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8cd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 0x7efc88198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8cd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 0x7efc88198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46210/0 (socket says 172.21.15.70:46210) 2023-10-18T12:24:30.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8cd9b700 1 -- 172.21.15.70:0/3630428469 learned_addr learned my addr 172.21.15.70:0/3630428469 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:30.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.775+0000 7efc8d59c700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc8810cee0 0x7efc8819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc87fff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc88073540 0x7efc881992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8cd9b700 1 -- 172.21.15.70:0/3630428469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc88073540 msgr2=0x7efc881992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8cd9b700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc88073540 0x7efc881992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8cd9b700 1 -- 172.21.15.70:0/3630428469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc8810cee0 msgr2=0x7efc8819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8cd9b700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc8810cee0 0x7efc8819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8cd9b700 1 -- 172.21.15.70:0/3630428469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc740095f0 con 0x7efc88072c20 2023-10-18T12:24:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8cd9b700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 0x7efc88198db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e9a0 tx=0x7efc7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:30.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c009790 con 0x7efc88072c20 2023-10-18T12:24:30.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7c010430 con 0x7efc88072c20 2023-10-18T12:24:30.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8819e2e0 con 0x7efc88072c20 2023-10-18T12:24:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c018680 con 0x7efc88072c20 2023-10-18T12:24:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.776+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc881a5b20 con 0x7efc88072c20 2023-10-18T12:24:30.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.777+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc881080f0 con 0x7efc88072c20 2023-10-18T12:24:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.779+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7efc7c0105a0 con 0x7efc88072c20 2023-10-18T12:24:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.781+0000 7efc85ffb700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc780a14d0 0x7efc780a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.781+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7efc7c014070 con 0x7efc88072c20 2023-10-18T12:24:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.781+0000 7efc87fff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc780a14d0 0x7efc780a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.781+0000 7efc87fff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc780a14d0 0x7efc780a3990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc8819a300 tx=0x7efc74005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:30.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.783+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efc7c096710 con 0x7efc88072c20 2023-10-18T12:24:31.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.999+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efc880742d0 con 0x7efc88072c20 2023-10-18T12:24:31.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:30.999+0000 7efc85ffb700 1 -- 172.21.15.70:0/3630428469 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v89) v1 ==== 81+0+884 (secure 0 0 0) 0x7efc7c026390 con 0x7efc88072c20 2023-10-18T12:24:31.003 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1564492170 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:31.003 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2836082256 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:31.003 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/392897085 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:31.003 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1990317867 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:31.003 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2942646472 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1990317867 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/771435827 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1853570886 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4154455091 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1853570886 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:31.005 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4120653611 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:31.005 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/678282316 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:31.005 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3661620656 2023-10-19T12:16:35.958063+0000 2023-10-18T12:24:31.005 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2836082256 2023-10-19T12:18:13.515343+0000 2023-10-18T12:24:31.005 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/477422870 2023-10-19T12:17:02.382050+0000 2023-10-18T12:24:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.001+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc780a14d0 msgr2=0x7efc780a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.001+0000 7efc8efff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc780a14d0 0x7efc780a3990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc8819a300 tx=0x7efc74005fd0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.001+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 msgr2=0x7efc88198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.001+0000 7efc8efff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 0x7efc88198db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e9a0 tx=0x7efc7c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.001+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 shutdown_connections 2023-10-18T12:24:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efc780a14d0 0x7efc780a3990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc8810cee0 0x7efc8819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc88072c20 0x7efc88198db0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 --2- 172.21.15.70:0/3630428469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc88073540 0x7efc881992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 >> 172.21.15.70:0/3630428469 conn(0x7efc880fc170 msgr2=0x7efc880fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:31.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 shutdown_connections 2023-10-18T12:24:31.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.002+0000 7efc8efff700 1 -- 172.21.15.70:0/3630428469 wait complete. 2023-10-18T12:24:31.009 INFO:teuthology.orchestra.run.smithi070.stderr:listed 15 entries 2023-10-18T12:24:31.023 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-10-18T12:24:31.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:31.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-18T12:24:31.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:ceph-fuse[160301]: fuse finished with error 0 and tester_r 0 2023-10-18T12:24:31.093 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:24:31.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:24:31.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:24:31.134 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-18T12:24:31.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:31.295 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:31 smithi070 ceph-mon[98639]: pgmap v265: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-10-18T12:24:31.295 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:31 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2998898203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:24:31.296 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:31 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3630428469' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:24:31.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 -- 172.21.15.70:0/401979779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 msgr2=0x7f1af410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 --2- 172.21.15.70:0/401979779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af410d570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b0a0 tx=0x7f1aec00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 -- 172.21.15.70:0/401979779 shutdown_connections 2023-10-18T12:24:31.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 --2- 172.21.15.70:0/401979779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af410d570 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 --2- 172.21.15.70:0/401979779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af41066b0 0x7f1af4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 --2- 172.21.15.70:0/401979779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 -- 172.21.15.70:0/401979779 >> 172.21.15.70:0/401979779 conn(0x7f1af40fc1d0 msgr2=0x7f1af40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:31.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 -- 172.21.15.70:0/401979779 shutdown_connections 2023-10-18T12:24:31.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.435+0000 7f1af99ff700 1 -- 172.21.15.70:0/401979779 wait complete. 2023-10-18T12:24:31.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.436+0000 7f1af99ff700 1 Processor -- start 2023-10-18T12:24:31.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.436+0000 7f1af99ff700 1 -- start start 2023-10-18T12:24:31.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.436+0000 7f1af99ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.436+0000 7f1af99ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.436+0000 7f1af99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af41066b0 0x7f1af4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af99ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41992e0 con 0x7f1af4101060 2023-10-18T12:24:31.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af99ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4199450 con 0x7f1af41006a0 2023-10-18T12:24:31.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4199610 con 0x7f1af41066b0 2023-10-18T12:24:31.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1aebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34820/0 (socket says 172.21.15.70:34820) 2023-10-18T12:24:31.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 -- 172.21.15.70:0/1345903925 learned_addr learned my addr 172.21.15.70:0/1345903925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:31.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af37fe700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af41066b0 0x7f1af4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 -- 172.21.15.70:0/1345903925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af41066b0 msgr2=0x7f1af4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af41066b0 0x7f1af4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 -- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 msgr2=0x7f1af419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 -- 172.21.15.70:0/1345903925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec009d00 con 0x7f1af41006a0 2023-10-18T12:24:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1aebfff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.437+0000 7f1af2ffd700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af419ea70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9a0 tx=0x7f1ae400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.438+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4009790 con 0x7f1af41006a0 2023-10-18T12:24:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.438+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af41998f0 con 0x7f1af41006a0 2023-10-18T12:24:31.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.438+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ae400cbd0 con 0x7f1af41006a0 2023-10-18T12:24:31.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.438+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4018680 con 0x7f1af41006a0 2023-10-18T12:24:31.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.438+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af41a59f0 con 0x7f1af41006a0 2023-10-18T12:24:31.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.439+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af4101de0 con 0x7f1af41006a0 2023-10-18T12:24:31.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.441+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1ae400cd40 con 0x7f1af41006a0 2023-10-18T12:24:31.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.443+0000 7f1af89fd700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ae00a14d0 0x7f1ae00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.443+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f1ae4014070 con 0x7f1af41006a0 2023-10-18T12:24:31.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.443+0000 7f1aebfff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ae00a14d0 0x7f1ae00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.443+0000 7f1aebfff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ae00a14d0 0x7f1ae00a3990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1af419a280 tx=0x7f1adc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:31.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.446+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ae4052100 con 0x7f1af41006a0 2023-10-18T12:24:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:31 smithi186 ceph-mon[135253]: pgmap v265: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-10-18T12:24:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:31 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2998898203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:24:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:31 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3630428469' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:24:31.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.652+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1af4067530 con 0x7f1af41006a0 2023-10-18T12:24:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.654+0000 7f1af89fd700 1 -- 172.21.15.70:0/1345903925 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f1ae40965c0 con 0x7f1af41006a0 2023-10-18T12:24:31.657 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:31.657 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ae00a14d0 msgr2=0x7f1ae00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ae00a14d0 0x7f1ae00a3990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1af419a280 tx=0x7f1adc009260 comp rx=0 tx=0).stop 2023-10-18T12:24:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 msgr2=0x7f1af419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af419ea70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9a0 tx=0x7f1ae400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 shutdown_connections 2023-10-18T12:24:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af41066b0 0x7f1af4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ae00a14d0 0x7f1ae00a3990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af41006a0 0x7f1af419ea70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 --2- 172.21.15.70:0/1345903925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4101060 0x7f1af419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 >> 172.21.15.70:0/1345903925 conn(0x7f1af40fc1d0 msgr2=0x7f1af410b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:31.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 shutdown_connections 2023-10-18T12:24:31.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.656+0000 7f1af99ff700 1 -- 172.21.15.70:0/1345903925 wait complete. 2023-10-18T12:24:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:31 smithi152 ceph-mon[134767]: pgmap v265: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-10-18T12:24:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:31 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2998898203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:24:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:31 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3630428469' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:24:31.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:31.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 -- 172.21.15.70:0/407096821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248073540 msgr2=0x7f824810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 --2- 172.21.15.70:0/407096821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248073540 0x7f824810c9a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8230009990 tx=0x7f8230009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 -- 172.21.15.70:0/407096821 shutdown_connections 2023-10-18T12:24:31.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 --2- 172.21.15.70:0/407096821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 0x7f824810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 --2- 172.21.15.70:0/407096821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8248072c20 0x7f8248073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 --2- 172.21.15.70:0/407096821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248073540 0x7f824810c9a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 -- 172.21.15.70:0/407096821 >> 172.21.15.70:0/407096821 conn(0x7f82480fc170 msgr2=0x7f82480fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:31.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 -- 172.21.15.70:0/407096821 shutdown_connections 2023-10-18T12:24:31.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.876+0000 7f824cfff700 1 -- 172.21.15.70:0/407096821 wait complete. 2023-10-18T12:24:31.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 Processor -- start 2023-10-18T12:24:31.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 -- start start 2023-10-18T12:24:31.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 0x7f8248198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8248073540 0x7f82481992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 0x7f824819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819dcb0 con 0x7f8248072c20 2023-10-18T12:24:31.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819de20 con 0x7f8248073540 2023-10-18T12:24:31.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819dfe0 con 0x7f824810cee0 2023-10-18T12:24:31.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.877+0000 7f824659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 0x7f8248198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 0x7f8248198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46232/0 (socket says 172.21.15.70:46232) 2023-10-18T12:24:31.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 -- 172.21.15.70:0/2185718011 learned_addr learned my addr 172.21.15.70:0/2185718011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:31.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f823ffff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8248073540 0x7f82481992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f8246d9d700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 0x7f824819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 -- 172.21.15.70:0/2185718011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 msgr2=0x7f824819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 0x7f824819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 -- 172.21.15.70:0/2185718011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8248073540 msgr2=0x7f82481992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:31.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8248073540 0x7f82481992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:31.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 -- 172.21.15.70:0/2185718011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300095f0 con 0x7f8248072c20 2023-10-18T12:24:31.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f8246d9d700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 0x7f824819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:31.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824659c700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 0x7f8248198d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f823800d950 tx=0x7f823800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:31.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238009940 con 0x7f8248072c20 2023-10-18T12:24:31.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8238010460 con 0x7f8248072c20 2023-10-18T12:24:31.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824819e2c0 con 0x7f8248072c20 2023-10-18T12:24:31.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.878+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823800f620 con 0x7f8248072c20 2023-10-18T12:24:31.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.879+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82481a5b00 con 0x7f8248072c20 2023-10-18T12:24:31.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.880+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82481080f0 con 0x7f8248072c20 2023-10-18T12:24:31.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.881+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f82380108d0 con 0x7f8248072c20 2023-10-18T12:24:31.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.883+0000 7f823f7fe700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82340a1550 0x7f82340a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:31.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.883+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f82380cce60 con 0x7f8248072c20 2023-10-18T12:24:31.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.883+0000 7f823ffff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82340a1550 0x7f82340a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:31.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.884+0000 7f823ffff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82340a1550 0x7f82340a3a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f824819a2e0 tx=0x7f823000b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:31.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:31.887+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8238096190 con 0x7f8248072c20 2023-10-18T12:24:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.087+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82480742d0 con 0x7f8248072c20 2023-10-18T12:24:32.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.089+0000 7f823f7fe700 1 -- 172.21.15.70:0/2185718011 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f8238016090 con 0x7f8248072c20 2023-10-18T12:24:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:32.095 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.091+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82340a1550 msgr2=0x7f82340a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.091+0000 7f824cfff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82340a1550 0x7f82340a3a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f824819a2e0 tx=0x7f823000b540 comp rx=0 tx=0).stop 2023-10-18T12:24:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.091+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 msgr2=0x7f8248198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.091+0000 7f824cfff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 0x7f8248198d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f823800d950 tx=0x7f823800dc60 comp rx=0 tx=0).stop 2023-10-18T12:24:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 shutdown_connections 2023-10-18T12:24:32.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f824810cee0 0x7f824819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8248072c20 0x7f8248198d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82340a1550 0x7f82340a3a10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 --2- 172.21.15.70:0/2185718011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8248073540 0x7f82481992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 >> 172.21.15.70:0/2185718011 conn(0x7f82480fc170 msgr2=0x7f82480fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 shutdown_connections 2023-10-18T12:24:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.092+0000 7f824cfff700 1 -- 172.21.15.70:0/2185718011 wait complete. 2023-10-18T12:24:32.114 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T12:24:32.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2382560433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 msgr2=0x7fdfa4106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:32.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2382560433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4106b10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009990 tx=0x7fdf94009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2382560433 shutdown_connections 2023-10-18T12:24:32.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2382560433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa41010e0 0x7fdfa410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2382560433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4106b10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2382560433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa4100720 0x7fdfa4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2382560433 >> 172.21.15.70:0/2382560433 conn(0x7fdfa40fc1b0 msgr2=0x7fdfa40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:32.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2382560433 shutdown_connections 2023-10-18T12:24:32.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.312+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2382560433 wait complete. 2023-10-18T12:24:32.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 Processor -- start 2023-10-18T12:24:32.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 -- start start 2023-10-18T12:24:32.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa4100720 0x7fdfa419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:32.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa41010e0 0x7fdfa419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:32.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:32.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4199340 con 0x7fdfa4106730 2023-10-18T12:24:32.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa41994b0 con 0x7fdfa4100720 2023-10-18T12:24:32.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.313+0000 7fdfaa9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4199670 con 0x7fdfa41010e0 2023-10-18T12:24:32.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:32.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa4100720 0x7fdfa419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:32.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46256/0 (socket says 172.21.15.70:46256) 2023-10-18T12:24:32.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 -- 172.21.15.70:0/2139993411 learned_addr learned my addr 172.21.15.70:0/2139993411 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:32.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdf9bfff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa41010e0 0x7fdfa419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:32.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 -- 172.21.15.70:0/2139993411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa41010e0 msgr2=0x7fdfa419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:32.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa41010e0 0x7fdfa419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 -- 172.21.15.70:0/2139993411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa4100720 msgr2=0x7fdfa419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:32.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa4100720 0x7fdfa419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:32.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 -- 172.21.15.70:0/2139993411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940095f0 con 0x7fdfa4106730 2023-10-18T12:24:32.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.314+0000 7fdfa8f9c700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4198b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c007ee0 tx=0x7fdf9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:32.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.315+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c017070 con 0x7fdfa4106730 2023-10-18T12:24:32.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.315+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf9c00f040 con 0x7fdfa4106730 2023-10-18T12:24:32.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.315+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa4199950 con 0x7fdfa4106730 2023-10-18T12:24:32.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.315+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c005740 con 0x7fdfa4106730 2023-10-18T12:24:32.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.315+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa41a5a50 con 0x7fdfa4106730 2023-10-18T12:24:32.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.316+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa4101e60 con 0x7fdfa4106730 2023-10-18T12:24:32.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.318+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fdf9c00dc60 con 0x7fdfa4106730 2023-10-18T12:24:32.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.319+0000 7fdfa1ffb700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a14d0 0x7fdf900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:32.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.319+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fdf9c0cce90 con 0x7fdfa4106730 2023-10-18T12:24:32.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.319+0000 7fdfa3fff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a14d0 0x7fdf900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:32.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.320+0000 7fdfa3fff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a14d0 0x7fdf900a3990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf94006010 tx=0x7fdf9400b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:32.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.321+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdf9c0961c0 con 0x7fdfa4106730 2023-10-18T12:24:32.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:32 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1345903925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:32 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2185718011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:32.570+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdfa40675a0 con 0x7fdfa4106730 2023-10-18T12:24:32.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:32 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1345903925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:32.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:32 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2185718011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:32.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:32 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1345903925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:32.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:32 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2185718011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:33.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.184+0000 7fdfa1ffb700 1 -- 172.21.15.70:0/2139993411 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdf9c0997e0 con 0x7fdfa4106730 2023-10-18T12:24:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a14d0 msgr2=0x7fdf900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a14d0 0x7fdf900a3990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf94006010 tx=0x7fdf9400b540 comp rx=0 tx=0).stop 2023-10-18T12:24:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 msgr2=0x7fdfa4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4198b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c007ee0 tx=0x7fdf9c00d3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 shutdown_connections 2023-10-18T12:24:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa4106730 0x7fdfa4198b70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a14d0 0x7fdf900a3990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa4100720 0x7fdfa419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 --2- 172.21.15.70:0/2139993411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa41010e0 0x7fdfa419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 >> 172.21.15.70:0/2139993411 conn(0x7fdfa40fc1b0 msgr2=0x7fdfa410bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:33.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 shutdown_connections 2023-10-18T12:24:33.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.188+0000 7fdfaa9ff700 1 -- 172.21.15.70:0/2139993411 wait complete. 2023-10-18T12:24:33.212 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-10-18T12:24:33.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-18T12:24:33.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 -- 172.21.15.70:0/935066355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac106730 msgr2=0x7fc8ac106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:33.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/935066355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac106730 0x7fc8ac106b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009990 tx=0x7fc89c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 -- 172.21.15.70:0/935066355 shutdown_connections 2023-10-18T12:24:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/935066355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/935066355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac106730 0x7fc8ac106b10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/935066355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ac100720 0x7fc8ac100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 -- 172.21.15.70:0/935066355 >> 172.21.15.70:0/935066355 conn(0x7fc8ac0fc1b0 msgr2=0x7fc8ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 -- 172.21.15.70:0/935066355 shutdown_connections 2023-10-18T12:24:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.412+0000 7fc8b1dff700 1 -- 172.21.15.70:0/935066355 wait complete. 2023-10-18T12:24:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.413+0000 7fc8b1dff700 1 Processor -- start 2023-10-18T12:24:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.413+0000 7fc8b1dff700 1 -- start start 2023-10-18T12:24:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.413+0000 7fc8b1dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ac100720 0x7fc8ac19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.413+0000 7fc8b1dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:33.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.413+0000 7fc8b1dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8ac106730 0x7fc8ac198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:33.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8b1dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac199200 con 0x7fc8ac1010e0 2023-10-18T12:24:33.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8b1dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac199370 con 0x7fc8ac106730 2023-10-18T12:24:33.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8b1dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac199530 con 0x7fc8ac100720 2023-10-18T12:24:33.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:33.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46262/0 (socket says 172.21.15.70:46262) 2023-10-18T12:24:33.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 -- 172.21.15.70:0/4149581955 learned_addr learned my addr 172.21.15.70:0/4149581955 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:33.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8abfff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8ac106730 0x7fc8ac198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:33.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8ab7fe700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ac100720 0x7fc8ac19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:33.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 -- 172.21.15.70:0/4149581955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ac100720 msgr2=0x7fc8ac19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:33.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ac100720 0x7fc8ac19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 -- 172.21.15.70:0/4149581955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8ac106730 msgr2=0x7fc8ac198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:33.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8ac106730 0x7fc8ac198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:33.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 -- 172.21.15.70:0/4149581955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0095f0 con 0x7fc8ac1010e0 2023-10-18T12:24:33.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.414+0000 7fc8a3fff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac19efb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc89400e9a0 tx=0x7fc89400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:33.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.415+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894009790 con 0x7fc8ac1010e0 2023-10-18T12:24:33.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.415+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc894010430 con 0x7fc8ac1010e0 2023-10-18T12:24:33.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.415+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894018680 con 0x7fc8ac1010e0 2023-10-18T12:24:33.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.415+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac199810 con 0x7fc8ac1010e0 2023-10-18T12:24:33.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.415+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac1a5a60 con 0x7fc8ac1010e0 2023-10-18T12:24:33.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.416+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac101e60 con 0x7fc8ac1010e0 2023-10-18T12:24:33.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.418+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc8940105a0 con 0x7fc8ac1010e0 2023-10-18T12:24:33.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.419+0000 7fc8a97fa700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc8980a1550 0x7fc8980a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:33.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.419+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fc894014070 con 0x7fc8ac1010e0 2023-10-18T12:24:33.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.419+0000 7fc8ab7fe700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc8980a1550 0x7fc8980a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:33.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.420+0000 7fc8ab7fe700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc8980a1550 0x7fc8980a3a10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc89c005b00 tx=0x7fc89c005a90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:33.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.423+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc894096510 con 0x7fc8ac1010e0 2023-10-18T12:24:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:33 smithi186 ceph-mon[135253]: pgmap v266: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T12:24:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:33 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2139993411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T12:24:33.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:33 smithi070 ceph-mon[98639]: pgmap v266: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T12:24:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:33 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2139993411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T12:24:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:33 smithi152 ceph-mon[134767]: pgmap v266: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T12:24:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:33 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2139993411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T12:24:33.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:33.671+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fc8ac0675a0 con 0x7fc8ac1010e0 2023-10-18T12:24:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.118+0000 7fc8a97fa700 1 -- 172.21.15.70:0/4149581955 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc894099b30 con 0x7fc8ac1010e0 2023-10-18T12:24:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.122+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc8980a1550 msgr2=0x7fc8980a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.122+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc8980a1550 0x7fc8980a3a10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc89c005b00 tx=0x7fc89c005a90 comp rx=0 tx=0).stop 2023-10-18T12:24:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.122+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 msgr2=0x7fc8ac19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.122+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac19efb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc89400e9a0 tx=0x7fc89400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 shutdown_connections 2023-10-18T12:24:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc8980a1550 0x7fc8980a3a10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8ac106730 0x7fc8ac198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ac100720 0x7fc8ac19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 --2- 172.21.15.70:0/4149581955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ac1010e0 0x7fc8ac19efb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 >> 172.21.15.70:0/4149581955 conn(0x7fc8ac0fc1b0 msgr2=0x7fc8ac10be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 shutdown_connections 2023-10-18T12:24:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.123+0000 7fc8b1dff700 1 -- 172.21.15.70:0/4149581955 wait complete. 2023-10-18T12:24:34.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-18T12:24:34.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.370+0000 7fb62e5ff700 1 -- 172.21.15.70:0/3436658316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628100720 msgr2=0x7fb628100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:34.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.370+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/3436658316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628100720 0x7fb628100ba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009990 tx=0x7fb61c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 -- 172.21.15.70:0/3436658316 shutdown_connections 2023-10-18T12:24:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/3436658316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6281010e0 0x7fb62810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/3436658316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb628106730 0x7fb628106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/3436658316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628100720 0x7fb628100ba0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 -- 172.21.15.70:0/3436658316 >> 172.21.15.70:0/3436658316 conn(0x7fb6280fc1b0 msgr2=0x7fb6280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 -- 172.21.15.70:0/3436658316 shutdown_connections 2023-10-18T12:24:34.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.371+0000 7fb62e5ff700 1 -- 172.21.15.70:0/3436658316 wait complete. 2023-10-18T12:24:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 Processor -- start 2023-10-18T12:24:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 -- start start 2023-10-18T12:24:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb628100720 0x7fb628072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6281010e0 0x7fb628073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 0x7fb628073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628074a70 con 0x7fb628106730 2023-10-18T12:24:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281a9570 con 0x7fb6281010e0 2023-10-18T12:24:34.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62e5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281a96e0 con 0x7fb628100720 2023-10-18T12:24:34.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62cb9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 0x7fb628073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:34.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62cb9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 0x7fb628073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46284/0 (socket says 172.21.15.70:46284) 2023-10-18T12:24:34.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb62cb9c700 1 -- 172.21.15.70:0/2293848519 learned_addr learned my addr 172.21.15.70:0/2293848519 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:34.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.372+0000 7fb627fff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb628100720 0x7fb628072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb6277fe700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6281010e0 0x7fb628073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62cb9c700 1 -- 172.21.15.70:0/2293848519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb628100720 msgr2=0x7fb628072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62cb9c700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb628100720 0x7fb628072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62cb9c700 1 -- 172.21.15.70:0/2293848519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6281010e0 msgr2=0x7fb628073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62cb9c700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6281010e0 0x7fb628073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62cb9c700 1 -- 172.21.15.70:0/2293848519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0095f0 con 0x7fb628106730 2023-10-18T12:24:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62cb9c700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 0x7fb628073570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb62000ef30 tx=0x7fb62000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620018070 con 0x7fb628106730 2023-10-18T12:24:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb62000f040 con 0x7fb628106730 2023-10-18T12:24:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6281a9960 con 0x7fb628106730 2023-10-18T12:24:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62000cd00 con 0x7fb628106730 2023-10-18T12:24:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.373+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6281a9e70 con 0x7fb628106730 2023-10-18T12:24:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.374+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb628108d40 con 0x7fb628106730 2023-10-18T12:24:34.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.376+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb620003a00 con 0x7fb628106730 2023-10-18T12:24:34.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.378+0000 7fb6257fa700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6100a1550 0x7fb6100a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:34.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.378+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fb6200ccb30 con 0x7fb628106730 2023-10-18T12:24:34.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.378+0000 7fb627fff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6100a1550 0x7fb6100a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:34.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.379+0000 7fb627fff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6100a1550 0x7fb6100a3a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb618009be0 tx=0x7fb618009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:34.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.382+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb620095ef0 con 0x7fb628106730 2023-10-18T12:24:34.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:34 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2139993411' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-18T12:24:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:34 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:34 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4149581955' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-18T12:24:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:34 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4149581955' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-18T12:24:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:34 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-18T12:24:34.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:34 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2139993411' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-18T12:24:34.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:34 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:34.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:34 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4149581955' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-18T12:24:34.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:34 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4149581955' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-18T12:24:34.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:34 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-18T12:24:34.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:34.630+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fb628101e60 con 0x7fb628106730 2023-10-18T12:24:34.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:34 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2139993411' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-18T12:24:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:34 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T12:24:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:34 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4149581955' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-18T12:24:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:34 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4149581955' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-18T12:24:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:34 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-18T12:24:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.201+0000 7fb6257fa700 1 -- 172.21.15.70:0/2293848519 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb620099510 con 0x7fb628106730 2023-10-18T12:24:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6100a1550 msgr2=0x7fb6100a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6100a1550 0x7fb6100a3a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb618009be0 tx=0x7fb618009260 comp rx=0 tx=0).stop 2023-10-18T12:24:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 msgr2=0x7fb628073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 0x7fb628073570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb62000ef30 tx=0x7fb62000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 shutdown_connections 2023-10-18T12:24:35.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb628106730 0x7fb628073570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb628100720 0x7fb628072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6100a1550 0x7fb6100a3a10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 --2- 172.21.15.70:0/2293848519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6281010e0 0x7fb628073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 >> 172.21.15.70:0/2293848519 conn(0x7fb6280fc1b0 msgr2=0x7fb6280fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 shutdown_connections 2023-10-18T12:24:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.204+0000 7fb62e5ff700 1 -- 172.21.15.70:0/2293848519 wait complete. 2023-10-18T12:24:35.226 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-10-18T12:24:35.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-18T12:24:35.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 -- 172.21.15.70:0/4111648174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510106730 msgr2=0x7f6510106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 --2- 172.21.15.70:0/4111648174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510106730 0x7f6510106b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f650c009990 tx=0x7f650c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 -- 172.21.15.70:0/4111648174 shutdown_connections 2023-10-18T12:24:35.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 --2- 172.21.15.70:0/4111648174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 --2- 172.21.15.70:0/4111648174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510106730 0x7f6510106b10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 --2- 172.21.15.70:0/4111648174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6510100720 0x7f6510100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.425+0000 7f65185ff700 1 -- 172.21.15.70:0/4111648174 >> 172.21.15.70:0/4111648174 conn(0x7f65100fc1b0 msgr2=0x7f65100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:35.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.426+0000 7f65185ff700 1 -- 172.21.15.70:0/4111648174 shutdown_connections 2023-10-18T12:24:35.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.426+0000 7f65185ff700 1 -- 172.21.15.70:0/4111648174 wait complete. 2023-10-18T12:24:35.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.426+0000 7f65185ff700 1 Processor -- start 2023-10-18T12:24:35.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 -- start start 2023-10-18T12:24:35.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510100720 0x7f651019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6510106730 0x7f6510198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101991f0 con 0x7f65101010e0 2023-10-18T12:24:35.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510199360 con 0x7f6510100720 2023-10-18T12:24:35.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f65185ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510199520 con 0x7f6510106730 2023-10-18T12:24:35.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f6515b9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f6515b9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46310/0 (socket says 172.21.15.70:46310) 2023-10-18T12:24:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f6516b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6510106730 0x7f6510198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f6515b9a700 1 -- 172.21.15.70:0/2455431527 learned_addr learned my addr 172.21.15.70:0/2455431527 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.427+0000 7f651639b700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510100720 0x7f651019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f6515b9a700 1 -- 172.21.15.70:0/2455431527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6510106730 msgr2=0x7f6510198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f6515b9a700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6510106730 0x7f6510198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f6515b9a700 1 -- 172.21.15.70:0/2455431527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510100720 msgr2=0x7f651019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f6515b9a700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510100720 0x7f651019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f6515b9a700 1 -- 172.21.15.70:0/2455431527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0095f0 con 0x7f65101010e0 2023-10-18T12:24:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f6515b9a700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651019efa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f650000e9a0 tx=0x7f650000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500009790 con 0x7f65101010e0 2023-10-18T12:24:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6510199800 con 0x7f65101010e0 2023-10-18T12:24:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6500010430 con 0x7f65101010e0 2023-10-18T12:24:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65101a5a50 con 0x7f65101010e0 2023-10-18T12:24:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.428+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500018620 con 0x7f65101010e0 2023-10-18T12:24:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.429+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6510101e60 con 0x7f65101010e0 2023-10-18T12:24:35.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.431+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f65000105a0 con 0x7f65101010e0 2023-10-18T12:24:35.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.433+0000 7f65077fe700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64fc0a14d0 0x7f64fc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.433+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f6500014070 con 0x7f65101010e0 2023-10-18T12:24:35.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.433+0000 7f651639b700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64fc0a14d0 0x7f64fc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.433+0000 7f651639b700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64fc0a14d0 0x7f64fc0a3990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098e0 tx=0x7f650c005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:35.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.434+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6500096730 con 0x7f65101010e0 2023-10-18T12:24:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:35 smithi186 ceph-mon[135253]: pgmap v267: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T12:24:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2293848519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-18T12:24:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-mon[98639]: pgmap v267: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T12:24:35.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2293848519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-18T12:24:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:35 smithi152 ceph-mon[134767]: pgmap v267: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T12:24:35.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2293848519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-18T12:24:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.669+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f6510110650 con 0x7f64fc0a14d0 2023-10-18T12:24:35.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.709+0000 7f65077fe700 1 -- 172.21.15.70:0/2455431527 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6510110650 con 0x7f64fc0a14d0 2023-10-18T12:24:35.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64fc0a14d0 msgr2=0x7f64fc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64fc0a14d0 0x7f64fc0a3990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098e0 tx=0x7f650c005960 comp rx=0 tx=0).stop 2023-10-18T12:24:35.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 msgr2=0x7f651019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651019efa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f650000e9a0 tx=0x7f650000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 shutdown_connections 2023-10-18T12:24:35.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6510106730 0x7f6510198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f64fc0a14d0 0x7f64fc0a3990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6510100720 0x7f651019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 --2- 172.21.15.70:0/2455431527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65101010e0 0x7f651019efa0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 >> 172.21.15.70:0/2455431527 conn(0x7f65100fc1b0 msgr2=0x7f651010be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:35.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.711+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 shutdown_connections 2023-10-18T12:24:35.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.712+0000 7f65185ff700 1 -- 172.21.15.70:0/2455431527 wait complete. 2023-10-18T12:24:35.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-18T12:24:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 -- 172.21.15.70:0/2515807943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67540ff640 msgr2=0x7f67540ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 --2- 172.21.15.70:0/2515807943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67540ff640 0x7f67540ffac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6740009990 tx=0x7f6740009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 -- 172.21.15.70:0/2515807943 shutdown_connections 2023-10-18T12:24:35.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 --2- 172.21.15.70:0/2515807943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6754100000 0x7f675410c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 --2- 172.21.15.70:0/2515807943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67541056e0 0x7f6754105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 --2- 172.21.15.70:0/2515807943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67540ff640 0x7f67540ffac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 -- 172.21.15.70:0/2515807943 >> 172.21.15.70:0/2515807943 conn(0x7f67540767f0 msgr2=0x7f6754076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 -- 172.21.15.70:0/2515807943 shutdown_connections 2023-10-18T12:24:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.932+0000 7f675afff700 1 -- 172.21.15.70:0/2515807943 wait complete. 2023-10-18T12:24:35.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.933+0000 7f675afff700 1 Processor -- start 2023-10-18T12:24:35.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.933+0000 7f675afff700 1 -- start start 2023-10-18T12:24:35.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675afff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 0x7f6754198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675afff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6754100000 0x7f67541993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675afff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67541056e0 0x7f675419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675afff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419dd90 con 0x7f67541056e0 2023-10-18T12:24:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675afff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419df00 con 0x7f67540ff640 2023-10-18T12:24:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675afff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419e0c0 con 0x7f6754100000 2023-10-18T12:24:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675959c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67541056e0 0x7f675419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f6758d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 0x7f6754198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f6758d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 0x7f6754198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34932/0 (socket says 172.21.15.70:34932) 2023-10-18T12:24:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f6758d9b700 1 -- 172.21.15.70:0/1259194176 learned_addr learned my addr 172.21.15.70:0/1259194176 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f6753fff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6754100000 0x7f67541993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675959c700 1 -- 172.21.15.70:0/1259194176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6754100000 msgr2=0x7f67541993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675959c700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6754100000 0x7f67541993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675959c700 1 -- 172.21.15.70:0/1259194176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 msgr2=0x7f6754198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675959c700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 0x7f6754198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f675959c700 1 -- 172.21.15.70:0/1259194176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400095f0 con 0x7f67541056e0 2023-10-18T12:24:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.934+0000 7f6758d9b700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 0x7f6754198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.935+0000 7f675959c700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67541056e0 0x7f675419d750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f674c007f10 tx=0x7f674c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.935+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c004db0 con 0x7f67541056e0 2023-10-18T12:24:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.935+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c00f040 con 0x7f67541056e0 2023-10-18T12:24:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.935+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0042f0 con 0x7f67541056e0 2023-10-18T12:24:35.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.935+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675419e3a0 con 0x7f67541056e0 2023-10-18T12:24:35.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.935+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67541a5be0 con 0x7f67541056e0 2023-10-18T12:24:35.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.936+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6754100d30 con 0x7f67541056e0 2023-10-18T12:24:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.938+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f674c003a00 con 0x7f67541056e0 2023-10-18T12:24:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.939+0000 7f6751ffb700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67440a14d0 0x7f67440a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.939+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f674c0cc830 con 0x7f67541056e0 2023-10-18T12:24:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.940+0000 7f6758d9b700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67440a14d0 0x7f67440a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.940+0000 7f6758d9b700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67440a14d0 0x7f67440a3990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6748005950 tx=0x7f674800b500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:35.943+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f674c095b60 con 0x7f67541056e0 2023-10-18T12:24:36.104 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:24:35.676+0000 7f3301fe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:24:36.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:24:35.676+0000 7f3301fe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:24:36.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:24:35.676+0000 7f3301fe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:24:36.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:24:35.676+0000 7f3301fe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:24:36.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:24:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:24:35.676+0000 7f3301fe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:24:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.180+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f675410f5c0 con 0x7f67440a14d0 2023-10-18T12:24:36.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.185+0000 7f6751ffb700 1 -- 172.21.15.70:0/1259194176 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f675410f5c0 con 0x7f67440a14d0 2023-10-18T12:24:36.188 INFO:teuthology.orchestra.run.smithi070.stdout:/volumes/_nogroup/sv_0/93c11cb8-71cf-463f-825b-3c0bba1cc4f0 2023-10-18T12:24:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67440a14d0 msgr2=0x7f67440a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67440a14d0 0x7f67440a3990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6748005950 tx=0x7f674800b500 comp rx=0 tx=0).stop 2023-10-18T12:24:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67541056e0 msgr2=0x7f675419d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67541056e0 0x7f675419d750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f674c007f10 tx=0x7f674c00d3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 shutdown_connections 2023-10-18T12:24:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67541056e0 0x7f675419d750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67440a14d0 0x7f67440a3990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67540ff640 0x7f6754198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 --2- 172.21.15.70:0/1259194176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6754100000 0x7f67541993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 >> 172.21.15.70:0/1259194176 conn(0x7f67540767f0 msgr2=0x7f6754067890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 shutdown_connections 2023-10-18T12:24:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.187+0000 7f675afff700 1 -- 172.21.15.70:0/1259194176 wait complete. 2023-10-18T12:24:36.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-18T12:24:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 -- 172.21.15.70:0/108634422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8100720 msgr2=0x7f08f8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 --2- 172.21.15.70:0/108634422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8100720 0x7f08f8100ba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009990 tx=0x7f08e4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 -- 172.21.15.70:0/108634422 shutdown_connections 2023-10-18T12:24:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 --2- 172.21.15.70:0/108634422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f81010e0 0x7f08f810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 --2- 172.21.15.70:0/108634422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08f8106730 0x7f08f8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 --2- 172.21.15.70:0/108634422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8100720 0x7f08f8100ba0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 -- 172.21.15.70:0/108634422 >> 172.21.15.70:0/108634422 conn(0x7f08f80fc1b0 msgr2=0x7f08f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 -- 172.21.15.70:0/108634422 shutdown_connections 2023-10-18T12:24:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.408+0000 7f08fefff700 1 -- 172.21.15.70:0/108634422 wait complete. 2023-10-18T12:24:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.409+0000 7f08fefff700 1 Processor -- start 2023-10-18T12:24:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.409+0000 7f08fefff700 1 -- start start 2023-10-18T12:24:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fefff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f8100720 0x7f08f8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fefff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08f81010e0 0x7f08f8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fefff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 0x7f08f819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fefff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819dca0 con 0x7f08f8106730 2023-10-18T12:24:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fefff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819de10 con 0x7f08f8100720 2023-10-18T12:24:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fefff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819dfd0 con 0x7f08f81010e0 2023-10-18T12:24:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 0x7f08f819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 0x7f08f819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46354/0 (socket says 172.21.15.70:46354) 2023-10-18T12:24:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 -- 172.21.15.70:0/3932629481 learned_addr learned my addr 172.21.15.70:0/3932629481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fcd9b700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f8100720 0x7f08f8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08f7fff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08f81010e0 0x7f08f8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 -- 172.21.15.70:0/3932629481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08f81010e0 msgr2=0x7f08f8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08f81010e0 0x7f08f8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 -- 172.21.15.70:0/3932629481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f8100720 msgr2=0x7f08f8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f8100720 0x7f08f8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fd59c700 1 -- 172.21.15.70:0/3932629481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40095f0 con 0x7f08f8106730 2023-10-18T12:24:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.410+0000 7f08fcd9b700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f8100720 0x7f08f8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.411+0000 7f08fd59c700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 0x7f08f819d660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ef30 tx=0x7f08f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.411+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0018070 con 0x7f08f8106730 2023-10-18T12:24:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.411+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f819e250 con 0x7f08f8106730 2023-10-18T12:24:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.411+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f000f040 con 0x7f08f8106730 2023-10-18T12:24:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.411+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f81a5af0 con 0x7f08f8106730 2023-10-18T12:24:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.411+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000cd00 con 0x7f08f8106730 2023-10-18T12:24:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.412+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f8101e60 con 0x7f08f8106730 2023-10-18T12:24:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.414+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f08f0003a00 con 0x7f08f8106730 2023-10-18T12:24:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.415+0000 7f08f5ffb700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f08e80a14d0 0x7f08e80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.415+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f08f00cd590 con 0x7f08f8106730 2023-10-18T12:24:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.416+0000 7f08fcd9b700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f08e80a14d0 0x7f08e80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.416+0000 7f08fcd9b700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f08e80a14d0 0x7f08e80a3990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009fd0 tx=0x7f08ec009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.417+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08f0096950 con 0x7f08f8106730 2023-10-18T12:24:36.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:36 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2293848519' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-18T12:24:36.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:36 smithi186 ceph-mon[135253]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-18T12:24:36.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:36 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2293848519' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-18T12:24:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:36 smithi070 ceph-mon[98639]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-18T12:24:36.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.650+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f08f819a080 con 0x7f08e80a14d0 2023-10-18T12:24:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:36 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2293848519' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-18T12:24:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:36 smithi152 ceph-mon[134767]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-18T12:24:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.676+0000 7f08f5ffb700 1 -- 172.21.15.70:0/3932629481 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f08f819a080 con 0x7f08e80a14d0 2023-10-18T12:24:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f08e80a14d0 msgr2=0x7f08e80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f08e80a14d0 0x7f08e80a3990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009fd0 tx=0x7f08ec009260 comp rx=0 tx=0).stop 2023-10-18T12:24:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 msgr2=0x7f08f819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 0x7f08f819d660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ef30 tx=0x7f08f000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 shutdown_connections 2023-10-18T12:24:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08f8106730 0x7f08f819d660 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f08e80a14d0 0x7f08e80a3990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08f8100720 0x7f08f8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 --2- 172.21.15.70:0/3932629481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08f81010e0 0x7f08f8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 >> 172.21.15.70:0/3932629481 conn(0x7f08f80fc1b0 msgr2=0x7f08f80fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:36.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 shutdown_connections 2023-10-18T12:24:36.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.678+0000 7f08fefff700 1 -- 172.21.15.70:0/3932629481 wait complete. 2023-10-18T12:24:36.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-18T12:24:36.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 -- 172.21.15.70:0/4147968762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b0106730 msgr2=0x7fa4b0106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/4147968762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b0106730 0x7fa4b0106b10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009990 tx=0x7fa4a0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 -- 172.21.15.70:0/4147968762 shutdown_connections 2023-10-18T12:24:36.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/4147968762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4b01010e0 0x7fa4b010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/4147968762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b0106730 0x7fa4b0106b10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/4147968762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0100720 0x7fa4b0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 -- 172.21.15.70:0/4147968762 >> 172.21.15.70:0/4147968762 conn(0x7fa4b00fc1b0 msgr2=0x7fa4b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:36.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 -- 172.21.15.70:0/4147968762 shutdown_connections 2023-10-18T12:24:36.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.899+0000 7fa4b61ff700 1 -- 172.21.15.70:0/4147968762 wait complete. 2023-10-18T12:24:36.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.900+0000 7fa4b61ff700 1 Processor -- start 2023-10-18T12:24:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.900+0000 7fa4b61ff700 1 -- start start 2023-10-18T12:24:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4b61ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4b0100720 0x7fa4b0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4b61ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 0x7fa4b0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4b61ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0106730 0x7fa4b0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4b61ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0074ac0 con 0x7fa4b01010e0 2023-10-18T12:24:36.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4b61ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01a9550 con 0x7fa4b0100720 2023-10-18T12:24:36.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4b61ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01a9710 con 0x7fa4b0106730 2023-10-18T12:24:36.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 0x7fa4b0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4af7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4b0100720 0x7fa4b0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 0x7fa4b0073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46370/0 (socket says 172.21.15.70:46370) 2023-10-18T12:24:36.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 -- 172.21.15.70:0/3289651321 learned_addr learned my addr 172.21.15.70:0/3289651321 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:36.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4affff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0106730 0x7fa4b0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 -- 172.21.15.70:0/3289651321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0106730 msgr2=0x7fa4b0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0106730 0x7fa4b0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 -- 172.21.15.70:0/3289651321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4b0100720 msgr2=0x7fa4b0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:36.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4b0100720 0x7fa4b0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:36.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4a7fff700 1 -- 172.21.15.70:0/3289651321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a00095f0 con 0x7fa4b01010e0 2023-10-18T12:24:36.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.901+0000 7fa4affff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0106730 0x7fa4b0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:36.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.902+0000 7fa4a7fff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 0x7fa4b0073030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa49800e9a0 tx=0x7fa49800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:36.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.902+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498009790 con 0x7fa4b01010e0 2023-10-18T12:24:36.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.902+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49800cbd0 con 0x7fa4b01010e0 2023-10-18T12:24:36.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.902+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498018680 con 0x7fa4b01010e0 2023-10-18T12:24:36.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.902+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b01a99f0 con 0x7fa4b01010e0 2023-10-18T12:24:36.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.902+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b01a9f00 con 0x7fa4b01010e0 2023-10-18T12:24:36.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.903+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b0108d20 con 0x7fa4b01010e0 2023-10-18T12:24:36.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.905+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa49800cd40 con 0x7fa4b01010e0 2023-10-18T12:24:36.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.906+0000 7fa4ad7fa700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa49c0a1550 0x7fa49c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:36.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.907+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fa498014070 con 0x7fa4b01010e0 2023-10-18T12:24:36.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.907+0000 7fa4af7fe700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa49c0a1550 0x7fa49c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:36.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.907+0000 7fa4af7fe700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa49c0a1550 0x7fa49c0a3a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0006010 tx=0x7fa4a000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:36.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:36.908+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa4980968b0 con 0x7fa4b01010e0 2023-10-18T12:24:37.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.146+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4b0073fd0 con 0x7fa49c0a1550 2023-10-18T12:24:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.150+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/3289651321 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fa4b0073fd0 con 0x7fa49c0a1550 2023-10-18T12:24:37.153 INFO:teuthology.orchestra.run.smithi070.stdout:/volumes/_nogroup/sv_1/ceb5577f-b957-4f73-96f3-adc6f44d9343 2023-10-18T12:24:37.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa49c0a1550 msgr2=0x7fa49c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa49c0a1550 0x7fa49c0a3a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0006010 tx=0x7fa4a000b260 comp rx=0 tx=0).stop 2023-10-18T12:24:37.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 msgr2=0x7fa4b0073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 0x7fa4b0073030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa49800e9a0 tx=0x7fa49800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 shutdown_connections 2023-10-18T12:24:37.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b0106730 0x7fa4b0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa49c0a1550 0x7fa49c0a3a10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4b0100720 0x7fa4b0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 --2- 172.21.15.70:0/3289651321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b01010e0 0x7fa4b0073030 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 >> 172.21.15.70:0/3289651321 conn(0x7fa4b00fc1b0 msgr2=0x7fa4b00fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:37.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 shutdown_connections 2023-10-18T12:24:37.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.152+0000 7fa4b61ff700 1 -- 172.21.15.70:0/3289651321 wait complete. 2023-10-18T12:24:37.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:24:37.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 -- 172.21.15.70:0/3031967989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c072b90 msgr2=0x7fb42c072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 --2- 172.21.15.70:0/3031967989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c072b90 0x7fb42c072f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb428009990 tx=0x7fb428009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 -- 172.21.15.70:0/3031967989 shutdown_connections 2023-10-18T12:24:37.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 --2- 172.21.15.70:0/3031967989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c1116b0 0x7fb42c113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 --2- 172.21.15.70:0/3031967989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c072b90 0x7fb42c072f70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 --2- 172.21.15.70:0/3031967989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c0734b0 0x7fb42c111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 -- 172.21.15.70:0/3031967989 >> 172.21.15.70:0/3031967989 conn(0x7fb42c0fc170 msgr2=0x7fb42c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:37.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 -- 172.21.15.70:0/3031967989 shutdown_connections 2023-10-18T12:24:37.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.368+0000 7fb433dff700 1 -- 172.21.15.70:0/3031967989 wait complete. 2023-10-18T12:24:37.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.369+0000 7fb433dff700 1 Processor -- start 2023-10-18T12:24:37.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 -- start start 2023-10-18T12:24:37.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c072b90 0x7fb42c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c0734b0 0x7fb42c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 0x7fb42c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1a20c0 con 0x7fb42c1116b0 2023-10-18T12:24:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1a2230 con 0x7fb42c0734b0 2023-10-18T12:24:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb433dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1a23f0 con 0x7fb42c072b90 2023-10-18T12:24:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb43239c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 0x7fb42c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb43239c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 0x7fb42c1a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46392/0 (socket says 172.21.15.70:46392) 2023-10-18T12:24:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb43239c700 1 -- 172.21.15.70:0/3607214241 learned_addr learned my addr 172.21.15.70:0/3607214241 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb431b9b700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c072b90 0x7fb42c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.370+0000 7fb43139a700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c0734b0 0x7fb42c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb431b9b700 1 -- 172.21.15.70:0/3607214241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c0734b0 msgr2=0x7fb42c19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb431b9b700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c0734b0 0x7fb42c19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb431b9b700 1 -- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 msgr2=0x7fb42c1a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb431b9b700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 0x7fb42c1a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb431b9b700 1 -- 172.21.15.70:0/3607214241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280095f0 con 0x7fb42c072b90 2023-10-18T12:24:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb43239c700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 0x7fb42c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb431b9b700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c072b90 0x7fb42c19d1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb4280098e0 tx=0x7fb428005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42801d070 con 0x7fb42c072b90 2023-10-18T12:24:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c1a2670 con 0x7fb42c072b90 2023-10-18T12:24:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb428022470 con 0x7fb42c072b90 2023-10-18T12:24:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.371+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428017650 con 0x7fb42c072b90 2023-10-18T12:24:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.372+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c1a9f10 con 0x7fb42c072b90 2023-10-18T12:24:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.372+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c103b90 con 0x7fb42c072b90 2023-10-18T12:24:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.374+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb4280225e0 con 0x7fb42c072b90 2023-10-18T12:24:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.376+0000 7fb422ffd700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4180a14d0 0x7fb4180a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.376+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fb4280ce3e0 con 0x7fb42c072b90 2023-10-18T12:24:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.376+0000 7fb43139a700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4180a14d0 0x7fb4180a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.376+0000 7fb43139a700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4180a14d0 0x7fb4180a3990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb42c19e6f0 tx=0x7fb41c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.379+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb4280977a0 con 0x7fb42c072b90 2023-10-18T12:24:37.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:37 smithi186 ceph-mon[135253]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 426 B/s wr, 0 op/s 2023-10-18T12:24:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:37 smithi186 ceph-mon[135253]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:37 smithi186 ceph-mon[135253]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:37 smithi186 ceph-mon[135253]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:37 smithi186 ceph-mon[135253]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:37 smithi070 ceph-mon[98639]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 426 B/s wr, 0 op/s 2023-10-18T12:24:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:37 smithi070 ceph-mon[98639]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:37 smithi070 ceph-mon[98639]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:37 smithi070 ceph-mon[98639]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:37 smithi070 ceph-mon[98639]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.621+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb42c0742d0 con 0x7fb42c072b90 2023-10-18T12:24:37.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.623+0000 7fb422ffd700 1 -- 172.21.15.70:0/3607214241 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fb428027090 con 0x7fb42c072b90 2023-10-18T12:24:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:37.626 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:24:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4180a14d0 msgr2=0x7fb4180a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4180a14d0 0x7fb4180a3990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb42c19e6f0 tx=0x7fb41c009260 comp rx=0 tx=0).stop 2023-10-18T12:24:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c072b90 msgr2=0x7fb42c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c072b90 0x7fb42c19d1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb4280098e0 tx=0x7fb428005740 comp rx=0 tx=0).stop 2023-10-18T12:24:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 shutdown_connections 2023-10-18T12:24:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb42c1116b0 0x7fb42c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42c072b90 0x7fb42c19d1f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4180a14d0 0x7fb4180a3990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 --2- 172.21.15.70:0/3607214241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb42c0734b0 0x7fb42c19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 >> 172.21.15.70:0/3607214241 conn(0x7fb42c0fc170 msgr2=0x7fb42c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 shutdown_connections 2023-10-18T12:24:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.625+0000 7fb433dff700 1 -- 172.21.15.70:0/3607214241 wait complete. 2023-10-18T12:24:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:24:37.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:24:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:37 smithi152 ceph-mon[134767]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 426 B/s wr, 0 op/s 2023-10-18T12:24:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:37 smithi152 ceph-mon[134767]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:37 smithi152 ceph-mon[134767]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:37 smithi152 ceph-mon[134767]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:37 smithi152 ceph-mon[134767]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:24:37.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 -- 172.21.15.70:0/2968650835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58281006a0 msgr2=0x7f5828100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 --2- 172.21.15.70:0/2968650835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58281006a0 0x7f5828100b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5818009990 tx=0x7f5818009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 -- 172.21.15.70:0/2968650835 shutdown_connections 2023-10-18T12:24:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 --2- 172.21.15.70:0/2968650835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5828101060 0x7f582810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 --2- 172.21.15.70:0/2968650835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 0x7f5828106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 --2- 172.21.15.70:0/2968650835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58281006a0 0x7f5828100b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 -- 172.21.15.70:0/2968650835 >> 172.21.15.70:0/2968650835 conn(0x7f58280fc170 msgr2=0x7f58280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.842+0000 7f582f9ff700 1 -- 172.21.15.70:0/2968650835 shutdown_connections 2023-10-18T12:24:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.843+0000 7f582f9ff700 1 -- 172.21.15.70:0/2968650835 wait complete. 2023-10-18T12:24:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.843+0000 7f582f9ff700 1 Processor -- start 2023-10-18T12:24:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 -- start start 2023-10-18T12:24:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 0x7f582819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5828101060 0x7f582819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 0x7f5828198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828199290 con 0x7f58281006a0 2023-10-18T12:24:37.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828199400 con 0x7f58281066b0 2023-10-18T12:24:37.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281995c0 con 0x7f5828101060 2023-10-18T12:24:37.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582d79b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 0x7f582819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582d79b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 0x7f582819ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44498/0 (socket says 172.21.15.70:44498) 2023-10-18T12:24:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582d79b700 1 -- 172.21.15.70:0/16484032 learned_addr learned my addr 172.21.15.70:0/16484032 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582cf9a700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5828101060 0x7f582819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.844+0000 7f582df9c700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 0x7f5828198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582d79b700 1 -- 172.21.15.70:0/16484032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5828101060 msgr2=0x7f582819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582d79b700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5828101060 0x7f582819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582d79b700 1 -- 172.21.15.70:0/16484032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 msgr2=0x7f5828198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582d79b700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 0x7f5828198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582d79b700 1 -- 172.21.15.70:0/16484032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180095f0 con 0x7f58281006a0 2023-10-18T12:24:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582df9c700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 0x7f5828198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:24:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582cf9a700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5828101060 0x7f582819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:24:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582d79b700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 0x7f582819ea20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f582400e9a0 tx=0x7f582400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824009790 con 0x7f58281006a0 2023-10-18T12:24:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58281998a0 con 0x7f58281006a0 2023-10-18T12:24:37.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582400cbd0 con 0x7f58281006a0 2023-10-18T12:24:37.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824018680 con 0x7f58281006a0 2023-10-18T12:24:37.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.845+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58281a59a0 con 0x7f58281006a0 2023-10-18T12:24:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.846+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5828101de0 con 0x7f58281006a0 2023-10-18T12:24:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.849+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f582400cd40 con 0x7f58281006a0 2023-10-18T12:24:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.851+0000 7f581e7fc700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.851+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f5824014070 con 0x7f58281006a0 2023-10-18T12:24:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.851+0000 7f582cf9a700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.852+0000 7f582cf9a700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5818006010 tx=0x7f581800b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:37.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:37.853+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5824096b10 con 0x7f58281006a0 2023-10-18T12:24:38.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.055+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5828067530 con 0x7f58281006a0 2023-10-18T12:24:38.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.057+0000 7f581e7fc700 1 -- 172.21.15.70:0/16484032 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f582409a130 con 0x7f58281006a0 2023-10-18T12:24:38.060 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:38.060 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:38.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 msgr2=0x7f58140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5818006010 tx=0x7f581800b260 comp rx=0 tx=0).stop 2023-10-18T12:24:38.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 msgr2=0x7f582819ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 0x7f582819ea20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f582400e9a0 tx=0x7f582400ed60 comp rx=0 tx=0).stop 2023-10-18T12:24:38.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 shutdown_connections 2023-10-18T12:24:38.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58140a14d0 0x7f58140a3990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58281066b0 0x7f5828198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58281006a0 0x7f582819ea20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 --2- 172.21.15.70:0/16484032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5828101060 0x7f582819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.059+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 >> 172.21.15.70:0/16484032 conn(0x7f58280fc170 msgr2=0x7f58280fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:38.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.060+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 shutdown_connections 2023-10-18T12:24:38.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.060+0000 7f582f9ff700 1 -- 172.21.15.70:0/16484032 wait complete. 2023-10-18T12:24:38.082 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T12:24:38.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:24:38.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 -- 172.21.15.70:0/3643932622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8738101060 msgr2=0x7f873810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3643932622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8738101060 0x7f873810d570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f873000b0a0 tx=0x7f873000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 -- 172.21.15.70:0/3643932622 shutdown_connections 2023-10-18T12:24:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3643932622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8738101060 0x7f873810d570 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3643932622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87381066b0 0x7f8738106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3643932622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87381006a0 0x7f8738100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 -- 172.21.15.70:0/3643932622 >> 172.21.15.70:0/3643932622 conn(0x7f87380fc170 msgr2=0x7f87380fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.287+0000 7f873d3ff700 1 -- 172.21.15.70:0/3643932622 shutdown_connections 2023-10-18T12:24:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.288+0000 7f873d3ff700 1 -- 172.21.15.70:0/3643932622 wait complete. 2023-10-18T12:24:38.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.288+0000 7f873d3ff700 1 Processor -- start 2023-10-18T12:24:38.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 -- start start 2023-10-18T12:24:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87381006a0 0x7f8738198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 0x7f8738199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87381066b0 0x7f873819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873819dc40 con 0x7f87381006a0 2023-10-18T12:24:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873819ddb0 con 0x7f87381066b0 2023-10-18T12:24:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f873d3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873819df70 con 0x7f8738101060 2023-10-18T12:24:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87381006a0 0x7f8738198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f87367fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 0x7f8738199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f87367fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 0x7f8738199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46668/0 (socket says 172.21.15.70:46668) 2023-10-18T12:24:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f87367fc700 1 -- 172.21.15.70:0/3514327486 learned_addr learned my addr 172.21.15.70:0/3514327486 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.289+0000 7f87377fe700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87381066b0 0x7f873819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f87367fc700 1 -- 172.21.15.70:0/3514327486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87381066b0 msgr2=0x7f873819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f87367fc700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87381066b0 0x7f873819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f87367fc700 1 -- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87381006a0 msgr2=0x7f8738198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f87367fc700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87381006a0 0x7f8738198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f87367fc700 1 -- 172.21.15.70:0/3514327486 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009d00 con 0x7f8738101060 2023-10-18T12:24:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f87367fc700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 0x7f8738199300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f872c00e9a0 tx=0x7f872c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c009790 con 0x7f8738101060 2023-10-18T12:24:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873819e250 con 0x7f8738101060 2023-10-18T12:24:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872c010430 con 0x7f8738101060 2023-10-18T12:24:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c018680 con 0x7f8738101060 2023-10-18T12:24:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.290+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87381a5a90 con 0x7f8738101060 2023-10-18T12:24:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.291+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8738101de0 con 0x7f8738101060 2023-10-18T12:24:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.293+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f872c0105a0 con 0x7f8738101060 2023-10-18T12:24:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.294+0000 7f871ffff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87200a1560 0x7f87200a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.295+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f872c014070 con 0x7f8738101060 2023-10-18T12:24:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.295+0000 7f8736ffd700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87200a1560 0x7f87200a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.295+0000 7f8736ffd700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87200a1560 0x7f87200a3a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8728009be0 tx=0x7f8728009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.299+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f872c052100 con 0x7f8738101060 2023-10-18T12:24:38.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:38 smithi070 ceph-mon[98639]: pgmap v269: 129 pgs: 129 active+clean; 598 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2023-10-18T12:24:38.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3607214241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:38.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/16484032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:38.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.495+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f873819a0c0 con 0x7f87200a1560 2023-10-18T12:24:38.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.510+0000 7f871ffff700 1 -- 172.21.15.70:0/3514327486 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422699 (secure 0 0 0) 0x7f873819a0c0 con 0x7f87200a1560 2023-10-18T12:24:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.515+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87200a1560 msgr2=0x7f87200a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.515+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87200a1560 0x7f87200a3a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8728009be0 tx=0x7f8728009260 comp rx=0 tx=0).stop 2023-10-18T12:24:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 msgr2=0x7f8738199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 0x7f8738199300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f872c00e9a0 tx=0x7f872c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 shutdown_connections 2023-10-18T12:24:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87200a1560 0x7f87200a3a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87381066b0 0x7f873819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87381006a0 0x7f8738198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 --2- 172.21.15.70:0/3514327486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8738101060 0x7f8738199300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 >> 172.21.15.70:0/3514327486 conn(0x7f87380fc170 msgr2=0x7f873810b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 shutdown_connections 2023-10-18T12:24:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.516+0000 7f873d3ff700 1 -- 172.21.15.70:0/3514327486 wait complete. 2023-10-18T12:24:38.526 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:24:38.537 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2023-10-18T12:24:37.596789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612389,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":150,"num_read_kb":149,"num_write":236,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337680,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604144,"statfs":{"total":1151940427776,"available":1151594643456,"internally_reserved":0,"allocated":13742080,"data_stored":7680065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19611,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":30,"num_write":10,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003236"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145077+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:24:14.145077+0000","last_peered":"2023-10-18T12:24:14.145077+0000","last_clean":"2023-10-18T12:24:14.145077+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:24:14.145077+0000","last_undegraded":"2023-10-18T12:24:14.145077+0000","last_fullsized":"2023-10-18T12:24:14.145077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144990+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:24:14.144990+0000","last_peered":"2023-10-18T12:24:14.144990+0000","last_clean":"2023-10-18T12:24:14.144990+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:24:14.144990+0000","last_undegraded":"2023-10-18T12:24:14.144990+0000","last_fullsized":"2023-10-18T12:24:14.144990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146895+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:24:14.146895+0000","last_peered":"2023-10-18T12:24:14.146895+0000","last_clean":"2023-10-18T12:24:14.146895+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:24:14.146895+0000","last_undegraded":"2023-10-18T12:24:14.146895+0000","last_fullsized":"2023-10-18T12:24:14.146895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145041+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:24:14.145041+0000","last_peered":"2023-10-18T12:24:14.145041+0000","last_clean":"2023-10-18T12:24:14.145041+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:24:14.145041+0000","last_undegraded":"2023-10-18T12:24:14.145041+0000","last_fullsized":"2023-10-18T12:24:14.145041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139989+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:24:14.139989+0000","last_peered":"2023-10-18T12:24:14.139989+0000","last_clean":"2023-10-18T12:24:14.139989+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:24:14.139989+0000","last_undegraded":"2023-10-18T12:24:14.139989+0000","last_fullsized":"2023-10-18T12:24:14.139989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178381+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:24:14.178381+0000","last_peered":"2023-10-18T12:24:14.178381+0000","last_clean":"2023-10-18T12:24:14.178381+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:24:14.178381+0000","last_undegraded":"2023-10-18T12:24:14.178381+0000","last_fullsized":"2023-10-18T12:24:14.178381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146931+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:24:14.146931+0000","last_peered":"2023-10-18T12:24:14.146931+0000","last_clean":"2023-10-18T12:24:14.146931+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:24:14.146931+0000","last_undegraded":"2023-10-18T12:24:14.146931+0000","last_fullsized":"2023-10-18T12:24:14.146931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144154+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:24:14.144154+0000","last_peered":"2023-10-18T12:24:14.144154+0000","last_clean":"2023-10-18T12:24:14.144154+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:24:14.144154+0000","last_undegraded":"2023-10-18T12:24:14.144154+0000","last_fullsized":"2023-10-18T12:24:14.144154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140950+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:24:14.140950+0000","last_peered":"2023-10-18T12:24:14.140950+0000","last_clean":"2023-10-18T12:24:14.140950+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:24:14.140950+0000","last_undegraded":"2023-10-18T12:24:14.140950+0000","last_fullsized":"2023-10-18T12:24:14.140950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178575+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:24:14.178575+0000","last_peered":"2023-10-18T12:24:14.178575+0000","last_clean":"2023-10-18T12:24:14.178575+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:24:14.178575+0000","last_undegraded":"2023-10-18T12:24:14.178575+0000","last_fullsized":"2023-10-18T12:24:14.178575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145872+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:24:14.145872+0000","last_peered":"2023-10-18T12:24:14.145872+0000","last_clean":"2023-10-18T12:24:14.145872+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:24:14.145872+0000","last_undegraded":"2023-10-18T12:24:14.145872+0000","last_fullsized":"2023-10-18T12:24:14.145872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144948+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:24:14.144948+0000","last_peered":"2023-10-18T12:24:14.144948+0000","last_clean":"2023-10-18T12:24:14.144948+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:24:14.144948+0000","last_undegraded":"2023-10-18T12:24:14.144948+0000","last_fullsized":"2023-10-18T12:24:14.144948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138429+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:24:14.138429+0000","last_peered":"2023-10-18T12:24:14.138429+0000","last_clean":"2023-10-18T12:24:14.138429+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:24:14.138429+0000","last_undegraded":"2023-10-18T12:24:14.138429+0000","last_fullsized":"2023-10-18T12:24:14.138429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143503+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:24:14.143503+0000","last_peered":"2023-10-18T12:24:14.143503+0000","last_clean":"2023-10-18T12:24:14.143503+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:24:14.143503+0000","last_undegraded":"2023-10-18T12:24:14.143503+0000","last_fullsized":"2023-10-18T12:24:14.143503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143334+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:24:14.143334+0000","last_peered":"2023-10-18T12:24:14.143334+0000","last_clean":"2023-10-18T12:24:14.143334+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:24:14.143334+0000","last_undegraded":"2023-10-18T12:24:14.143334+0000","last_fullsized":"2023-10-18T12:24:14.143334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140092+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:24:14.140092+0000","last_peered":"2023-10-18T12:24:14.140092+0000","last_clean":"2023-10-18T12:24:14.140092+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:24:14.140092+0000","last_undegraded":"2023-10-18T12:24:14.140092+0000","last_fullsized":"2023-10-18T12:24:14.140092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145255+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:24:14.145255+0000","last_peered":"2023-10-18T12:24:14.145255+0000","last_clean":"2023-10-18T12:24:14.145255+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:24:14.145255+0000","last_undegraded":"2023-10-18T12:24:14.145255+0000","last_fullsized":"2023-10-18T12:24:14.145255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.163381+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:24:14.163381+0000","last_peered":"2023-10-18T12:24:14.163381+0000","last_clean":"2023-10-18T12:24:14.163381+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:24:14.163381+0000","last_undegraded":"2023-10-18T12:24:14.163381+0000","last_fullsized":"2023-10-18T12:24:14.163381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145652+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:24:14.145652+0000","last_peered":"2023-10-18T12:24:14.145652+0000","last_clean":"2023-10-18T12:24:14.145652+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:24:14.145652+0000","last_undegraded":"2023-10-18T12:24:14.145652+0000","last_fullsized":"2023-10-18T12:24:14.145652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350574+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:24:16.350574+0000","last_peered":"2023-10-18T12:24:16.350574+0000","last_clean":"2023-10-18T12:24:16.350574+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:24:16.350574+0000","last_undegraded":"2023-10-18T12:24:16.350574+0000","last_fullsized":"2023-10-18T12:24:16.350574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350162+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:24:16.350162+0000","last_peered":"2023-10-18T12:24:16.350162+0000","last_clean":"2023-10-18T12:24:16.350162+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:24:16.350162+0000","last_undegraded":"2023-10-18T12:24:16.350162+0000","last_fullsized":"2023-10-18T12:24:16.350162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144626+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:24:14.144626+0000","last_peered":"2023-10-18T12:24:14.144626+0000","last_clean":"2023-10-18T12:24:14.144626+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:24:14.144626+0000","last_undegraded":"2023-10-18T12:24:14.144626+0000","last_fullsized":"2023-10-18T12:24:14.144626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.143271+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:24:15.143271+0000","last_peered":"2023-10-18T12:24:15.143271+0000","last_clean":"2023-10-18T12:24:15.143271+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:24:15.143271+0000","last_undegraded":"2023-10-18T12:24:15.143271+0000","last_fullsized":"2023-10-18T12:24:15.143271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145925+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:24:14.145925+0000","last_peered":"2023-10-18T12:24:14.145925+0000","last_clean":"2023-10-18T12:24:14.145925+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:24:14.145925+0000","last_undegraded":"2023-10-18T12:24:14.145925+0000","last_fullsized":"2023-10-18T12:24:14.145925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145923+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:24:14.145923+0000","last_peered":"2023-10-18T12:24:14.145923+0000","last_clean":"2023-10-18T12:24:14.145923+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:24:14.145923+0000","last_undegraded":"2023-10-18T12:24:14.145923+0000","last_fullsized":"2023-10-18T12:24:14.145923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144332+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:24:15.144332+0000","last_peered":"2023-10-18T12:24:15.144332+0000","last_clean":"2023-10-18T12:24:15.144332+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:24:15.144332+0000","last_undegraded":"2023-10-18T12:24:15.144332+0000","last_fullsized":"2023-10-18T12:24:15.144332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140356+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:24:14.140356+0000","last_peered":"2023-10-18T12:24:14.140356+0000","last_clean":"2023-10-18T12:24:14.140356+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:24:14.140356+0000","last_undegraded":"2023-10-18T12:24:14.140356+0000","last_fullsized":"2023-10-18T12:24:14.140356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140523+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:24:14.140523+0000","last_peered":"2023-10-18T12:24:14.140523+0000","last_clean":"2023-10-18T12:24:14.140523+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:24:14.140523+0000","last_undegraded":"2023-10-18T12:24:14.140523+0000","last_fullsized":"2023-10-18T12:24:14.140523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.143443+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:24:15.143443+0000","last_peered":"2023-10-18T12:24:15.143443+0000","last_clean":"2023-10-18T12:24:15.143443+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:24:15.143443+0000","last_undegraded":"2023-10-18T12:24:15.143443+0000","last_fullsized":"2023-10-18T12:24:15.143443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140163+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:24:14.140163+0000","last_peered":"2023-10-18T12:24:14.140163+0000","last_clean":"2023-10-18T12:24:14.140163+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:24:14.140163+0000","last_undegraded":"2023-10-18T12:24:14.140163+0000","last_fullsized":"2023-10-18T12:24:14.140163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143520+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:24:14.143520+0000","last_peered":"2023-10-18T12:24:14.143520+0000","last_clean":"2023-10-18T12:24:14.143520+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:24:14.143520+0000","last_undegraded":"2023-10-18T12:24:14.143520+0000","last_fullsized":"2023-10-18T12:24:14.143520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349939+0000","last_change":"2023-10-18T12:24:14.152969+0000","last_active":"2023-10-18T12:24:16.349939+0000","last_peered":"2023-10-18T12:24:16.349939+0000","last_clean":"2023-10-18T12:24:16.349939+0000","last_became_active":"2023-10-18T12:24:14.152535+0000","last_became_peered":"2023-10-18T12:24:14.152535+0000","last_unstale":"2023-10-18T12:24:16.349939+0000","last_undegraded":"2023-10-18T12:24:16.349939+0000","last_fullsized":"2023-10-18T12:24:16.349939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:28.206795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146186+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:24:14.146186+0000","last_peered":"2023-10-18T12:24:14.146186+0000","last_clean":"2023-10-18T12:24:14.146186+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:24:14.146186+0000","last_undegraded":"2023-10-18T12:24:14.146186+0000","last_fullsized":"2023-10-18T12:24:14.146186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139918+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:24:14.139918+0000","last_peered":"2023-10-18T12:24:14.139918+0000","last_clean":"2023-10-18T12:24:14.139918+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:24:14.139918+0000","last_undegraded":"2023-10-18T12:24:14.139918+0000","last_fullsized":"2023-10-18T12:24:14.139918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349871+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:24:16.349871+0000","last_peered":"2023-10-18T12:24:16.349871+0000","last_clean":"2023-10-18T12:24:16.349871+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:24:16.349871+0000","last_undegraded":"2023-10-18T12:24:16.349871+0000","last_fullsized":"2023-10-18T12:24:16.349871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178262+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:24:14.178262+0000","last_peered":"2023-10-18T12:24:14.178262+0000","last_clean":"2023-10-18T12:24:14.178262+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:24:14.178262+0000","last_undegraded":"2023-10-18T12:24:14.178262+0000","last_fullsized":"2023-10-18T12:24:14.178262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146288+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:24:14.146288+0000","last_peered":"2023-10-18T12:24:14.146288+0000","last_clean":"2023-10-18T12:24:14.146288+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:24:14.146288+0000","last_undegraded":"2023-10-18T12:24:14.146288+0000","last_fullsized":"2023-10-18T12:24:14.146288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.141268+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:24:15.141268+0000","last_peered":"2023-10-18T12:24:15.141268+0000","last_clean":"2023-10-18T12:24:15.141268+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:24:15.141268+0000","last_undegraded":"2023-10-18T12:24:15.141268+0000","last_fullsized":"2023-10-18T12:24:15.141268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138214+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:24:14.138214+0000","last_peered":"2023-10-18T12:24:14.138214+0000","last_clean":"2023-10-18T12:24:14.138214+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:24:14.138214+0000","last_undegraded":"2023-10-18T12:24:14.138214+0000","last_fullsized":"2023-10-18T12:24:14.138214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143994+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:24:14.143994+0000","last_peered":"2023-10-18T12:24:14.143994+0000","last_clean":"2023-10-18T12:24:14.143994+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:24:14.143994+0000","last_undegraded":"2023-10-18T12:24:14.143994+0000","last_fullsized":"2023-10-18T12:24:14.143994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.467590+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:24:16.467590+0000","last_peered":"2023-10-18T12:24:16.467590+0000","last_clean":"2023-10-18T12:24:16.467590+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:24:16.467590+0000","last_undegraded":"2023-10-18T12:24:16.467590+0000","last_fullsized":"2023-10-18T12:24:16.467590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143686+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:24:14.143686+0000","last_peered":"2023-10-18T12:24:14.143686+0000","last_clean":"2023-10-18T12:24:14.143686+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:24:14.143686+0000","last_undegraded":"2023-10-18T12:24:14.143686+0000","last_fullsized":"2023-10-18T12:24:14.143686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.162058+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:24:14.162058+0000","last_peered":"2023-10-18T12:24:14.162058+0000","last_clean":"2023-10-18T12:24:14.162058+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:24:14.162058+0000","last_undegraded":"2023-10-18T12:24:14.162058+0000","last_fullsized":"2023-10-18T12:24:14.162058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.425610+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:24:15.425610+0000","last_peered":"2023-10-18T12:24:15.425610+0000","last_clean":"2023-10-18T12:24:15.425610+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:24:15.425610+0000","last_undegraded":"2023-10-18T12:24:15.425610+0000","last_fullsized":"2023-10-18T12:24:15.425610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178617+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:24:14.178617+0000","last_peered":"2023-10-18T12:24:14.178617+0000","last_clean":"2023-10-18T12:24:14.178617+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:24:14.178617+0000","last_undegraded":"2023-10-18T12:24:14.178617+0000","last_fullsized":"2023-10-18T12:24:14.178617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140765+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:24:14.140765+0000","last_peered":"2023-10-18T12:24:14.140765+0000","last_clean":"2023-10-18T12:24:14.140765+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:24:14.140765+0000","last_undegraded":"2023-10-18T12:24:14.140765+0000","last_fullsized":"2023-10-18T12:24:14.140765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349603+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:24:16.349603+0000","last_peered":"2023-10-18T12:24:16.349603+0000","last_clean":"2023-10-18T12:24:16.349603+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:24:16.349603+0000","last_undegraded":"2023-10-18T12:24:16.349603+0000","last_fullsized":"2023-10-18T12:24:16.349603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145654+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:24:14.145654+0000","last_peered":"2023-10-18T12:24:14.145654+0000","last_clean":"2023-10-18T12:24:14.145654+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:24:14.145654+0000","last_undegraded":"2023-10-18T12:24:14.145654+0000","last_fullsized":"2023-10-18T12:24:14.145654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145994+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:24:14.145994+0000","last_peered":"2023-10-18T12:24:14.145994+0000","last_clean":"2023-10-18T12:24:14.145994+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:24:14.145994+0000","last_undegraded":"2023-10-18T12:24:14.145994+0000","last_fullsized":"2023-10-18T12:24:14.145994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144712+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:24:15.144712+0000","last_peered":"2023-10-18T12:24:15.144712+0000","last_clean":"2023-10-18T12:24:15.144712+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:24:15.144712+0000","last_undegraded":"2023-10-18T12:24:15.144712+0000","last_fullsized":"2023-10-18T12:24:15.144712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143428+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:24:14.143428+0000","last_peered":"2023-10-18T12:24:14.143428+0000","last_clean":"2023-10-18T12:24:14.143428+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:24:14.143428+0000","last_undegraded":"2023-10-18T12:24:14.143428+0000","last_fullsized":"2023-10-18T12:24:14.143428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140698+0000","last_change":"2023-10-18T12:24:09.974796+0000","last_active":"2023-10-18T12:24:14.140698+0000","last_peered":"2023-10-18T12:24:14.140698+0000","last_clean":"2023-10-18T12:24:14.140698+0000","last_became_active":"2023-10-18T12:24:09.974498+0000","last_became_peered":"2023-10-18T12:24:09.974498+0000","last_unstale":"2023-10-18T12:24:14.140698+0000","last_undegraded":"2023-10-18T12:24:14.140698+0000","last_fullsized":"2023-10-18T12:24:14.140698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:11:54.782709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349745+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:24:16.349745+0000","last_peered":"2023-10-18T12:24:16.349745+0000","last_clean":"2023-10-18T12:24:16.349745+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:24:16.349745+0000","last_undegraded":"2023-10-18T12:24:16.349745+0000","last_fullsized":"2023-10-18T12:24:16.349745+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'16","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:36.135028+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:24:36.135028+0000","last_peered":"2023-10-18T12:24:36.135028+0000","last_clean":"2023-10-18T12:24:36.135028+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:24:36.135028+0000","last_undegraded":"2023-10-18T12:24:36.135028+0000","last_fullsized":"2023-10-18T12:24:36.135028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21223,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":25,"num_write":16,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140163+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:24:14.140163+0000","last_peered":"2023-10-18T12:24:14.140163+0000","last_clean":"2023-10-18T12:24:14.140163+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:24:14.140163+0000","last_undegraded":"2023-10-18T12:24:14.140163+0000","last_fullsized":"2023-10-18T12:24:14.140163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350870+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:24:16.350870+0000","last_peered":"2023-10-18T12:24:16.350870+0000","last_clean":"2023-10-18T12:24:16.350870+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:24:16.350870+0000","last_undegraded":"2023-10-18T12:24:16.350870+0000","last_fullsized":"2023-10-18T12:24:16.350870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144571+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:24:14.144571+0000","last_peered":"2023-10-18T12:24:14.144571+0000","last_clean":"2023-10-18T12:24:14.144571+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:24:14.144571+0000","last_undegraded":"2023-10-18T12:24:14.144571+0000","last_fullsized":"2023-10-18T12:24:14.144571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139188+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:24:14.139188+0000","last_peered":"2023-10-18T12:24:14.139188+0000","last_clean":"2023-10-18T12:24:14.139188+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:24:14.139188+0000","last_undegraded":"2023-10-18T12:24:14.139188+0000","last_fullsized":"2023-10-18T12:24:14.139188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.141167+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:24:15.141167+0000","last_peered":"2023-10-18T12:24:15.141167+0000","last_clean":"2023-10-18T12:24:15.141167+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:24:15.141167+0000","last_undegraded":"2023-10-18T12:24:15.141167+0000","last_fullsized":"2023-10-18T12:24:15.141167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.148373+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:24:14.148373+0000","last_peered":"2023-10-18T12:24:14.148373+0000","last_clean":"2023-10-18T12:24:14.148373+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:24:14.148373+0000","last_undegraded":"2023-10-18T12:24:14.148373+0000","last_fullsized":"2023-10-18T12:24:14.148373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144711+0000","last_change":"2023-10-18T12:24:09.968891+0000","last_active":"2023-10-18T12:24:14.144711+0000","last_peered":"2023-10-18T12:24:14.144711+0000","last_clean":"2023-10-18T12:24:14.144711+0000","last_became_active":"2023-10-18T12:24:09.968566+0000","last_became_peered":"2023-10-18T12:24:09.968566+0000","last_unstale":"2023-10-18T12:24:14.144711+0000","last_undegraded":"2023-10-18T12:24:14.144711+0000","last_fullsized":"2023-10-18T12:24:14.144711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:54.830870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350000+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:24:16.350000+0000","last_peered":"2023-10-18T12:24:16.350000+0000","last_clean":"2023-10-18T12:24:16.350000+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:24:16.350000+0000","last_undegraded":"2023-10-18T12:24:16.350000+0000","last_fullsized":"2023-10-18T12:24:16.350000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143710+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:24:14.143710+0000","last_peered":"2023-10-18T12:24:14.143710+0000","last_clean":"2023-10-18T12:24:14.143710+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:24:14.143710+0000","last_undegraded":"2023-10-18T12:24:14.143710+0000","last_fullsized":"2023-10-18T12:24:14.143710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144141+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:24:14.144141+0000","last_peered":"2023-10-18T12:24:14.144141+0000","last_clean":"2023-10-18T12:24:14.144141+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:24:14.144141+0000","last_undegraded":"2023-10-18T12:24:14.144141+0000","last_fullsized":"2023-10-18T12:24:14.144141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.140919+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:24:15.140919+0000","last_peered":"2023-10-18T12:24:15.140919+0000","last_clean":"2023-10-18T12:24:15.140919+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:24:15.140919+0000","last_undegraded":"2023-10-18T12:24:15.140919+0000","last_fullsized":"2023-10-18T12:24:15.140919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139531+0000","last_change":"2023-10-18T12:24:08.977239+0000","last_active":"2023-10-18T12:24:14.139531+0000","last_peered":"2023-10-18T12:24:14.139531+0000","last_clean":"2023-10-18T12:24:14.139531+0000","last_became_active":"2023-10-18T12:24:08.976918+0000","last_became_peered":"2023-10-18T12:24:08.976918+0000","last_unstale":"2023-10-18T12:24:14.139531+0000","last_undegraded":"2023-10-18T12:24:14.139531+0000","last_fullsized":"2023-10-18T12:24:14.139531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:30.752782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144893+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:24:14.144893+0000","last_peered":"2023-10-18T12:24:14.144893+0000","last_clean":"2023-10-18T12:24:14.144893+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:24:14.144893+0000","last_undegraded":"2023-10-18T12:24:14.144893+0000","last_fullsized":"2023-10-18T12:24:14.144893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349566+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:24:16.349566+0000","last_peered":"2023-10-18T12:24:16.349566+0000","last_clean":"2023-10-18T12:24:16.349566+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:24:16.349566+0000","last_undegraded":"2023-10-18T12:24:16.349566+0000","last_fullsized":"2023-10-18T12:24:16.349566+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.085302+0000","last_change":"2023-10-18T12:24:08.966712+0000","last_active":"2023-10-18T12:24:16.085302+0000","last_peered":"2023-10-18T12:24:16.085302+0000","last_clean":"2023-10-18T12:24:16.085302+0000","last_became_active":"2023-10-18T12:24:08.966392+0000","last_became_peered":"2023-10-18T12:24:08.966392+0000","last_unstale":"2023-10-18T12:24:16.085302+0000","last_undegraded":"2023-10-18T12:24:16.085302+0000","last_fullsized":"2023-10-18T12:24:16.085302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:44:14.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145237+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:24:14.145237+0000","last_peered":"2023-10-18T12:24:14.145237+0000","last_clean":"2023-10-18T12:24:14.145237+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:24:14.145237+0000","last_undegraded":"2023-10-18T12:24:14.145237+0000","last_fullsized":"2023-10-18T12:24:14.145237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.084865+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:24:16.084865+0000","last_peered":"2023-10-18T12:24:16.084865+0000","last_clean":"2023-10-18T12:24:16.084865+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:24:16.084865+0000","last_undegraded":"2023-10-18T12:24:16.084865+0000","last_fullsized":"2023-10-18T12:24:16.084865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.162569+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:24:14.162569+0000","last_peered":"2023-10-18T12:24:14.162569+0000","last_clean":"2023-10-18T12:24:14.162569+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:24:14.162569+0000","last_undegraded":"2023-10-18T12:24:14.162569+0000","last_fullsized":"2023-10-18T12:24:14.162569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178757+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:24:14.178757+0000","last_peered":"2023-10-18T12:24:14.178757+0000","last_clean":"2023-10-18T12:24:14.178757+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:24:14.178757+0000","last_undegraded":"2023-10-18T12:24:14.178757+0000","last_fullsized":"2023-10-18T12:24:14.178757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.140694+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:24:15.140694+0000","last_peered":"2023-10-18T12:24:15.140694+0000","last_clean":"2023-10-18T12:24:15.140694+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:24:15.140694+0000","last_undegraded":"2023-10-18T12:24:15.140694+0000","last_fullsized":"2023-10-18T12:24:15.140694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145112+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:24:14.145112+0000","last_peered":"2023-10-18T12:24:14.145112+0000","last_clean":"2023-10-18T12:24:14.145112+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:24:14.145112+0000","last_undegraded":"2023-10-18T12:24:14.145112+0000","last_fullsized":"2023-10-18T12:24:14.145112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178544+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:24:14.178544+0000","last_peered":"2023-10-18T12:24:14.178544+0000","last_clean":"2023-10-18T12:24:14.178544+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:24:14.178544+0000","last_undegraded":"2023-10-18T12:24:14.178544+0000","last_fullsized":"2023-10-18T12:24:14.178544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.140703+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:24:15.140703+0000","last_peered":"2023-10-18T12:24:15.140703+0000","last_clean":"2023-10-18T12:24:15.140703+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:24:15.140703+0000","last_undegraded":"2023-10-18T12:24:15.140703+0000","last_fullsized":"2023-10-18T12:24:15.140703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143868+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:24:14.143868+0000","last_peered":"2023-10-18T12:24:14.143868+0000","last_clean":"2023-10-18T12:24:14.143868+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:24:14.143868+0000","last_undegraded":"2023-10-18T12:24:14.143868+0000","last_fullsized":"2023-10-18T12:24:14.143868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138126+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:24:14.138126+0000","last_peered":"2023-10-18T12:24:14.138126+0000","last_clean":"2023-10-18T12:24:14.138126+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:24:14.138126+0000","last_undegraded":"2023-10-18T12:24:14.138126+0000","last_fullsized":"2023-10-18T12:24:14.138126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.469368+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:24:15.469368+0000","last_peered":"2023-10-18T12:24:15.469368+0000","last_clean":"2023-10-18T12:24:15.469368+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:24:15.469368+0000","last_undegraded":"2023-10-18T12:24:15.469368+0000","last_fullsized":"2023-10-18T12:24:15.469368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145960+0000","last_change":"2023-10-18T12:24:09.985663+0000","last_active":"2023-10-18T12:24:14.145960+0000","last_peered":"2023-10-18T12:24:14.145960+0000","last_clean":"2023-10-18T12:24:14.145960+0000","last_became_active":"2023-10-18T12:24:09.983659+0000","last_became_peered":"2023-10-18T12:24:09.983659+0000","last_unstale":"2023-10-18T12:24:14.145960+0000","last_undegraded":"2023-10-18T12:24:14.145960+0000","last_fullsized":"2023-10-18T12:24:14.145960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:39.706073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144355+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:24:14.144355+0000","last_peered":"2023-10-18T12:24:14.144355+0000","last_clean":"2023-10-18T12:24:14.144355+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:24:14.144355+0000","last_undegraded":"2023-10-18T12:24:14.144355+0000","last_fullsized":"2023-10-18T12:24:14.144355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":324,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143960+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:24:14.143960+0000","last_peered":"2023-10-18T12:24:14.143960+0000","last_clean":"2023-10-18T12:24:14.143960+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:24:14.143960+0000","last_undegraded":"2023-10-18T12:24:14.143960+0000","last_fullsized":"2023-10-18T12:24:14.143960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350522+0000","last_change":"2023-10-18T12:24:14.179768+0000","last_active":"2023-10-18T12:24:16.350522+0000","last_peered":"2023-10-18T12:24:16.350522+0000","last_clean":"2023-10-18T12:24:16.350522+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:24:16.350522+0000","last_undegraded":"2023-10-18T12:24:16.350522+0000","last_fullsized":"2023-10-18T12:24:16.350522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:17.695525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138189+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:24:14.138189+0000","last_peered":"2023-10-18T12:24:14.138189+0000","last_clean":"2023-10-18T12:24:14.138189+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:24:14.138189+0000","last_undegraded":"2023-10-18T12:24:14.138189+0000","last_fullsized":"2023-10-18T12:24:14.138189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144049+0000","last_change":"2023-10-18T12:24:08.964976+0000","last_active":"2023-10-18T12:24:14.144049+0000","last_peered":"2023-10-18T12:24:14.144049+0000","last_clean":"2023-10-18T12:24:14.144049+0000","last_became_active":"2023-10-18T12:24:08.963055+0000","last_became_peered":"2023-10-18T12:24:08.963055+0000","last_unstale":"2023-10-18T12:24:14.144049+0000","last_undegraded":"2023-10-18T12:24:14.144049+0000","last_fullsized":"2023-10-18T12:24:14.144049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:05.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.141032+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:24:15.141032+0000","last_peered":"2023-10-18T12:24:15.141032+0000","last_clean":"2023-10-18T12:24:15.141032+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:24:15.141032+0000","last_undegraded":"2023-10-18T12:24:15.141032+0000","last_fullsized":"2023-10-18T12:24:15.141032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140232+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:24:14.140232+0000","last_peered":"2023-10-18T12:24:14.140232+0000","last_clean":"2023-10-18T12:24:14.140232+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:24:14.140232+0000","last_undegraded":"2023-10-18T12:24:14.140232+0000","last_fullsized":"2023-10-18T12:24:14.140232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138956+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:24:14.138956+0000","last_peered":"2023-10-18T12:24:14.138956+0000","last_clean":"2023-10-18T12:24:14.138956+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:24:14.138956+0000","last_undegraded":"2023-10-18T12:24:14.138956+0000","last_fullsized":"2023-10-18T12:24:14.138956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.425448+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:24:15.425448+0000","last_peered":"2023-10-18T12:24:15.425448+0000","last_clean":"2023-10-18T12:24:15.425448+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:24:15.425448+0000","last_undegraded":"2023-10-18T12:24:15.425448+0000","last_fullsized":"2023-10-18T12:24:15.425448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.163474+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:24:14.163474+0000","last_peered":"2023-10-18T12:24:14.163474+0000","last_clean":"2023-10-18T12:24:14.163474+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:24:14.163474+0000","last_undegraded":"2023-10-18T12:24:14.163474+0000","last_fullsized":"2023-10-18T12:24:14.163474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144729+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:24:14.144729+0000","last_peered":"2023-10-18T12:24:14.144729+0000","last_clean":"2023-10-18T12:24:14.144729+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:24:14.144729+0000","last_undegraded":"2023-10-18T12:24:14.144729+0000","last_fullsized":"2023-10-18T12:24:14.144729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349881+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:24:16.349881+0000","last_peered":"2023-10-18T12:24:16.349881+0000","last_clean":"2023-10-18T12:24:16.349881+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:24:16.349881+0000","last_undegraded":"2023-10-18T12:24:16.349881+0000","last_fullsized":"2023-10-18T12:24:16.349881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143601+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:24:14.143601+0000","last_peered":"2023-10-18T12:24:14.143601+0000","last_clean":"2023-10-18T12:24:14.143601+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:24:14.143601+0000","last_undegraded":"2023-10-18T12:24:14.143601+0000","last_fullsized":"2023-10-18T12:24:14.143601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145074+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:24:14.145074+0000","last_peered":"2023-10-18T12:24:14.145074+0000","last_clean":"2023-10-18T12:24:14.145074+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:24:14.145074+0000","last_undegraded":"2023-10-18T12:24:14.145074+0000","last_fullsized":"2023-10-18T12:24:14.145074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144663+0000","last_change":"2023-10-18T12:24:14.179741+0000","last_active":"2023-10-18T12:24:15.144663+0000","last_peered":"2023-10-18T12:24:15.144663+0000","last_clean":"2023-10-18T12:24:15.144663+0000","last_became_active":"2023-10-18T12:24:14.179297+0000","last_became_peered":"2023-10-18T12:24:14.179297+0000","last_unstale":"2023-10-18T12:24:15.144663+0000","last_undegraded":"2023-10-18T12:24:15.144663+0000","last_fullsized":"2023-10-18T12:24:15.144663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:52.997230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140138+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:24:14.140138+0000","last_peered":"2023-10-18T12:24:14.140138+0000","last_clean":"2023-10-18T12:24:14.140138+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:24:14.140138+0000","last_undegraded":"2023-10-18T12:24:14.140138+0000","last_fullsized":"2023-10-18T12:24:14.140138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139570+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:24:14.139570+0000","last_peered":"2023-10-18T12:24:14.139570+0000","last_clean":"2023-10-18T12:24:14.139570+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:24:14.139570+0000","last_undegraded":"2023-10-18T12:24:14.139570+0000","last_fullsized":"2023-10-18T12:24:14.139570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350131+0000","last_change":"2023-10-18T12:24:14.168625+0000","last_active":"2023-10-18T12:24:16.350131+0000","last_peered":"2023-10-18T12:24:16.350131+0000","last_clean":"2023-10-18T12:24:16.350131+0000","last_became_active":"2023-10-18T12:24:14.168203+0000","last_became_peered":"2023-10-18T12:24:14.168203+0000","last_unstale":"2023-10-18T12:24:16.350131+0000","last_undegraded":"2023-10-18T12:24:16.350131+0000","last_fullsized":"2023-10-18T12:24:16.350131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:21.516180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143008+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:24:14.143008+0000","last_peered":"2023-10-18T12:24:14.143008+0000","last_clean":"2023-10-18T12:24:14.143008+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:24:14.143008+0000","last_undegraded":"2023-10-18T12:24:14.143008+0000","last_fullsized":"2023-10-18T12:24:14.143008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144036+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:24:14.144036+0000","last_peered":"2023-10-18T12:24:14.144036+0000","last_clean":"2023-10-18T12:24:14.144036+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:24:14.144036+0000","last_undegraded":"2023-10-18T12:24:14.144036+0000","last_fullsized":"2023-10-18T12:24:14.144036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144476+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:24:15.144476+0000","last_peered":"2023-10-18T12:24:15.144476+0000","last_clean":"2023-10-18T12:24:15.144476+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:24:15.144476+0000","last_undegraded":"2023-10-18T12:24:15.144476+0000","last_fullsized":"2023-10-18T12:24:15.144476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144540+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:24:14.144540+0000","last_peered":"2023-10-18T12:24:14.144540+0000","last_clean":"2023-10-18T12:24:14.144540+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:24:14.144540+0000","last_undegraded":"2023-10-18T12:24:14.144540+0000","last_fullsized":"2023-10-18T12:24:14.144540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144483+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:24:14.144483+0000","last_peered":"2023-10-18T12:24:14.144483+0000","last_clean":"2023-10-18T12:24:14.144483+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:24:14.144483+0000","last_undegraded":"2023-10-18T12:24:14.144483+0000","last_fullsized":"2023-10-18T12:24:14.144483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144609+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:24:15.144609+0000","last_peered":"2023-10-18T12:24:15.144609+0000","last_clean":"2023-10-18T12:24:15.144609+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:24:15.144609+0000","last_undegraded":"2023-10-18T12:24:15.144609+0000","last_fullsized":"2023-10-18T12:24:15.144609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.084944+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:24:16.084944+0000","last_peered":"2023-10-18T12:24:16.084944+0000","last_clean":"2023-10-18T12:24:16.084944+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:24:16.084944+0000","last_undegraded":"2023-10-18T12:24:16.084944+0000","last_fullsized":"2023-10-18T12:24:16.084944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145280+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:24:14.145280+0000","last_peered":"2023-10-18T12:24:14.145280+0000","last_clean":"2023-10-18T12:24:14.145280+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:24:14.145280+0000","last_undegraded":"2023-10-18T12:24:14.145280+0000","last_fullsized":"2023-10-18T12:24:14.145280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.145439+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:24:15.145439+0000","last_peered":"2023-10-18T12:24:15.145439+0000","last_clean":"2023-10-18T12:24:15.145439+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:24:15.145439+0000","last_undegraded":"2023-10-18T12:24:15.145439+0000","last_fullsized":"2023-10-18T12:24:15.145439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145394+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:24:14.145394+0000","last_peered":"2023-10-18T12:24:14.145394+0000","last_clean":"2023-10-18T12:24:14.145394+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:24:14.145394+0000","last_undegraded":"2023-10-18T12:24:14.145394+0000","last_fullsized":"2023-10-18T12:24:14.145394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145984+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:24:14.145984+0000","last_peered":"2023-10-18T12:24:14.145984+0000","last_clean":"2023-10-18T12:24:14.145984+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:24:14.145984+0000","last_undegraded":"2023-10-18T12:24:14.145984+0000","last_fullsized":"2023-10-18T12:24:14.145984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.467096+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:24:16.467096+0000","last_peered":"2023-10-18T12:24:16.467096+0000","last_clean":"2023-10-18T12:24:16.467096+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:24:16.467096+0000","last_undegraded":"2023-10-18T12:24:16.467096+0000","last_fullsized":"2023-10-18T12:24:16.467096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350670+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:24:16.350670+0000","last_peered":"2023-10-18T12:24:16.350670+0000","last_clean":"2023-10-18T12:24:16.350670+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:24:16.350670+0000","last_undegraded":"2023-10-18T12:24:16.350670+0000","last_fullsized":"2023-10-18T12:24:16.350670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144824+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:24:14.144824+0000","last_peered":"2023-10-18T12:24:14.144824+0000","last_clean":"2023-10-18T12:24:14.144824+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:24:14.144824+0000","last_undegraded":"2023-10-18T12:24:14.144824+0000","last_fullsized":"2023-10-18T12:24:14.144824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349374+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:24:16.349374+0000","last_peered":"2023-10-18T12:24:16.349374+0000","last_clean":"2023-10-18T12:24:16.349374+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:24:16.349374+0000","last_undegraded":"2023-10-18T12:24:16.349374+0000","last_fullsized":"2023-10-18T12:24:16.349374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139059+0000","last_change":"2023-10-18T12:24:08.975858+0000","last_active":"2023-10-18T12:24:14.139059+0000","last_peered":"2023-10-18T12:24:14.139059+0000","last_clean":"2023-10-18T12:24:14.139059+0000","last_became_active":"2023-10-18T12:24:08.974568+0000","last_became_peered":"2023-10-18T12:24:08.974568+0000","last_unstale":"2023-10-18T12:24:14.139059+0000","last_undegraded":"2023-10-18T12:24:14.139059+0000","last_fullsized":"2023-10-18T12:24:14.139059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:42.951310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145872+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:24:14.145872+0000","last_peered":"2023-10-18T12:24:14.145872+0000","last_clean":"2023-10-18T12:24:14.145872+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:24:14.145872+0000","last_undegraded":"2023-10-18T12:24:14.145872+0000","last_fullsized":"2023-10-18T12:24:14.145872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144132+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:24:14.144132+0000","last_peered":"2023-10-18T12:24:14.144132+0000","last_clean":"2023-10-18T12:24:14.144132+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:24:14.144132+0000","last_undegraded":"2023-10-18T12:24:14.144132+0000","last_fullsized":"2023-10-18T12:24:14.144132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144196+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:24:14.144196+0000","last_peered":"2023-10-18T12:24:14.144196+0000","last_clean":"2023-10-18T12:24:14.144196+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:24:14.144196+0000","last_undegraded":"2023-10-18T12:24:14.144196+0000","last_fullsized":"2023-10-18T12:24:14.144196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144453+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:24:14.144453+0000","last_peered":"2023-10-18T12:24:14.144453+0000","last_clean":"2023-10-18T12:24:14.144453+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:24:14.144453+0000","last_undegraded":"2023-10-18T12:24:14.144453+0000","last_fullsized":"2023-10-18T12:24:14.144453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145771+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:24:14.145771+0000","last_peered":"2023-10-18T12:24:14.145771+0000","last_clean":"2023-10-18T12:24:14.145771+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:24:14.145771+0000","last_undegraded":"2023-10-18T12:24:14.145771+0000","last_fullsized":"2023-10-18T12:24:14.145771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138902+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:24:14.138902+0000","last_peered":"2023-10-18T12:24:14.138902+0000","last_clean":"2023-10-18T12:24:14.138902+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:24:14.138902+0000","last_undegraded":"2023-10-18T12:24:14.138902+0000","last_fullsized":"2023-10-18T12:24:14.138902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145071+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:24:14.145071+0000","last_peered":"2023-10-18T12:24:14.145071+0000","last_clean":"2023-10-18T12:24:14.145071+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:24:14.145071+0000","last_undegraded":"2023-10-18T12:24:14.145071+0000","last_fullsized":"2023-10-18T12:24:14.145071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144958+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:24:14.144958+0000","last_peered":"2023-10-18T12:24:14.144958+0000","last_clean":"2023-10-18T12:24:14.144958+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:24:14.144958+0000","last_undegraded":"2023-10-18T12:24:14.144958+0000","last_fullsized":"2023-10-18T12:24:14.144958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144812+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:24:14.144812+0000","last_peered":"2023-10-18T12:24:14.144812+0000","last_clean":"2023-10-18T12:24:14.144812+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:24:14.144812+0000","last_undegraded":"2023-10-18T12:24:14.144812+0000","last_fullsized":"2023-10-18T12:24:14.144812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143781+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:24:14.143781+0000","last_peered":"2023-10-18T12:24:14.143781+0000","last_clean":"2023-10-18T12:24:14.143781+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:24:14.143781+0000","last_undegraded":"2023-10-18T12:24:14.143781+0000","last_fullsized":"2023-10-18T12:24:14.143781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.142915+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:24:14.142915+0000","last_peered":"2023-10-18T12:24:14.142915+0000","last_clean":"2023-10-18T12:24:14.142915+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:24:14.142915+0000","last_undegraded":"2023-10-18T12:24:14.142915+0000","last_fullsized":"2023-10-18T12:24:14.142915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140337+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:24:14.140337+0000","last_peered":"2023-10-18T12:24:14.140337+0000","last_clean":"2023-10-18T12:24:14.140337+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:24:14.140337+0000","last_undegraded":"2023-10-18T12:24:14.140337+0000","last_fullsized":"2023-10-18T12:24:14.140337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138476+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:24:14.138476+0000","last_peered":"2023-10-18T12:24:14.138476+0000","last_clean":"2023-10-18T12:24:14.138476+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:24:14.138476+0000","last_undegraded":"2023-10-18T12:24:14.138476+0000","last_fullsized":"2023-10-18T12:24:14.138476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144772+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:24:14.144772+0000","last_peered":"2023-10-18T12:24:14.144772+0000","last_clean":"2023-10-18T12:24:14.144772+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:24:14.144772+0000","last_undegraded":"2023-10-18T12:24:14.144772+0000","last_fullsized":"2023-10-18T12:24:14.144772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22021,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":14,"num_read_kb":30,"num_write":55,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":31301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":489514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1,"5min":1,"15min":1},"last":0.78600000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.873},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.92500000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.89400000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1101299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.443}]},{"osd":2,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.497}]},{"osd":3,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66200000000000003}]},{"osd":9,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73199999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.70699999999999996}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":490096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.64700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.77600000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70699999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.745},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.75600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":490118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.751},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.44800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.436,"15min":0.435},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.54800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47299999999999998,"15min":0.47799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.54600000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.85099999999999998,"15min":0.879},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.4399999999999999,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.629},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.76600000000000001,"15min":0.80300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69399999999999995,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.51300000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71999999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":1.117,"15min":1.117},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66300000000000003,"5min":1.099,"15min":1.099},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.325,"15min":1.325},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91200000000000003,"5min":1.272,"15min":1.272},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84999999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:23:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.80600000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1585152,"data_stored":1080464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.41999999999999998},"min":{"1min":0.314,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.42099999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.62}]},{"osd":3,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85899999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52000000000000002,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.47599999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.55200000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.70999999999999996},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.56299999999999994,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72699999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:23:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462949,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":489514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.502,"15min":0.502},"min":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.4470000000000001,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58099999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.878}]},{"osd":3,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85499999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.62}]},{"osd":4,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.746,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84099999999999997,"5min":1.028,"15min":1.028},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99299999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":1.234,"15min":1.234},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75700000000000001,"15min":0.76000000000000001},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.351,"15min":1.351},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91500000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.80300000000000005,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.80000000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:23:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76100000000000001}]},{"osd":10,"last update":"Wed Oct 18 12:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.371,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.439},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.439,"15min":0.41899999999999998},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41899999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:24:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85299999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83399999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.74299999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71099999999999997}]},{"osd":8,"last update":"Wed Oct 18 12:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64300000000000002,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Wed Oct 18 12:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.66200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.432,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79800000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.874}]},{"osd":3,"last update":"Wed Oct 18 12:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.432},"min":{"1min":0.38,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.753},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60999999999999999,"15min":0.58999999999999997},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.505}]},{"osd":5,"last update":"Wed Oct 18 12:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66200000000000003,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.94199999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.434,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.52600000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.52000000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.53800000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.81499999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.59299999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.83999999999999997}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.69199999999999995}]}]},{"osd":2,"up_from":19,"seq":81604378682,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.90000000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.50900000000000001},"min":{"1min":0.41799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82399999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.5},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.5},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79600000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:24:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.437}]},{"osd":6,"last update":"Wed Oct 18 12:23:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88300000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64800000000000002,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.82799999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.58099999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.84099999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":489618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69299999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.60499999999999998},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93200000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.66200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76200000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:24:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83999999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.72599999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.79700000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.626},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":489536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.28599999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.28999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78500000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:24:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66000000000000003,"15min":0.64500000000000002},"min":{"1min":0.47299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90000000000000002,"5min":1.077,"15min":1.077},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.2529999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.82999999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.95699999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1083829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74299999999999999,"5min":0.878,"15min":0.878},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41499999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.36599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40200000000000002,"15min":0.40000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40699999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74099999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.434,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.38900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.47899999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46300000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68400000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.627}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.548,"5min":4.548,"15min":4.548},"last":0.64400000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":21417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:24:38.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:24:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:38 smithi152 ceph-mon[134767]: pgmap v269: 129 pgs: 129 active+clean; 598 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2023-10-18T12:24:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3607214241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/16484032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:38.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:38 smithi186 ceph-mon[135253]: pgmap v269: 129 pgs: 129 active+clean; 598 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2023-10-18T12:24:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3607214241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:24:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/16484032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/167903684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88100f80 msgr2=0x7f0a8810d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/167903684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88100f80 0x7f0a8810d490 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000b0a0 tx=0x7f0a8000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/167903684 shutdown_connections 2023-10-18T12:24:38.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/167903684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88100f80 0x7f0a8810d490 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/167903684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a88106660 0x7f0a88106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/167903684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a881005c0 0x7f0a88100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/167903684 >> 172.21.15.70:0/167903684 conn(0x7f0a880fc070 msgr2=0x7f0a880fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:38.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/167903684 shutdown_connections 2023-10-18T12:24:38.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.773+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/167903684 wait complete. 2023-10-18T12:24:38.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 Processor -- start 2023-10-18T12:24:38.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 -- start start 2023-10-18T12:24:38.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 0x7f0a88072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a88100f80 0x7f0a88073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88106660 0x7f0a88073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881a4cb0 con 0x7f0a88106660 2023-10-18T12:24:38.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881a4e20 con 0x7f0a881005c0 2023-10-18T12:24:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.774+0000 7f0a8f1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881a4fe0 con 0x7f0a88100f80 2023-10-18T12:24:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 0x7f0a88072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88106660 0x7f0a88073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 0x7f0a88072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43344/0 (socket says 172.21.15.70:43344) 2023-10-18T12:24:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8cf9b700 1 -- 172.21.15.70:0/737099292 learned_addr learned my addr 172.21.15.70:0/737099292 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a87fff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a88100f80 0x7f0a88073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 -- 172.21.15.70:0/737099292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a88100f80 msgr2=0x7f0a88073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a88100f80 0x7f0a88073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 -- 172.21.15.70:0/737099292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 msgr2=0x7f0a88072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 0x7f0a88072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:38.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 -- 172.21.15.70:0/737099292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80009d00 con 0x7f0a88106660 2023-10-18T12:24:38.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8cf9b700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 0x7f0a88072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:38.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.775+0000 7f0a8d79c700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88106660 0x7f0a88073570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000b070 tx=0x7f0a80007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:38.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.776+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8000e040 con 0x7f0a88106660 2023-10-18T12:24:38.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.776+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a8001dd50 con 0x7f0a88106660 2023-10-18T12:24:38.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.776+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800123f0 con 0x7f0a88106660 2023-10-18T12:24:38.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.776+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a881a5260 con 0x7f0a88106660 2023-10-18T12:24:38.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.776+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a881a5690 con 0x7f0a88106660 2023-10-18T12:24:38.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.777+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a88101d00 con 0x7f0a88106660 2023-10-18T12:24:38.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.779+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f0a8001d890 con 0x7f0a88106660 2023-10-18T12:24:38.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.780+0000 7f0a85ffb700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a780a15e0 0x7f0a780a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:38.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.780+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f0a800cee60 con 0x7f0a88106660 2023-10-18T12:24:38.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.781+0000 7f0a8cf9b700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a780a15e0 0x7f0a780a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:38.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.781+0000 7f0a8cf9b700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a780a15e0 0x7f0a780a3aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0103d0 tx=0x7f0a7c007480 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:38.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.784+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a800980c0 con 0x7f0a88106660 2023-10-18T12:24:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.982+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a88110560 con 0x7f0a780a15e0 2023-10-18T12:24:38.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:38.994+0000 7f0a85ffb700 1 -- 172.21.15.70:0/737099292 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422699 (secure 0 0 0) 0x7f0a88110560 con 0x7f0a780a15e0 2023-10-18T12:24:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:39.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a780a15e0 msgr2=0x7f0a780a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:39.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a780a15e0 0x7f0a780a3aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0103d0 tx=0x7f0a7c007480 comp rx=0 tx=0).stop 2023-10-18T12:24:39.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88106660 msgr2=0x7f0a88073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:39.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88106660 0x7f0a88073570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000b070 tx=0x7f0a80007b60 comp rx=0 tx=0).stop 2023-10-18T12:24:39.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 shutdown_connections 2023-10-18T12:24:39.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a88106660 0x7f0a88073570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:39.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a780a15e0 0x7f0a780a3aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:39.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a881005c0 0x7f0a88072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:39.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 --2- 172.21.15.70:0/737099292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a88100f80 0x7f0a88073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:39.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 >> 172.21.15.70:0/737099292 conn(0x7f0a880fc070 msgr2=0x7f0a880fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:39.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 shutdown_connections 2023-10-18T12:24:39.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:39.000+0000 7f0a8f1ff700 1 -- 172.21.15.70:0/737099292 wait complete. 2023-10-18T12:24:39.009 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:24:39.021 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2023-10-18T12:24:37.596789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612389,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":150,"num_read_kb":149,"num_write":236,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337680,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604144,"statfs":{"total":1151940427776,"available":1151594643456,"internally_reserved":0,"allocated":13742080,"data_stored":7680065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19611,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":30,"num_write":10,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003236"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145077+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:24:14.145077+0000","last_peered":"2023-10-18T12:24:14.145077+0000","last_clean":"2023-10-18T12:24:14.145077+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:24:14.145077+0000","last_undegraded":"2023-10-18T12:24:14.145077+0000","last_fullsized":"2023-10-18T12:24:14.145077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144990+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:24:14.144990+0000","last_peered":"2023-10-18T12:24:14.144990+0000","last_clean":"2023-10-18T12:24:14.144990+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:24:14.144990+0000","last_undegraded":"2023-10-18T12:24:14.144990+0000","last_fullsized":"2023-10-18T12:24:14.144990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146895+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:24:14.146895+0000","last_peered":"2023-10-18T12:24:14.146895+0000","last_clean":"2023-10-18T12:24:14.146895+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:24:14.146895+0000","last_undegraded":"2023-10-18T12:24:14.146895+0000","last_fullsized":"2023-10-18T12:24:14.146895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145041+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:24:14.145041+0000","last_peered":"2023-10-18T12:24:14.145041+0000","last_clean":"2023-10-18T12:24:14.145041+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:24:14.145041+0000","last_undegraded":"2023-10-18T12:24:14.145041+0000","last_fullsized":"2023-10-18T12:24:14.145041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139989+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:24:14.139989+0000","last_peered":"2023-10-18T12:24:14.139989+0000","last_clean":"2023-10-18T12:24:14.139989+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:24:14.139989+0000","last_undegraded":"2023-10-18T12:24:14.139989+0000","last_fullsized":"2023-10-18T12:24:14.139989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178381+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:24:14.178381+0000","last_peered":"2023-10-18T12:24:14.178381+0000","last_clean":"2023-10-18T12:24:14.178381+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:24:14.178381+0000","last_undegraded":"2023-10-18T12:24:14.178381+0000","last_fullsized":"2023-10-18T12:24:14.178381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146931+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:24:14.146931+0000","last_peered":"2023-10-18T12:24:14.146931+0000","last_clean":"2023-10-18T12:24:14.146931+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:24:14.146931+0000","last_undegraded":"2023-10-18T12:24:14.146931+0000","last_fullsized":"2023-10-18T12:24:14.146931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144154+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:24:14.144154+0000","last_peered":"2023-10-18T12:24:14.144154+0000","last_clean":"2023-10-18T12:24:14.144154+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:24:14.144154+0000","last_undegraded":"2023-10-18T12:24:14.144154+0000","last_fullsized":"2023-10-18T12:24:14.144154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140950+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:24:14.140950+0000","last_peered":"2023-10-18T12:24:14.140950+0000","last_clean":"2023-10-18T12:24:14.140950+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:24:14.140950+0000","last_undegraded":"2023-10-18T12:24:14.140950+0000","last_fullsized":"2023-10-18T12:24:14.140950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178575+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:24:14.178575+0000","last_peered":"2023-10-18T12:24:14.178575+0000","last_clean":"2023-10-18T12:24:14.178575+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:24:14.178575+0000","last_undegraded":"2023-10-18T12:24:14.178575+0000","last_fullsized":"2023-10-18T12:24:14.178575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145872+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:24:14.145872+0000","last_peered":"2023-10-18T12:24:14.145872+0000","last_clean":"2023-10-18T12:24:14.145872+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:24:14.145872+0000","last_undegraded":"2023-10-18T12:24:14.145872+0000","last_fullsized":"2023-10-18T12:24:14.145872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144948+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:24:14.144948+0000","last_peered":"2023-10-18T12:24:14.144948+0000","last_clean":"2023-10-18T12:24:14.144948+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:24:14.144948+0000","last_undegraded":"2023-10-18T12:24:14.144948+0000","last_fullsized":"2023-10-18T12:24:14.144948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138429+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:24:14.138429+0000","last_peered":"2023-10-18T12:24:14.138429+0000","last_clean":"2023-10-18T12:24:14.138429+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:24:14.138429+0000","last_undegraded":"2023-10-18T12:24:14.138429+0000","last_fullsized":"2023-10-18T12:24:14.138429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143503+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:24:14.143503+0000","last_peered":"2023-10-18T12:24:14.143503+0000","last_clean":"2023-10-18T12:24:14.143503+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:24:14.143503+0000","last_undegraded":"2023-10-18T12:24:14.143503+0000","last_fullsized":"2023-10-18T12:24:14.143503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143334+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:24:14.143334+0000","last_peered":"2023-10-18T12:24:14.143334+0000","last_clean":"2023-10-18T12:24:14.143334+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:24:14.143334+0000","last_undegraded":"2023-10-18T12:24:14.143334+0000","last_fullsized":"2023-10-18T12:24:14.143334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140092+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:24:14.140092+0000","last_peered":"2023-10-18T12:24:14.140092+0000","last_clean":"2023-10-18T12:24:14.140092+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:24:14.140092+0000","last_undegraded":"2023-10-18T12:24:14.140092+0000","last_fullsized":"2023-10-18T12:24:14.140092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145255+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:24:14.145255+0000","last_peered":"2023-10-18T12:24:14.145255+0000","last_clean":"2023-10-18T12:24:14.145255+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:24:14.145255+0000","last_undegraded":"2023-10-18T12:24:14.145255+0000","last_fullsized":"2023-10-18T12:24:14.145255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.163381+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:24:14.163381+0000","last_peered":"2023-10-18T12:24:14.163381+0000","last_clean":"2023-10-18T12:24:14.163381+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:24:14.163381+0000","last_undegraded":"2023-10-18T12:24:14.163381+0000","last_fullsized":"2023-10-18T12:24:14.163381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145652+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:24:14.145652+0000","last_peered":"2023-10-18T12:24:14.145652+0000","last_clean":"2023-10-18T12:24:14.145652+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:24:14.145652+0000","last_undegraded":"2023-10-18T12:24:14.145652+0000","last_fullsized":"2023-10-18T12:24:14.145652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350574+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:24:16.350574+0000","last_peered":"2023-10-18T12:24:16.350574+0000","last_clean":"2023-10-18T12:24:16.350574+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:24:16.350574+0000","last_undegraded":"2023-10-18T12:24:16.350574+0000","last_fullsized":"2023-10-18T12:24:16.350574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350162+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:24:16.350162+0000","last_peered":"2023-10-18T12:24:16.350162+0000","last_clean":"2023-10-18T12:24:16.350162+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:24:16.350162+0000","last_undegraded":"2023-10-18T12:24:16.350162+0000","last_fullsized":"2023-10-18T12:24:16.350162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144626+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:24:14.144626+0000","last_peered":"2023-10-18T12:24:14.144626+0000","last_clean":"2023-10-18T12:24:14.144626+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:24:14.144626+0000","last_undegraded":"2023-10-18T12:24:14.144626+0000","last_fullsized":"2023-10-18T12:24:14.144626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.143271+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:24:15.143271+0000","last_peered":"2023-10-18T12:24:15.143271+0000","last_clean":"2023-10-18T12:24:15.143271+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:24:15.143271+0000","last_undegraded":"2023-10-18T12:24:15.143271+0000","last_fullsized":"2023-10-18T12:24:15.143271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145925+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:24:14.145925+0000","last_peered":"2023-10-18T12:24:14.145925+0000","last_clean":"2023-10-18T12:24:14.145925+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:24:14.145925+0000","last_undegraded":"2023-10-18T12:24:14.145925+0000","last_fullsized":"2023-10-18T12:24:14.145925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145923+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:24:14.145923+0000","last_peered":"2023-10-18T12:24:14.145923+0000","last_clean":"2023-10-18T12:24:14.145923+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:24:14.145923+0000","last_undegraded":"2023-10-18T12:24:14.145923+0000","last_fullsized":"2023-10-18T12:24:14.145923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144332+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:24:15.144332+0000","last_peered":"2023-10-18T12:24:15.144332+0000","last_clean":"2023-10-18T12:24:15.144332+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:24:15.144332+0000","last_undegraded":"2023-10-18T12:24:15.144332+0000","last_fullsized":"2023-10-18T12:24:15.144332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140356+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:24:14.140356+0000","last_peered":"2023-10-18T12:24:14.140356+0000","last_clean":"2023-10-18T12:24:14.140356+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:24:14.140356+0000","last_undegraded":"2023-10-18T12:24:14.140356+0000","last_fullsized":"2023-10-18T12:24:14.140356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140523+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:24:14.140523+0000","last_peered":"2023-10-18T12:24:14.140523+0000","last_clean":"2023-10-18T12:24:14.140523+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:24:14.140523+0000","last_undegraded":"2023-10-18T12:24:14.140523+0000","last_fullsized":"2023-10-18T12:24:14.140523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.143443+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:24:15.143443+0000","last_peered":"2023-10-18T12:24:15.143443+0000","last_clean":"2023-10-18T12:24:15.143443+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:24:15.143443+0000","last_undegraded":"2023-10-18T12:24:15.143443+0000","last_fullsized":"2023-10-18T12:24:15.143443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140163+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:24:14.140163+0000","last_peered":"2023-10-18T12:24:14.140163+0000","last_clean":"2023-10-18T12:24:14.140163+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:24:14.140163+0000","last_undegraded":"2023-10-18T12:24:14.140163+0000","last_fullsized":"2023-10-18T12:24:14.140163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143520+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:24:14.143520+0000","last_peered":"2023-10-18T12:24:14.143520+0000","last_clean":"2023-10-18T12:24:14.143520+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:24:14.143520+0000","last_undegraded":"2023-10-18T12:24:14.143520+0000","last_fullsized":"2023-10-18T12:24:14.143520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349939+0000","last_change":"2023-10-18T12:24:14.152969+0000","last_active":"2023-10-18T12:24:16.349939+0000","last_peered":"2023-10-18T12:24:16.349939+0000","last_clean":"2023-10-18T12:24:16.349939+0000","last_became_active":"2023-10-18T12:24:14.152535+0000","last_became_peered":"2023-10-18T12:24:14.152535+0000","last_unstale":"2023-10-18T12:24:16.349939+0000","last_undegraded":"2023-10-18T12:24:16.349939+0000","last_fullsized":"2023-10-18T12:24:16.349939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:28.206795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146186+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:24:14.146186+0000","last_peered":"2023-10-18T12:24:14.146186+0000","last_clean":"2023-10-18T12:24:14.146186+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:24:14.146186+0000","last_undegraded":"2023-10-18T12:24:14.146186+0000","last_fullsized":"2023-10-18T12:24:14.146186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139918+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:24:14.139918+0000","last_peered":"2023-10-18T12:24:14.139918+0000","last_clean":"2023-10-18T12:24:14.139918+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:24:14.139918+0000","last_undegraded":"2023-10-18T12:24:14.139918+0000","last_fullsized":"2023-10-18T12:24:14.139918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349871+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:24:16.349871+0000","last_peered":"2023-10-18T12:24:16.349871+0000","last_clean":"2023-10-18T12:24:16.349871+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:24:16.349871+0000","last_undegraded":"2023-10-18T12:24:16.349871+0000","last_fullsized":"2023-10-18T12:24:16.349871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178262+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:24:14.178262+0000","last_peered":"2023-10-18T12:24:14.178262+0000","last_clean":"2023-10-18T12:24:14.178262+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:24:14.178262+0000","last_undegraded":"2023-10-18T12:24:14.178262+0000","last_fullsized":"2023-10-18T12:24:14.178262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.146288+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:24:14.146288+0000","last_peered":"2023-10-18T12:24:14.146288+0000","last_clean":"2023-10-18T12:24:14.146288+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:24:14.146288+0000","last_undegraded":"2023-10-18T12:24:14.146288+0000","last_fullsized":"2023-10-18T12:24:14.146288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.141268+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:24:15.141268+0000","last_peered":"2023-10-18T12:24:15.141268+0000","last_clean":"2023-10-18T12:24:15.141268+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:24:15.141268+0000","last_undegraded":"2023-10-18T12:24:15.141268+0000","last_fullsized":"2023-10-18T12:24:15.141268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138214+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:24:14.138214+0000","last_peered":"2023-10-18T12:24:14.138214+0000","last_clean":"2023-10-18T12:24:14.138214+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:24:14.138214+0000","last_undegraded":"2023-10-18T12:24:14.138214+0000","last_fullsized":"2023-10-18T12:24:14.138214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143994+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:24:14.143994+0000","last_peered":"2023-10-18T12:24:14.143994+0000","last_clean":"2023-10-18T12:24:14.143994+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:24:14.143994+0000","last_undegraded":"2023-10-18T12:24:14.143994+0000","last_fullsized":"2023-10-18T12:24:14.143994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.467590+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:24:16.467590+0000","last_peered":"2023-10-18T12:24:16.467590+0000","last_clean":"2023-10-18T12:24:16.467590+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:24:16.467590+0000","last_undegraded":"2023-10-18T12:24:16.467590+0000","last_fullsized":"2023-10-18T12:24:16.467590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143686+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:24:14.143686+0000","last_peered":"2023-10-18T12:24:14.143686+0000","last_clean":"2023-10-18T12:24:14.143686+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:24:14.143686+0000","last_undegraded":"2023-10-18T12:24:14.143686+0000","last_fullsized":"2023-10-18T12:24:14.143686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.162058+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:24:14.162058+0000","last_peered":"2023-10-18T12:24:14.162058+0000","last_clean":"2023-10-18T12:24:14.162058+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:24:14.162058+0000","last_undegraded":"2023-10-18T12:24:14.162058+0000","last_fullsized":"2023-10-18T12:24:14.162058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.425610+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:24:15.425610+0000","last_peered":"2023-10-18T12:24:15.425610+0000","last_clean":"2023-10-18T12:24:15.425610+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:24:15.425610+0000","last_undegraded":"2023-10-18T12:24:15.425610+0000","last_fullsized":"2023-10-18T12:24:15.425610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178617+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:24:14.178617+0000","last_peered":"2023-10-18T12:24:14.178617+0000","last_clean":"2023-10-18T12:24:14.178617+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:24:14.178617+0000","last_undegraded":"2023-10-18T12:24:14.178617+0000","last_fullsized":"2023-10-18T12:24:14.178617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140765+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:24:14.140765+0000","last_peered":"2023-10-18T12:24:14.140765+0000","last_clean":"2023-10-18T12:24:14.140765+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:24:14.140765+0000","last_undegraded":"2023-10-18T12:24:14.140765+0000","last_fullsized":"2023-10-18T12:24:14.140765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349603+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:24:16.349603+0000","last_peered":"2023-10-18T12:24:16.349603+0000","last_clean":"2023-10-18T12:24:16.349603+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:24:16.349603+0000","last_undegraded":"2023-10-18T12:24:16.349603+0000","last_fullsized":"2023-10-18T12:24:16.349603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145654+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:24:14.145654+0000","last_peered":"2023-10-18T12:24:14.145654+0000","last_clean":"2023-10-18T12:24:14.145654+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:24:14.145654+0000","last_undegraded":"2023-10-18T12:24:14.145654+0000","last_fullsized":"2023-10-18T12:24:14.145654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145994+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:24:14.145994+0000","last_peered":"2023-10-18T12:24:14.145994+0000","last_clean":"2023-10-18T12:24:14.145994+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:24:14.145994+0000","last_undegraded":"2023-10-18T12:24:14.145994+0000","last_fullsized":"2023-10-18T12:24:14.145994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144712+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:24:15.144712+0000","last_peered":"2023-10-18T12:24:15.144712+0000","last_clean":"2023-10-18T12:24:15.144712+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:24:15.144712+0000","last_undegraded":"2023-10-18T12:24:15.144712+0000","last_fullsized":"2023-10-18T12:24:15.144712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143428+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:24:14.143428+0000","last_peered":"2023-10-18T12:24:14.143428+0000","last_clean":"2023-10-18T12:24:14.143428+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:24:14.143428+0000","last_undegraded":"2023-10-18T12:24:14.143428+0000","last_fullsized":"2023-10-18T12:24:14.143428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140698+0000","last_change":"2023-10-18T12:24:09.974796+0000","last_active":"2023-10-18T12:24:14.140698+0000","last_peered":"2023-10-18T12:24:14.140698+0000","last_clean":"2023-10-18T12:24:14.140698+0000","last_became_active":"2023-10-18T12:24:09.974498+0000","last_became_peered":"2023-10-18T12:24:09.974498+0000","last_unstale":"2023-10-18T12:24:14.140698+0000","last_undegraded":"2023-10-18T12:24:14.140698+0000","last_fullsized":"2023-10-18T12:24:14.140698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:11:54.782709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349745+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:24:16.349745+0000","last_peered":"2023-10-18T12:24:16.349745+0000","last_clean":"2023-10-18T12:24:16.349745+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:24:16.349745+0000","last_undegraded":"2023-10-18T12:24:16.349745+0000","last_fullsized":"2023-10-18T12:24:16.349745+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'16","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:36.135028+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:24:36.135028+0000","last_peered":"2023-10-18T12:24:36.135028+0000","last_clean":"2023-10-18T12:24:36.135028+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:24:36.135028+0000","last_undegraded":"2023-10-18T12:24:36.135028+0000","last_fullsized":"2023-10-18T12:24:36.135028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21223,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":25,"num_write":16,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140163+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:24:14.140163+0000","last_peered":"2023-10-18T12:24:14.140163+0000","last_clean":"2023-10-18T12:24:14.140163+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:24:14.140163+0000","last_undegraded":"2023-10-18T12:24:14.140163+0000","last_fullsized":"2023-10-18T12:24:14.140163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350870+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:24:16.350870+0000","last_peered":"2023-10-18T12:24:16.350870+0000","last_clean":"2023-10-18T12:24:16.350870+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:24:16.350870+0000","last_undegraded":"2023-10-18T12:24:16.350870+0000","last_fullsized":"2023-10-18T12:24:16.350870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144571+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:24:14.144571+0000","last_peered":"2023-10-18T12:24:14.144571+0000","last_clean":"2023-10-18T12:24:14.144571+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:24:14.144571+0000","last_undegraded":"2023-10-18T12:24:14.144571+0000","last_fullsized":"2023-10-18T12:24:14.144571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139188+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:24:14.139188+0000","last_peered":"2023-10-18T12:24:14.139188+0000","last_clean":"2023-10-18T12:24:14.139188+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:24:14.139188+0000","last_undegraded":"2023-10-18T12:24:14.139188+0000","last_fullsized":"2023-10-18T12:24:14.139188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.141167+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:24:15.141167+0000","last_peered":"2023-10-18T12:24:15.141167+0000","last_clean":"2023-10-18T12:24:15.141167+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:24:15.141167+0000","last_undegraded":"2023-10-18T12:24:15.141167+0000","last_fullsized":"2023-10-18T12:24:15.141167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.148373+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:24:14.148373+0000","last_peered":"2023-10-18T12:24:14.148373+0000","last_clean":"2023-10-18T12:24:14.148373+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:24:14.148373+0000","last_undegraded":"2023-10-18T12:24:14.148373+0000","last_fullsized":"2023-10-18T12:24:14.148373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144711+0000","last_change":"2023-10-18T12:24:09.968891+0000","last_active":"2023-10-18T12:24:14.144711+0000","last_peered":"2023-10-18T12:24:14.144711+0000","last_clean":"2023-10-18T12:24:14.144711+0000","last_became_active":"2023-10-18T12:24:09.968566+0000","last_became_peered":"2023-10-18T12:24:09.968566+0000","last_unstale":"2023-10-18T12:24:14.144711+0000","last_undegraded":"2023-10-18T12:24:14.144711+0000","last_fullsized":"2023-10-18T12:24:14.144711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:54.830870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350000+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:24:16.350000+0000","last_peered":"2023-10-18T12:24:16.350000+0000","last_clean":"2023-10-18T12:24:16.350000+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:24:16.350000+0000","last_undegraded":"2023-10-18T12:24:16.350000+0000","last_fullsized":"2023-10-18T12:24:16.350000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143710+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:24:14.143710+0000","last_peered":"2023-10-18T12:24:14.143710+0000","last_clean":"2023-10-18T12:24:14.143710+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:24:14.143710+0000","last_undegraded":"2023-10-18T12:24:14.143710+0000","last_fullsized":"2023-10-18T12:24:14.143710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144141+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:24:14.144141+0000","last_peered":"2023-10-18T12:24:14.144141+0000","last_clean":"2023-10-18T12:24:14.144141+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:24:14.144141+0000","last_undegraded":"2023-10-18T12:24:14.144141+0000","last_fullsized":"2023-10-18T12:24:14.144141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.140919+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:24:15.140919+0000","last_peered":"2023-10-18T12:24:15.140919+0000","last_clean":"2023-10-18T12:24:15.140919+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:24:15.140919+0000","last_undegraded":"2023-10-18T12:24:15.140919+0000","last_fullsized":"2023-10-18T12:24:15.140919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139531+0000","last_change":"2023-10-18T12:24:08.977239+0000","last_active":"2023-10-18T12:24:14.139531+0000","last_peered":"2023-10-18T12:24:14.139531+0000","last_clean":"2023-10-18T12:24:14.139531+0000","last_became_active":"2023-10-18T12:24:08.976918+0000","last_became_peered":"2023-10-18T12:24:08.976918+0000","last_unstale":"2023-10-18T12:24:14.139531+0000","last_undegraded":"2023-10-18T12:24:14.139531+0000","last_fullsized":"2023-10-18T12:24:14.139531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:30.752782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144893+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:24:14.144893+0000","last_peered":"2023-10-18T12:24:14.144893+0000","last_clean":"2023-10-18T12:24:14.144893+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:24:14.144893+0000","last_undegraded":"2023-10-18T12:24:14.144893+0000","last_fullsized":"2023-10-18T12:24:14.144893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349566+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:24:16.349566+0000","last_peered":"2023-10-18T12:24:16.349566+0000","last_clean":"2023-10-18T12:24:16.349566+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:24:16.349566+0000","last_undegraded":"2023-10-18T12:24:16.349566+0000","last_fullsized":"2023-10-18T12:24:16.349566+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.085302+0000","last_change":"2023-10-18T12:24:08.966712+0000","last_active":"2023-10-18T12:24:16.085302+0000","last_peered":"2023-10-18T12:24:16.085302+0000","last_clean":"2023-10-18T12:24:16.085302+0000","last_became_active":"2023-10-18T12:24:08.966392+0000","last_became_peered":"2023-10-18T12:24:08.966392+0000","last_unstale":"2023-10-18T12:24:16.085302+0000","last_undegraded":"2023-10-18T12:24:16.085302+0000","last_fullsized":"2023-10-18T12:24:16.085302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:44:14.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145237+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:24:14.145237+0000","last_peered":"2023-10-18T12:24:14.145237+0000","last_clean":"2023-10-18T12:24:14.145237+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:24:14.145237+0000","last_undegraded":"2023-10-18T12:24:14.145237+0000","last_fullsized":"2023-10-18T12:24:14.145237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.084865+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:24:16.084865+0000","last_peered":"2023-10-18T12:24:16.084865+0000","last_clean":"2023-10-18T12:24:16.084865+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:24:16.084865+0000","last_undegraded":"2023-10-18T12:24:16.084865+0000","last_fullsized":"2023-10-18T12:24:16.084865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.162569+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:24:14.162569+0000","last_peered":"2023-10-18T12:24:14.162569+0000","last_clean":"2023-10-18T12:24:14.162569+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:24:14.162569+0000","last_undegraded":"2023-10-18T12:24:14.162569+0000","last_fullsized":"2023-10-18T12:24:14.162569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178757+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:24:14.178757+0000","last_peered":"2023-10-18T12:24:14.178757+0000","last_clean":"2023-10-18T12:24:14.178757+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:24:14.178757+0000","last_undegraded":"2023-10-18T12:24:14.178757+0000","last_fullsized":"2023-10-18T12:24:14.178757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.140694+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:24:15.140694+0000","last_peered":"2023-10-18T12:24:15.140694+0000","last_clean":"2023-10-18T12:24:15.140694+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:24:15.140694+0000","last_undegraded":"2023-10-18T12:24:15.140694+0000","last_fullsized":"2023-10-18T12:24:15.140694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145112+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:24:14.145112+0000","last_peered":"2023-10-18T12:24:14.145112+0000","last_clean":"2023-10-18T12:24:14.145112+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:24:14.145112+0000","last_undegraded":"2023-10-18T12:24:14.145112+0000","last_fullsized":"2023-10-18T12:24:14.145112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.178544+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:24:14.178544+0000","last_peered":"2023-10-18T12:24:14.178544+0000","last_clean":"2023-10-18T12:24:14.178544+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:24:14.178544+0000","last_undegraded":"2023-10-18T12:24:14.178544+0000","last_fullsized":"2023-10-18T12:24:14.178544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.140703+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:24:15.140703+0000","last_peered":"2023-10-18T12:24:15.140703+0000","last_clean":"2023-10-18T12:24:15.140703+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:24:15.140703+0000","last_undegraded":"2023-10-18T12:24:15.140703+0000","last_fullsized":"2023-10-18T12:24:15.140703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143868+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:24:14.143868+0000","last_peered":"2023-10-18T12:24:14.143868+0000","last_clean":"2023-10-18T12:24:14.143868+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:24:14.143868+0000","last_undegraded":"2023-10-18T12:24:14.143868+0000","last_fullsized":"2023-10-18T12:24:14.143868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138126+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:24:14.138126+0000","last_peered":"2023-10-18T12:24:14.138126+0000","last_clean":"2023-10-18T12:24:14.138126+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:24:14.138126+0000","last_undegraded":"2023-10-18T12:24:14.138126+0000","last_fullsized":"2023-10-18T12:24:14.138126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.469368+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:24:15.469368+0000","last_peered":"2023-10-18T12:24:15.469368+0000","last_clean":"2023-10-18T12:24:15.469368+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:24:15.469368+0000","last_undegraded":"2023-10-18T12:24:15.469368+0000","last_fullsized":"2023-10-18T12:24:15.469368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145960+0000","last_change":"2023-10-18T12:24:09.985663+0000","last_active":"2023-10-18T12:24:14.145960+0000","last_peered":"2023-10-18T12:24:14.145960+0000","last_clean":"2023-10-18T12:24:14.145960+0000","last_became_active":"2023-10-18T12:24:09.983659+0000","last_became_peered":"2023-10-18T12:24:09.983659+0000","last_unstale":"2023-10-18T12:24:14.145960+0000","last_undegraded":"2023-10-18T12:24:14.145960+0000","last_fullsized":"2023-10-18T12:24:14.145960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:39.706073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144355+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:24:14.144355+0000","last_peered":"2023-10-18T12:24:14.144355+0000","last_clean":"2023-10-18T12:24:14.144355+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:24:14.144355+0000","last_undegraded":"2023-10-18T12:24:14.144355+0000","last_fullsized":"2023-10-18T12:24:14.144355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":324,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143960+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:24:14.143960+0000","last_peered":"2023-10-18T12:24:14.143960+0000","last_clean":"2023-10-18T12:24:14.143960+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:24:14.143960+0000","last_undegraded":"2023-10-18T12:24:14.143960+0000","last_fullsized":"2023-10-18T12:24:14.143960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350522+0000","last_change":"2023-10-18T12:24:14.179768+0000","last_active":"2023-10-18T12:24:16.350522+0000","last_peered":"2023-10-18T12:24:16.350522+0000","last_clean":"2023-10-18T12:24:16.350522+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:24:16.350522+0000","last_undegraded":"2023-10-18T12:24:16.350522+0000","last_fullsized":"2023-10-18T12:24:16.350522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:17.695525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138189+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:24:14.138189+0000","last_peered":"2023-10-18T12:24:14.138189+0000","last_clean":"2023-10-18T12:24:14.138189+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:24:14.138189+0000","last_undegraded":"2023-10-18T12:24:14.138189+0000","last_fullsized":"2023-10-18T12:24:14.138189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144049+0000","last_change":"2023-10-18T12:24:08.964976+0000","last_active":"2023-10-18T12:24:14.144049+0000","last_peered":"2023-10-18T12:24:14.144049+0000","last_clean":"2023-10-18T12:24:14.144049+0000","last_became_active":"2023-10-18T12:24:08.963055+0000","last_became_peered":"2023-10-18T12:24:08.963055+0000","last_unstale":"2023-10-18T12:24:14.144049+0000","last_undegraded":"2023-10-18T12:24:14.144049+0000","last_fullsized":"2023-10-18T12:24:14.144049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:05.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.141032+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:24:15.141032+0000","last_peered":"2023-10-18T12:24:15.141032+0000","last_clean":"2023-10-18T12:24:15.141032+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:24:15.141032+0000","last_undegraded":"2023-10-18T12:24:15.141032+0000","last_fullsized":"2023-10-18T12:24:15.141032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140232+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:24:14.140232+0000","last_peered":"2023-10-18T12:24:14.140232+0000","last_clean":"2023-10-18T12:24:14.140232+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:24:14.140232+0000","last_undegraded":"2023-10-18T12:24:14.140232+0000","last_fullsized":"2023-10-18T12:24:14.140232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138956+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:24:14.138956+0000","last_peered":"2023-10-18T12:24:14.138956+0000","last_clean":"2023-10-18T12:24:14.138956+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:24:14.138956+0000","last_undegraded":"2023-10-18T12:24:14.138956+0000","last_fullsized":"2023-10-18T12:24:14.138956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.425448+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:24:15.425448+0000","last_peered":"2023-10-18T12:24:15.425448+0000","last_clean":"2023-10-18T12:24:15.425448+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:24:15.425448+0000","last_undegraded":"2023-10-18T12:24:15.425448+0000","last_fullsized":"2023-10-18T12:24:15.425448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.163474+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:24:14.163474+0000","last_peered":"2023-10-18T12:24:14.163474+0000","last_clean":"2023-10-18T12:24:14.163474+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:24:14.163474+0000","last_undegraded":"2023-10-18T12:24:14.163474+0000","last_fullsized":"2023-10-18T12:24:14.163474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144729+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:24:14.144729+0000","last_peered":"2023-10-18T12:24:14.144729+0000","last_clean":"2023-10-18T12:24:14.144729+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:24:14.144729+0000","last_undegraded":"2023-10-18T12:24:14.144729+0000","last_fullsized":"2023-10-18T12:24:14.144729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349881+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:24:16.349881+0000","last_peered":"2023-10-18T12:24:16.349881+0000","last_clean":"2023-10-18T12:24:16.349881+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:24:16.349881+0000","last_undegraded":"2023-10-18T12:24:16.349881+0000","last_fullsized":"2023-10-18T12:24:16.349881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143601+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:24:14.143601+0000","last_peered":"2023-10-18T12:24:14.143601+0000","last_clean":"2023-10-18T12:24:14.143601+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:24:14.143601+0000","last_undegraded":"2023-10-18T12:24:14.143601+0000","last_fullsized":"2023-10-18T12:24:14.143601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145074+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:24:14.145074+0000","last_peered":"2023-10-18T12:24:14.145074+0000","last_clean":"2023-10-18T12:24:14.145074+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:24:14.145074+0000","last_undegraded":"2023-10-18T12:24:14.145074+0000","last_fullsized":"2023-10-18T12:24:14.145074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144663+0000","last_change":"2023-10-18T12:24:14.179741+0000","last_active":"2023-10-18T12:24:15.144663+0000","last_peered":"2023-10-18T12:24:15.144663+0000","last_clean":"2023-10-18T12:24:15.144663+0000","last_became_active":"2023-10-18T12:24:14.179297+0000","last_became_peered":"2023-10-18T12:24:14.179297+0000","last_unstale":"2023-10-18T12:24:15.144663+0000","last_undegraded":"2023-10-18T12:24:15.144663+0000","last_fullsized":"2023-10-18T12:24:15.144663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:52.997230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140138+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:24:14.140138+0000","last_peered":"2023-10-18T12:24:14.140138+0000","last_clean":"2023-10-18T12:24:14.140138+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:24:14.140138+0000","last_undegraded":"2023-10-18T12:24:14.140138+0000","last_fullsized":"2023-10-18T12:24:14.140138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139570+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:24:14.139570+0000","last_peered":"2023-10-18T12:24:14.139570+0000","last_clean":"2023-10-18T12:24:14.139570+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:24:14.139570+0000","last_undegraded":"2023-10-18T12:24:14.139570+0000","last_fullsized":"2023-10-18T12:24:14.139570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350131+0000","last_change":"2023-10-18T12:24:14.168625+0000","last_active":"2023-10-18T12:24:16.350131+0000","last_peered":"2023-10-18T12:24:16.350131+0000","last_clean":"2023-10-18T12:24:16.350131+0000","last_became_active":"2023-10-18T12:24:14.168203+0000","last_became_peered":"2023-10-18T12:24:14.168203+0000","last_unstale":"2023-10-18T12:24:16.350131+0000","last_undegraded":"2023-10-18T12:24:16.350131+0000","last_fullsized":"2023-10-18T12:24:16.350131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:21.516180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143008+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:24:14.143008+0000","last_peered":"2023-10-18T12:24:14.143008+0000","last_clean":"2023-10-18T12:24:14.143008+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:24:14.143008+0000","last_undegraded":"2023-10-18T12:24:14.143008+0000","last_fullsized":"2023-10-18T12:24:14.143008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144036+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:24:14.144036+0000","last_peered":"2023-10-18T12:24:14.144036+0000","last_clean":"2023-10-18T12:24:14.144036+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:24:14.144036+0000","last_undegraded":"2023-10-18T12:24:14.144036+0000","last_fullsized":"2023-10-18T12:24:14.144036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144476+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:24:15.144476+0000","last_peered":"2023-10-18T12:24:15.144476+0000","last_clean":"2023-10-18T12:24:15.144476+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:24:15.144476+0000","last_undegraded":"2023-10-18T12:24:15.144476+0000","last_fullsized":"2023-10-18T12:24:15.144476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144540+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:24:14.144540+0000","last_peered":"2023-10-18T12:24:14.144540+0000","last_clean":"2023-10-18T12:24:14.144540+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:24:14.144540+0000","last_undegraded":"2023-10-18T12:24:14.144540+0000","last_fullsized":"2023-10-18T12:24:14.144540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144483+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:24:14.144483+0000","last_peered":"2023-10-18T12:24:14.144483+0000","last_clean":"2023-10-18T12:24:14.144483+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:24:14.144483+0000","last_undegraded":"2023-10-18T12:24:14.144483+0000","last_fullsized":"2023-10-18T12:24:14.144483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.144609+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:24:15.144609+0000","last_peered":"2023-10-18T12:24:15.144609+0000","last_clean":"2023-10-18T12:24:15.144609+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:24:15.144609+0000","last_undegraded":"2023-10-18T12:24:15.144609+0000","last_fullsized":"2023-10-18T12:24:15.144609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.084944+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:24:16.084944+0000","last_peered":"2023-10-18T12:24:16.084944+0000","last_clean":"2023-10-18T12:24:16.084944+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:24:16.084944+0000","last_undegraded":"2023-10-18T12:24:16.084944+0000","last_fullsized":"2023-10-18T12:24:16.084944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145280+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:24:14.145280+0000","last_peered":"2023-10-18T12:24:14.145280+0000","last_clean":"2023-10-18T12:24:14.145280+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:24:14.145280+0000","last_undegraded":"2023-10-18T12:24:14.145280+0000","last_fullsized":"2023-10-18T12:24:14.145280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:15.145439+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:24:15.145439+0000","last_peered":"2023-10-18T12:24:15.145439+0000","last_clean":"2023-10-18T12:24:15.145439+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:24:15.145439+0000","last_undegraded":"2023-10-18T12:24:15.145439+0000","last_fullsized":"2023-10-18T12:24:15.145439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145394+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:24:14.145394+0000","last_peered":"2023-10-18T12:24:14.145394+0000","last_clean":"2023-10-18T12:24:14.145394+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:24:14.145394+0000","last_undegraded":"2023-10-18T12:24:14.145394+0000","last_fullsized":"2023-10-18T12:24:14.145394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145984+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:24:14.145984+0000","last_peered":"2023-10-18T12:24:14.145984+0000","last_clean":"2023-10-18T12:24:14.145984+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:24:14.145984+0000","last_undegraded":"2023-10-18T12:24:14.145984+0000","last_fullsized":"2023-10-18T12:24:14.145984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.467096+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:24:16.467096+0000","last_peered":"2023-10-18T12:24:16.467096+0000","last_clean":"2023-10-18T12:24:16.467096+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:24:16.467096+0000","last_undegraded":"2023-10-18T12:24:16.467096+0000","last_fullsized":"2023-10-18T12:24:16.467096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.350670+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:24:16.350670+0000","last_peered":"2023-10-18T12:24:16.350670+0000","last_clean":"2023-10-18T12:24:16.350670+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:24:16.350670+0000","last_undegraded":"2023-10-18T12:24:16.350670+0000","last_fullsized":"2023-10-18T12:24:16.350670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144824+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:24:14.144824+0000","last_peered":"2023-10-18T12:24:14.144824+0000","last_clean":"2023-10-18T12:24:14.144824+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:24:14.144824+0000","last_undegraded":"2023-10-18T12:24:14.144824+0000","last_fullsized":"2023-10-18T12:24:14.144824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T12:24:16.349374+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:24:16.349374+0000","last_peered":"2023-10-18T12:24:16.349374+0000","last_clean":"2023-10-18T12:24:16.349374+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:24:16.349374+0000","last_undegraded":"2023-10-18T12:24:16.349374+0000","last_fullsized":"2023-10-18T12:24:16.349374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.139059+0000","last_change":"2023-10-18T12:24:08.975858+0000","last_active":"2023-10-18T12:24:14.139059+0000","last_peered":"2023-10-18T12:24:14.139059+0000","last_clean":"2023-10-18T12:24:14.139059+0000","last_became_active":"2023-10-18T12:24:08.974568+0000","last_became_peered":"2023-10-18T12:24:08.974568+0000","last_unstale":"2023-10-18T12:24:14.139059+0000","last_undegraded":"2023-10-18T12:24:14.139059+0000","last_fullsized":"2023-10-18T12:24:14.139059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:42.951310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145872+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:24:14.145872+0000","last_peered":"2023-10-18T12:24:14.145872+0000","last_clean":"2023-10-18T12:24:14.145872+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:24:14.145872+0000","last_undegraded":"2023-10-18T12:24:14.145872+0000","last_fullsized":"2023-10-18T12:24:14.145872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144132+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:24:14.144132+0000","last_peered":"2023-10-18T12:24:14.144132+0000","last_clean":"2023-10-18T12:24:14.144132+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:24:14.144132+0000","last_undegraded":"2023-10-18T12:24:14.144132+0000","last_fullsized":"2023-10-18T12:24:14.144132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144196+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:24:14.144196+0000","last_peered":"2023-10-18T12:24:14.144196+0000","last_clean":"2023-10-18T12:24:14.144196+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:24:14.144196+0000","last_undegraded":"2023-10-18T12:24:14.144196+0000","last_fullsized":"2023-10-18T12:24:14.144196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144453+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:24:14.144453+0000","last_peered":"2023-10-18T12:24:14.144453+0000","last_clean":"2023-10-18T12:24:14.144453+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:24:14.144453+0000","last_undegraded":"2023-10-18T12:24:14.144453+0000","last_fullsized":"2023-10-18T12:24:14.144453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145771+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:24:14.145771+0000","last_peered":"2023-10-18T12:24:14.145771+0000","last_clean":"2023-10-18T12:24:14.145771+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:24:14.145771+0000","last_undegraded":"2023-10-18T12:24:14.145771+0000","last_fullsized":"2023-10-18T12:24:14.145771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138902+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:24:14.138902+0000","last_peered":"2023-10-18T12:24:14.138902+0000","last_clean":"2023-10-18T12:24:14.138902+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:24:14.138902+0000","last_undegraded":"2023-10-18T12:24:14.138902+0000","last_fullsized":"2023-10-18T12:24:14.138902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.145071+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:24:14.145071+0000","last_peered":"2023-10-18T12:24:14.145071+0000","last_clean":"2023-10-18T12:24:14.145071+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:24:14.145071+0000","last_undegraded":"2023-10-18T12:24:14.145071+0000","last_fullsized":"2023-10-18T12:24:14.145071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144958+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:24:14.144958+0000","last_peered":"2023-10-18T12:24:14.144958+0000","last_clean":"2023-10-18T12:24:14.144958+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:24:14.144958+0000","last_undegraded":"2023-10-18T12:24:14.144958+0000","last_fullsized":"2023-10-18T12:24:14.144958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144812+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:24:14.144812+0000","last_peered":"2023-10-18T12:24:14.144812+0000","last_clean":"2023-10-18T12:24:14.144812+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:24:14.144812+0000","last_undegraded":"2023-10-18T12:24:14.144812+0000","last_fullsized":"2023-10-18T12:24:14.144812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.143781+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:24:14.143781+0000","last_peered":"2023-10-18T12:24:14.143781+0000","last_clean":"2023-10-18T12:24:14.143781+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:24:14.143781+0000","last_undegraded":"2023-10-18T12:24:14.143781+0000","last_fullsized":"2023-10-18T12:24:14.143781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.142915+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:24:14.142915+0000","last_peered":"2023-10-18T12:24:14.142915+0000","last_clean":"2023-10-18T12:24:14.142915+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:24:14.142915+0000","last_undegraded":"2023-10-18T12:24:14.142915+0000","last_fullsized":"2023-10-18T12:24:14.142915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.140337+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:24:14.140337+0000","last_peered":"2023-10-18T12:24:14.140337+0000","last_clean":"2023-10-18T12:24:14.140337+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:24:14.140337+0000","last_undegraded":"2023-10-18T12:24:14.140337+0000","last_fullsized":"2023-10-18T12:24:14.140337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.138476+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:24:14.138476+0000","last_peered":"2023-10-18T12:24:14.138476+0000","last_clean":"2023-10-18T12:24:14.138476+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:24:14.138476+0000","last_undegraded":"2023-10-18T12:24:14.138476+0000","last_fullsized":"2023-10-18T12:24:14.138476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T12:24:14.144772+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:24:14.144772+0000","last_peered":"2023-10-18T12:24:14.144772+0000","last_clean":"2023-10-18T12:24:14.144772+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:24:14.144772+0000","last_undegraded":"2023-10-18T12:24:14.144772+0000","last_fullsized":"2023-10-18T12:24:14.144772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22021,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":14,"num_read_kb":30,"num_write":55,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":31301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":489514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1,"5min":1,"15min":1},"last":0.78600000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.873},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.92500000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.89400000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1101299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.443}]},{"osd":2,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.497}]},{"osd":3,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66200000000000003}]},{"osd":9,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73199999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.70699999999999996}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":490096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.64700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.77600000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70699999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.745},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.75600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":490118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.751},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.44800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.436,"15min":0.435},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.54800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47299999999999998,"15min":0.47799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.54600000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.85099999999999998,"15min":0.879},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.4399999999999999,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.629},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.76600000000000001,"15min":0.80300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69399999999999995,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.51300000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71999999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":1.117,"15min":1.117},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66300000000000003,"5min":1.099,"15min":1.099},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.325,"15min":1.325},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91200000000000003,"5min":1.272,"15min":1.272},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84999999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:23:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.80600000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1585152,"data_stored":1080464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.41999999999999998},"min":{"1min":0.314,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.42099999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.62}]},{"osd":3,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.432,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85899999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52000000000000002,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.47599999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.55200000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.70999999999999996},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.56299999999999994,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72699999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:23:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462949,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":489514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.502,"15min":0.502},"min":{"1min":0.33200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.4470000000000001,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58099999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.878}]},{"osd":3,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85499999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.62}]},{"osd":4,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.746,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84099999999999997,"5min":1.028,"15min":1.028},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99299999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:23:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":1.234,"15min":1.234},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75700000000000001,"15min":0.76000000000000001},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.351,"15min":1.351},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91500000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.80300000000000005,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.80000000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:23:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76100000000000001}]},{"osd":10,"last update":"Wed Oct 18 12:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.371,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.439},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.439,"15min":0.41899999999999998},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41899999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:24:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85299999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83399999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.74299999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71099999999999997}]},{"osd":8,"last update":"Wed Oct 18 12:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64300000000000002,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Wed Oct 18 12:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.66200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.432,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79800000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.874}]},{"osd":3,"last update":"Wed Oct 18 12:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.432},"min":{"1min":0.38,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.753},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60999999999999999,"15min":0.58999999999999997},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.505}]},{"osd":5,"last update":"Wed Oct 18 12:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66200000000000003,"15min":0.64400000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.94199999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.434,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.52600000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.52000000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.53800000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.81499999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.59299999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.83999999999999997}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.69199999999999995}]}]},{"osd":2,"up_from":19,"seq":81604378682,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.90000000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.50900000000000001},"min":{"1min":0.41799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82399999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.5},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.5},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79600000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:24:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.437}]},{"osd":6,"last update":"Wed Oct 18 12:23:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88300000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64800000000000002,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.82799999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.58099999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.84099999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":489618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69299999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.60499999999999998},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93200000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.66200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76200000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:24:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83999999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.72599999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.79700000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.626},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":489536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.28599999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.28999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.44700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78500000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:24:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66000000000000003,"15min":0.64500000000000002},"min":{"1min":0.47299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90000000000000002,"5min":1.077,"15min":1.077},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.2529999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.93899999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.82999999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.95699999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1083829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74299999999999999,"5min":0.878,"15min":0.878},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41499999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.36599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40200000000000002,"15min":0.40000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40699999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74099999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.434,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.38900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.47899999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46300000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68400000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.627}]},{"osd":9,"last update":"Wed Oct 18 12:23:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.548,"5min":4.548,"15min":4.548},"last":0.64400000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":21417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:24:39.030 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T12:24:39.031 INFO:tasks.cephadm:Setting up client nodes... 2023-10-18T12:24:39.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-18T12:24:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:39 smithi070 ceph-mon[98639]: mgrmap e21: x(active, since 6m), standbys: z, y 2023-10-18T12:24:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:39 smithi070 ceph-mon[98639]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:24:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:39 smithi070 ceph-mon[98639]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:24:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:39 smithi152 ceph-mon[134767]: mgrmap e21: x(active, since 6m), standbys: z, y 2023-10-18T12:24:39.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:39 smithi152 ceph-mon[134767]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:24:39.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:39 smithi152 ceph-mon[134767]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:24:39.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:39 smithi186 ceph-mon[135253]: mgrmap e21: x(active, since 6m), standbys: z, y 2023-10-18T12:24:39.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:39 smithi186 ceph-mon[135253]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:24:39.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:39 smithi186 ceph-mon[135253]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:24:39.795 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:40.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:40 smithi070 ceph-mon[98639]: pgmap v270: 129 pgs: 129 active+clean; 598 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2023-10-18T12:24:40.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:40 smithi152 ceph-mon[134767]: pgmap v270: 129 pgs: 129 active+clean; 598 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2023-10-18T12:24:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:40 smithi186 ceph-mon[135253]: pgmap v270: 129 pgs: 129 active+clean; 598 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2023-10-18T12:24:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 -- 172.21.15.70:0/3524763160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b800747a0 msgr2=0x7f2b80072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 --2- 172.21.15.70:0/3524763160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b800747a0 0x7f2b80072a20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009990 tx=0x7f2b70009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 -- 172.21.15.70:0/3524763160 shutdown_connections 2023-10-18T12:24:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 --2- 172.21.15.70:0/3524763160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b801114e0 0x7f2b801138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 --2- 172.21.15.70:0/3524763160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b800747a0 0x7f2b80072a20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 --2- 172.21.15.70:0/3524763160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b80072f60 0x7f2b800733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.292+0000 7f2b86524700 1 -- 172.21.15.70:0/3524763160 >> 172.21.15.70:0/3524763160 conn(0x7f2b80077950 msgr2=0x7f2b80077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.293+0000 7f2b86524700 1 -- 172.21.15.70:0/3524763160 shutdown_connections 2023-10-18T12:24:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.293+0000 7f2b86524700 1 -- 172.21.15.70:0/3524763160 wait complete. 2023-10-18T12:24:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 Processor -- start 2023-10-18T12:24:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 -- start start 2023-10-18T12:24:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b80072f60 0x7f2b8019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 0x7f2b8019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b801114e0 0x7f2b801a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801a2130 con 0x7f2b801114e0 2023-10-18T12:24:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801a22a0 con 0x7f2b800747a0 2023-10-18T12:24:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.294+0000 7f2b86524700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801a2460 con 0x7f2b80072f60 2023-10-18T12:24:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 0x7f2b8019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 0x7f2b8019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43354/0 (socket says 172.21.15.70:43354) 2023-10-18T12:24:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 -- 172.21.15.70:0/1945979863 learned_addr learned my addr 172.21.15.70:0/1945979863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b84ac1700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b801114e0 0x7f2b801a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7ffff700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b80072f60 0x7f2b8019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 -- 172.21.15.70:0/1945979863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b80072f60 msgr2=0x7f2b8019d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b80072f60 0x7f2b8019d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 -- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b801114e0 msgr2=0x7f2b801a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b801114e0 0x7f2b801a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 -- 172.21.15.70:0/1945979863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b700095f0 con 0x7f2b800747a0 2023-10-18T12:24:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b84ac1700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b801114e0 0x7f2b801a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7f7fe700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 0x7f2b8019d770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d750 tx=0x7f2b7400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.295+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74004020 con 0x7f2b800747a0 2023-10-18T12:24:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.296+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b74007590 con 0x7f2b800747a0 2023-10-18T12:24:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.296+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b801a2740 con 0x7f2b800747a0 2023-10-18T12:24:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.296+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74005440 con 0x7f2b800747a0 2023-10-18T12:24:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.296+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b801a9fa0 con 0x7f2b800747a0 2023-10-18T12:24:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.297+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b80074050 con 0x7f2b800747a0 2023-10-18T12:24:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.299+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f2b74004a80 con 0x7f2b800747a0 2023-10-18T12:24:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.300+0000 7f2b7d7fa700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b680a1560 0x7f2b680a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.300+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2b740cce80 con 0x7f2b800747a0 2023-10-18T12:24:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.300+0000 7f2b7ffff700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b680a1560 0x7f2b680a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.301+0000 7f2b7ffff700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b680a1560 0x7f2b680a3a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b70006010 tx=0x7f2b7000b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.302+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2b74096050 con 0x7f2b800747a0 2023-10-18T12:24:41.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.580+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2b8004e4c0 con 0x7f2b800747a0 2023-10-18T12:24:41.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.587+0000 7f2b7d7fa700 1 -- 172.21.15.70:0/1945979863 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2b740140f0 con 0x7f2b800747a0 2023-10-18T12:24:41.590 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-18T12:24:41.590 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCJzi9lhKnDIhAAIluPdag5zyuq2WSN7l5yrQ== 2023-10-18T12:24:41.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b680a1560 msgr2=0x7f2b680a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:41.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b680a1560 0x7f2b680a3a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b70006010 tx=0x7f2b7000b540 comp rx=0 tx=0).stop 2023-10-18T12:24:41.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 msgr2=0x7f2b8019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:41.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 0x7f2b8019d770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d750 tx=0x7f2b7400da60 comp rx=0 tx=0).stop 2023-10-18T12:24:41.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 shutdown_connections 2023-10-18T12:24:41.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b801114e0 0x7f2b801a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b80072f60 0x7f2b8019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b680a1560 0x7f2b680a3a20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.589+0000 7f2b86524700 1 --2- 172.21.15.70:0/1945979863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b800747a0 0x7f2b8019d770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:41.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.590+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 >> 172.21.15.70:0/1945979863 conn(0x7f2b80077950 msgr2=0x7f2b8010f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:41.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.590+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 shutdown_connections 2023-10-18T12:24:41.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:41.590+0000 7f2b86524700 1 -- 172.21.15.70:0/1945979863 wait complete. 2023-10-18T12:24:41.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1945979863' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T12:24:41.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:41 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T12:24:41.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:41 smithi152 ceph-mon[134767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-18T12:24:41.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1945979863' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T12:24:41.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:41 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T12:24:41.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:41 smithi186 ceph-mon[135253]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-18T12:24:41.989 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1945979863' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T12:24:41.989 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:41 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T12:24:41.989 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:41 smithi070 ceph-mon[98639]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-18T12:24:41.991 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:24:41.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-18T12:24:41.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T12:24:42.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph config log 1 --format=json 2023-10-18T12:24:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:42.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:42 smithi152 ceph-mon[134767]: pgmap v271: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:42 smithi186 ceph-mon[135253]: pgmap v271: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:43.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:42 smithi070 ceph-mon[98639]: pgmap v271: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:44.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 -- 172.21.15.70:0/4208408790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 msgr2=0x7f3a8810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:44.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/4208408790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a8810d650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b0a0 tx=0x7f3a8000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 -- 172.21.15.70:0/4208408790 shutdown_connections 2023-10-18T12:24:44.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/4208408790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a8810d650 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/4208408790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 0x7f3a88105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/4208408790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 -- 172.21.15.70:0/4208408790 >> 172.21.15.70:0/4208408790 conn(0x7f3a880779e0 msgr2=0x7f3a88077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:44.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 -- 172.21.15.70:0/4208408790 shutdown_connections 2023-10-18T12:24:44.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.400+0000 7f3a8cd25700 1 -- 172.21.15.70:0/4208408790 wait complete. 2023-10-18T12:24:44.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 Processor -- start 2023-10-18T12:24:44.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 -- start start 2023-10-18T12:24:44.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:44.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a88199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:44.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 0x7f3a8819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:44.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819de10 con 0x7f3a88101260 2023-10-18T12:24:44.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819df80 con 0x7f3a881008a0 2023-10-18T12:24:44.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.402+0000 7f3a8cd25700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819e140 con 0x7f3a881057c0 2023-10-18T12:24:44.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a8659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:44.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a88199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:44.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a86d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 0x7f3a8819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:44.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a8659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43384/0 (socket says 172.21.15.70:43384) 2023-10-18T12:24:44.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a8659c700 1 -- 172.21.15.70:0/826139944 learned_addr learned my addr 172.21.15.70:0/826139944 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:44.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 -- 172.21.15.70:0/826139944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 msgr2=0x7f3a8819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:44.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 0x7f3a8819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 -- 172.21.15.70:0/826139944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 msgr2=0x7f3a88198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:44.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 -- 172.21.15.70:0/826139944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009d00 con 0x7f3a88101260 2023-10-18T12:24:44.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a8659c700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:44.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a86d9d700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 0x7f3a8819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:44.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.403+0000 7f3a85d9b700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a88199450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00d750 tx=0x7f3a7c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:44.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.404+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c004020 con 0x7f3a88101260 2023-10-18T12:24:44.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.404+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7c004830 con 0x7f3a88101260 2023-10-18T12:24:44.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.404+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8819e420 con 0x7f3a88101260 2023-10-18T12:24:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.404+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0054c0 con 0x7f3a88101260 2023-10-18T12:24:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.404+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a881a5c80 con 0x7f3a88101260 2023-10-18T12:24:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.405+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a88101f30 con 0x7f3a88101260 2023-10-18T12:24:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.407+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3a7c0049a0 con 0x7f3a88101260 2023-10-18T12:24:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.408+0000 7f3a777fe700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3a700a15e0 0x7f3a700a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.408+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f3a7c0ccaa0 con 0x7f3a88101260 2023-10-18T12:24:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.408+0000 7f3a8659c700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3a700a15e0 0x7f3a700a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.409+0000 7f3a8659c700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3a700a15e0 0x7f3a700a3aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800bfd0 tx=0x7f3a7800b4a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.412+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3a7c095c70 con 0x7f3a88101260 2023-10-18T12:24:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.613+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f3a8804e4c0 con 0x7f3a88101260 2023-10-18T12:24:44.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.613+0000 7f3a777fe700 1 -- 172.21.15.70:0/826139944 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v46) v1 ==== 86+0+133 (secure 0 0 0) 0x7f3a7c099290 con 0x7f3a88101260 2023-10-18T12:24:44.618 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:44.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.615+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3a700a15e0 msgr2=0x7f3a700a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:44.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.615+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3a700a15e0 0x7f3a700a3aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800bfd0 tx=0x7f3a7800b4a0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.615+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 msgr2=0x7f3a88199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.615+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a88199450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00d750 tx=0x7f3a7c00da60 comp rx=0 tx=0).stop 2023-10-18T12:24:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 shutdown_connections 2023-10-18T12:24:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a881057c0 0x7f3a8819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3a700a15e0 0x7f3a700a3aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a881008a0 0x7f3a88198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 --2- 172.21.15.70:0/826139944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a88101260 0x7f3a88199450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 >> 172.21.15.70:0/826139944 conn(0x7f3a880779e0 msgr2=0x7f3a8810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 shutdown_connections 2023-10-18T12:24:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:44.616+0000 7f3a8cd25700 1 -- 172.21.15.70:0/826139944 wait complete. 2023-10-18T12:24:45.060 INFO:teuthology.orchestra.run.smithi070.stdout:[{"version":46,"timestamp":"2023-10-18T12:24:03.229385+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-10-18T12:24:45.061 INFO:tasks.ceph_manager:config epoch is 46 2023-10-18T12:24:45.061 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T12:24:45.062 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-18T12:24:45.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-18T12:24:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:45 smithi070 ceph-mon[98639]: pgmap v272: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:45.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:45 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/826139944' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-18T12:24:45.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:45 smithi152 ceph-mon[134767]: pgmap v272: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:45.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:45 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/826139944' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-18T12:24:45.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:45 smithi186 ceph-mon[135253]: pgmap v272: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:45.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:45 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/826139944' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-18T12:24:45.793 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:47.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:47 smithi152 ceph-mon[134767]: pgmap v273: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/1110028193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4072a20 msgr2=0x7fd1e4072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/1110028193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4072a20 0x7fd1e4072e00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40098f0 tx=0x7fd1d4009c00 comp rx=0 tx=0).stop 2023-10-18T12:24:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/1110028193 shutdown_connections 2023-10-18T12:24:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/1110028193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e410d250 0x7fd1e410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/1110028193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4072a20 0x7fd1e4072e00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/1110028193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1e4073340 0x7fd1e410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/1110028193 >> 172.21.15.70:0/1110028193 conn(0x7fd1e40779e0 msgr2=0x7fd1e4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/1110028193 shutdown_connections 2023-10-18T12:24:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.418+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/1110028193 wait complete. 2023-10-18T12:24:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.419+0000 7fd1e8f1e700 1 Processor -- start 2023-10-18T12:24:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.419+0000 7fd1e8f1e700 1 -- start start 2023-10-18T12:24:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e8f1e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 0x7fd1e4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e8f1e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4073340 0x7fd1e4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e8f1e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1e410d250 0x7fd1e419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e8f1e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419ded0 con 0x7fd1e4072a20 2023-10-18T12:24:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e8f1e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419e040 con 0x7fd1e410d250 2023-10-18T12:24:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e8f1e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419e200 con 0x7fd1e4073340 2023-10-18T12:24:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 0x7fd1e4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:47.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 0x7fd1e4198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44584/0 (socket says 172.21.15.70:44584) 2023-10-18T12:24:47.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 -- 172.21.15.70:0/226939601 learned_addr learned my addr 172.21.15.70:0/226939601 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:47.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e1d9b700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4073340 0x7fd1e4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:47.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e2d9d700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1e410d250 0x7fd1e419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 -- 172.21.15.70:0/226939601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4073340 msgr2=0x7fd1e4199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4073340 0x7fd1e4199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 -- 172.21.15.70:0/226939601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1e410d250 msgr2=0x7fd1e419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1e410d250 0x7fd1e419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.420+0000 7fd1e259c700 1 -- 172.21.15.70:0/226939601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d80095f0 con 0x7fd1e4072a20 2023-10-18T12:24:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.421+0000 7fd1e259c700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 0x7fd1e4198fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400fa00 tx=0x7fd1d400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.421+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d401c070 con 0x7fd1e4072a20 2023-10-18T12:24:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.421+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d4005040 con 0x7fd1e4072a20 2023-10-18T12:24:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.421+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc009c40 con 0x7fd1e4072a20 2023-10-18T12:24:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.421+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4004a90 con 0x7fd1e4072a20 2023-10-18T12:24:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.421+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d4009520 con 0x7fd1e4072a20 2023-10-18T12:24:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.422+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e4108460 con 0x7fd1e4072a20 2023-10-18T12:24:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.424+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd1d40051b0 con 0x7fd1e4072a20 2023-10-18T12:24:47.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.425+0000 7fd1d37fe700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1cc0a15e0 0x7fd1cc0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:47.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.425+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fd1d40ce2e0 con 0x7fd1e4072a20 2023-10-18T12:24:47.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.425+0000 7fd1e1d9b700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1cc0a15e0 0x7fd1cc0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:47.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.426+0000 7fd1e1d9b700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1cc0a15e0 0x7fd1cc0a3aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e419a530 tx=0x7fd1d8009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.429+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd1d4097540 con 0x7fd1e4072a20 2023-10-18T12:24:47.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:47 smithi186 ceph-mon[135253]: pgmap v273: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:47.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:47 smithi070 ceph-mon[98639]: pgmap v273: 129 pgs: 129 active+clean; 620 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2023-10-18T12:24:47.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.685+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd1e404e4c0 con 0x7fd1e4072a20 2023-10-18T12:24:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.691+0000 7fd1d37fe700 1 -- 172.21.15.70:0/226939601 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) v1 ==== 74+0+260420 (secure 0 0 0) 0x7fd1d409ab60 con 0x7fd1e4072a20 2023-10-18T12:24:47.698 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1cc0a15e0 msgr2=0x7fd1cc0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1cc0a15e0 0x7fd1cc0a3aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e419a530 tx=0x7fd1d8009310 comp rx=0 tx=0).stop 2023-10-18T12:24:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 msgr2=0x7fd1e4198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 0x7fd1e4198fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400fa00 tx=0x7fd1d400fa30 comp rx=0 tx=0).stop 2023-10-18T12:24:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 shutdown_connections 2023-10-18T12:24:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1cc0a15e0 0x7fd1cc0a3aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1e410d250 0x7fd1e419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1e4072a20 0x7fd1e4198fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 --2- 172.21.15.70:0/226939601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e4073340 0x7fd1e4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 >> 172.21.15.70:0/226939601 conn(0x7fd1e40779e0 msgr2=0x7fd1e40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 shutdown_connections 2023-10-18T12:24:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:47.695+0000 7fd1e8f1e700 1 -- 172.21.15.70:0/226939601 wait complete. 2023-10-18T12:24:48.187 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":21,"active_gid":14214,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":223035468},{"type":"v1","addr":"172.21.15.70:6801","nonce":223035468}]},"active_addr":"172.21.15.70:6801/223035468","active_change":"2023-10-18T12:18:13.515456+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24145,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24167,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.70:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2279769141}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":808797426}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2387190081}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3565418172}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2273592403}]}]} 2023-10-18T12:24:48.194 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-18T12:24:48.194 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-18T12:24:48.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-18T12:24:48.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:48 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/226939601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-18T12:24:48.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:48 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/226939601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-18T12:24:48.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:48 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/226939601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-18T12:24:48.931 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:49.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:49 smithi152 ceph-mon[134767]: pgmap v274: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2023-10-18T12:24:49.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:49 smithi186 ceph-mon[135253]: pgmap v274: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2023-10-18T12:24:49.520 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:49 smithi070 ceph-mon[98639]: pgmap v274: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2023-10-18T12:24:50.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 -- 172.21.15.70:0/3464080265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 msgr2=0x7f909010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:50.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3464080265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f909010d650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f908800b0a0 tx=0x7f908800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 -- 172.21.15.70:0/3464080265 shutdown_connections 2023-10-18T12:24:50.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3464080265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f909010d650 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3464080265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90901057c0 0x7f9090105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3464080265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 0x7f9090100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 -- 172.21.15.70:0/3464080265 >> 172.21.15.70:0/3464080265 conn(0x7f90900779e0 msgr2=0x7f9090077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:50.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 -- 172.21.15.70:0/3464080265 shutdown_connections 2023-10-18T12:24:50.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.424+0000 7f9094e0e700 1 -- 172.21.15.70:0/3464080265 wait complete. 2023-10-18T12:24:50.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.425+0000 7f9094e0e700 1 Processor -- start 2023-10-18T12:24:50.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.425+0000 7f9094e0e700 1 -- start start 2023-10-18T12:24:50.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f9094e0e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 0x7f9090198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:50.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f9094e0e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f9090199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:50.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f9094e0e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90901057c0 0x7f909019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:50.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f9094e0e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019ddf0 con 0x7f9090101260 2023-10-18T12:24:50.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f9094e0e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019df60 con 0x7f90901008a0 2023-10-18T12:24:50.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f9094e0e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019e120 con 0x7f90901057c0 2023-10-18T12:24:50.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f9090199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:50.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 0x7f9090198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:50.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f9090199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56126/0 (socket says 172.21.15.70:56126) 2023-10-18T12:24:50.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908dd9b700 1 -- 172.21.15.70:0/3602440794 learned_addr learned my addr 172.21.15.70:0/3602440794 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:50.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908ed9d700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90901057c0 0x7f909019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:50.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908e59c700 1 -- 172.21.15.70:0/3602440794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90901057c0 msgr2=0x7f909019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908e59c700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90901057c0 0x7f909019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908e59c700 1 -- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 msgr2=0x7f9090199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908e59c700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f9090199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908e59c700 1 -- 172.21.15.70:0/3602440794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9088009d00 con 0x7f90901008a0 2023-10-18T12:24:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.426+0000 7f908dd9b700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f9090199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:24:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.427+0000 7f908e59c700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 0x7f9090198f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f908000d740 tx=0x7f908000da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.427+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080004220 con 0x7f90901008a0 2023-10-18T12:24:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.427+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909019e400 con 0x7f90901008a0 2023-10-18T12:24:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.427+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9080004380 con 0x7f90901008a0 2023-10-18T12:24:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.427+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080003c60 con 0x7f90901008a0 2023-10-18T12:24:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.427+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901a5c60 con 0x7f90901008a0 2023-10-18T12:24:50.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.428+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9090101f30 con 0x7f90901008a0 2023-10-18T12:24:50.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.430+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9080044780 con 0x7f90901008a0 2023-10-18T12:24:50.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.431+0000 7f907f7fe700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90780a1560 0x7f90780a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:50.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.432+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f90800ccb50 con 0x7f90901008a0 2023-10-18T12:24:50.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.432+0000 7f908dd9b700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90780a1560 0x7f90780a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:50.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.432+0000 7f908dd9b700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90780a1560 0x7f90780a3a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f909019a450 tx=0x7f9084009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.434+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9080095d20 con 0x7f90901008a0 2023-10-18T12:24:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.635+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f909004e4c0 con 0x7f90901008a0 2023-10-18T12:24:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.637+0000 7f907f7fe700 1 -- 172.21.15.70:0/3602440794 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f9080099340 con 0x7f90901008a0 2023-10-18T12:24:50.640 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:50.640 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90780a1560 msgr2=0x7f90780a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90780a1560 0x7f90780a3a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f909019a450 tx=0x7f9084009260 comp rx=0 tx=0).stop 2023-10-18T12:24:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 msgr2=0x7f9090198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 0x7f9090198f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f908000d740 tx=0x7f908000da50 comp rx=0 tx=0).stop 2023-10-18T12:24:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 shutdown_connections 2023-10-18T12:24:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90901057c0 0x7f909019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90780a1560 0x7f90780a3a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90901008a0 0x7f9090198f40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 --2- 172.21.15.70:0/3602440794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9090101260 0x7f9090199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 >> 172.21.15.70:0/3602440794 conn(0x7f90900779e0 msgr2=0x7f90900fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 shutdown_connections 2023-10-18T12:24:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:50.639+0000 7f9094e0e700 1 -- 172.21.15.70:0/3602440794 wait complete. 2023-10-18T12:24:51.027 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-18T12:24:51.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-18T12:24:51.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:51 smithi070 ceph-mon[98639]: pgmap v275: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 3 op/s 2023-10-18T12:24:51.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:51 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3602440794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:51.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:51 smithi152 ceph-mon[134767]: pgmap v275: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 3 op/s 2023-10-18T12:24:51.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:51 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3602440794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:51 smithi186 ceph-mon[135253]: pgmap v275: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 3 op/s 2023-10-18T12:24:51.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:51 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3602440794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:51.766 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.248+0000 7f7315abf700 1 -- 172.21.15.70:0/1754343144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 msgr2=0x7f7310100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.248+0000 7f7315abf700 1 --2- 172.21.15.70:0/1754343144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310100d20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7304009990 tx=0x7f7304009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 -- 172.21.15.70:0/1754343144 shutdown_connections 2023-10-18T12:24:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 --2- 172.21.15.70:0/1754343144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7310101260 0x7f731010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 --2- 172.21.15.70:0/1754343144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73101057c0 0x7f7310105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 --2- 172.21.15.70:0/1754343144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310100d20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 -- 172.21.15.70:0/1754343144 >> 172.21.15.70:0/1754343144 conn(0x7f73100779e0 msgr2=0x7f7310077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:53.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 -- 172.21.15.70:0/1754343144 shutdown_connections 2023-10-18T12:24:53.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.249+0000 7f7315abf700 1 -- 172.21.15.70:0/1754343144 wait complete. 2023-10-18T12:24:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 Processor -- start 2023-10-18T12:24:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 -- start start 2023-10-18T12:24:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7310101260 0x7f7310073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73101057c0 0x7f7310073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101a4f90 con 0x7f73101008a0 2023-10-18T12:24:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101a5100 con 0x7f73101057c0 2023-10-18T12:24:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f7315abf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101a52c0 con 0x7f7310101260 2023-10-18T12:24:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.250+0000 7f730f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56144/0 (socket says 172.21.15.70:56144) 2023-10-18T12:24:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 -- 172.21.15.70:0/2783034892 learned_addr learned my addr 172.21.15.70:0/2783034892 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730ffff700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73101057c0 0x7f7310073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730effd700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7310101260 0x7f7310073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 -- 172.21.15.70:0/2783034892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7310101260 msgr2=0x7f7310073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7310101260 0x7f7310073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 -- 172.21.15.70:0/2783034892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73101057c0 msgr2=0x7f7310073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73101057c0 0x7f7310073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 -- 172.21.15.70:0/2783034892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040095f0 con 0x7f73101008a0 2023-10-18T12:24:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730f7fe700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310072af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f730000e970 tx=0x7f730000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300009790 con 0x7f73101008a0 2023-10-18T12:24:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.251+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f730000c7e0 con 0x7f73101008a0 2023-10-18T12:24:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.252+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73101a55a0 con 0x7f73101008a0 2023-10-18T12:24:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.252+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300003c70 con 0x7f73101008a0 2023-10-18T12:24:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.252+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73101a5aa0 con 0x7f73101008a0 2023-10-18T12:24:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.253+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7310108da0 con 0x7f73101008a0 2023-10-18T12:24:53.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.255+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f730000cc50 con 0x7f73101008a0 2023-10-18T12:24:53.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.256+0000 7f730cff9700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f72f80a1560 0x7f72f80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:53.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.256+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f7300014070 con 0x7f73101008a0 2023-10-18T12:24:53.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.256+0000 7f730effd700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f72f80a1560 0x7f72f80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:53.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.257+0000 7f730effd700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f72f80a1560 0x7f72f80a3a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73100741d0 tx=0x7f7304005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:53.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.258+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f73000966b0 con 0x7f73101008a0 2023-10-18T12:24:53.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:53 smithi070 ceph-mon[98639]: pgmap v276: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2023-10-18T12:24:53.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:53 smithi152 ceph-mon[134767]: pgmap v276: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2023-10-18T12:24:53.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.458+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7310073f80 con 0x7f73101008a0 2023-10-18T12:24:53.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.460+0000 7f730cff9700 1 -- 172.21.15.70:0/2783034892 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24286 (secure 0 0 0) 0x7f7300099cd0 con 0x7f73101008a0 2023-10-18T12:24:53.462 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:24:53.462 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:24:14.634206+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:24:53.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f72f80a1560 msgr2=0x7f72f80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:53.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f72f80a1560 0x7f72f80a3a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73100741d0 tx=0x7f7304005fd0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 msgr2=0x7f7310072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:53.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310072af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f730000e970 tx=0x7f730000ec80 comp rx=0 tx=0).stop 2023-10-18T12:24:53.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 shutdown_connections 2023-10-18T12:24:53.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f72f80a1560 0x7f72f80a3a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73101057c0 0x7f7310073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73101008a0 0x7f7310072af0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 --2- 172.21.15.70:0/2783034892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7310101260 0x7f7310073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:53.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 >> 172.21.15.70:0/2783034892 conn(0x7f73100779e0 msgr2=0x7f73100fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:53.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 shutdown_connections 2023-10-18T12:24:53.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:53.462+0000 7f7315abf700 1 -- 172.21.15.70:0/2783034892 wait complete. 2023-10-18T12:24:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:53 smithi186 ceph-mon[135253]: pgmap v276: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2023-10-18T12:24:53.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-18T12:24:53.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-18T12:24:53.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-18T12:24:53.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-18T12:24:53.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-18T12:24:53.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-18T12:24:53.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-18T12:24:53.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-18T12:24:53.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-18T12:24:53.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-18T12:24:53.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-18T12:24:53.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-18T12:24:54.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:54 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2783034892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:54.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:54 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2783034892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:54.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:54 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2783034892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:24:55.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:55 smithi152 ceph-mon[134767]: pgmap v277: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-10-18T12:24:55.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:55 smithi186 ceph-mon[135253]: pgmap v277: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-10-18T12:24:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:55 smithi070 ceph-mon[98639]: pgmap v277: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-10-18T12:24:55.637 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:55.723 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:55.812 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:56.021 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:56.871 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:57.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:57 smithi070 ceph-mon[98639]: pgmap v278: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-10-18T12:24:57.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:57 smithi152 ceph-mon[134767]: pgmap v278: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-10-18T12:24:57.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:57 smithi186 ceph-mon[135253]: pgmap v278: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-10-18T12:24:57.469 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:58.151 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:58.152 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:58.161 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:24:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 -- 172.21.15.70:0/3064710939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1008a0 msgr2=0x7f092c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 --2- 172.21.15.70:0/3064710939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1008a0 0x7f092c100d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0920009990 tx=0x7f0920009ca0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 -- 172.21.15.70:0/3064710939 shutdown_connections 2023-10-18T12:24:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 --2- 172.21.15.70:0/3064710939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f092c101260 0x7f092c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 --2- 172.21.15.70:0/3064710939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092c1057c0 0x7f092c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 --2- 172.21.15.70:0/3064710939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1008a0 0x7f092c100d20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 -- 172.21.15.70:0/3064710939 >> 172.21.15.70:0/3064710939 conn(0x7f092c0779e0 msgr2=0x7f092c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.343+0000 7f093129e700 1 -- 172.21.15.70:0/3064710939 shutdown_connections 2023-10-18T12:24:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.344+0000 7f093129e700 1 -- 172.21.15.70:0/3064710939 wait complete. 2023-10-18T12:24:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.344+0000 7f093129e700 1 Processor -- start 2023-10-18T12:24:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 -- start start 2023-10-18T12:24:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092c1008a0 0x7f092c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f092c101260 0x7f092c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1057c0 0x7f092c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c19dd30 con 0x7f092c1057c0 2023-10-18T12:24:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c19dea0 con 0x7f092c101260 2023-10-18T12:24:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f093129e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c19e060 con 0x7f092c1008a0 2023-10-18T12:24:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1057c0 0x7f092c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1057c0 0x7f092c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33000/0 (socket says 172.21.15.70:33000) 2023-10-18T12:24:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f092b7fe700 1 -- 172.21.15.70:0/1240054669 learned_addr learned my addr 172.21.15.70:0/1240054669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:24:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f092affd700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092c1008a0 0x7f092c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.345+0000 7f092a7fc700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f092c101260 0x7f092c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f092b7fe700 1 -- 172.21.15.70:0/1240054669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092c1008a0 msgr2=0x7f092c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f092b7fe700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092c1008a0 0x7f092c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f092b7fe700 1 -- 172.21.15.70:0/1240054669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f092c101260 msgr2=0x7f092c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f092b7fe700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f092c101260 0x7f092c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f092b7fe700 1 -- 172.21.15.70:0/1240054669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200095f0 con 0x7f092c1057c0 2023-10-18T12:24:59.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f092b7fe700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1057c0 0x7f092c19d720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f092400efc0 tx=0x7f092400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:59.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924018070 con 0x7f092c1057c0 2023-10-18T12:24:59.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f092400f040 con 0x7f092c1057c0 2023-10-18T12:24:59.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400cc10 con 0x7f092c1057c0 2023-10-18T12:24:59.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.346+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c19e2e0 con 0x7f092c1057c0 2023-10-18T12:24:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.347+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c1a5ba0 con 0x7f092c1057c0 2023-10-18T12:24:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.347+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f092c110a60 con 0x7f092c1057c0 2023-10-18T12:24:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.349+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f092400cd90 con 0x7f092c1057c0 2023-10-18T12:24:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f0913fff700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09140a15e0 0x7f09140a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f09240ccb10 con 0x7f092c1057c0 2023-10-18T12:24:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f092affd700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09140a15e0 0x7f09140a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f0913fff700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] conn(0x7f09140ac270 0x7f09140ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:24:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 --> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09140aed80 con 0x7f09140ac270 2023-10-18T12:24:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09240d12a0 con 0x7f092c1057c0 2023-10-18T12:24:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.351+0000 7f092a7fc700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] conn(0x7f09140ac270 0x7f09140ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:24:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.352+0000 7f092affd700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09140a15e0 0x7f09140a3aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f092c101930 tx=0x7f091c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.352+0000 7f092a7fc700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] conn(0x7f09140ac270 0x7f09140ae6b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:24:59.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:24:59 smithi070 ceph-mon[98639]: pgmap v279: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-10-18T12:24:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.353+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== osd.0 v2:172.21.15.70:6802/2979990222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f09140aed80 con 0x7f09140ac270 2023-10-18T12:24:59.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.369+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 --> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f092c101f30 con 0x7f09140ac270 2023-10-18T12:24:59.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f0913fff700 1 -- 172.21.15.70:0/1240054669 <== osd.0 v2:172.21.15.70:6802/2979990222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f092c101f30 con 0x7f09140ac270 2023-10-18T12:24:59.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] conn(0x7f09140ac270 msgr2=0x7f09140ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:59.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] conn(0x7f09140ac270 0x7f09140ae6b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09140a15e0 msgr2=0x7f09140a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:59.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09140a15e0 0x7f09140a3aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f092c101930 tx=0x7f091c009260 comp rx=0 tx=0).stop 2023-10-18T12:24:59.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1057c0 msgr2=0x7f092c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:24:59.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 --2- 172.21.15.70:0/1240054669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092c1057c0 0x7f092c19d720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f092400efc0 tx=0x7f092400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:24:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f092b7fe700 1 -- 172.21.15.70:0/1240054669 reap_dead start 2023-10-18T12:24:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 shutdown_connections 2023-10-18T12:24:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 >> 172.21.15.70:0/1240054669 conn(0x7f092c0779e0 msgr2=0x7f092c10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:24:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.370+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 shutdown_connections 2023-10-18T12:24:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:24:59.371+0000 7f093129e700 1 -- 172.21.15.70:0/1240054669 wait complete. 2023-10-18T12:24:59.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:24:59 smithi186 ceph-mon[135253]: pgmap v279: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-10-18T12:24:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:24:59 smithi152 ceph-mon[134767]: pgmap v279: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-10-18T12:25:01.210 INFO:teuthology.orchestra.run.smithi070.stdout:38654705734 2023-10-18T12:25:01.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-18T12:25:01.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:01 smithi070 ceph-mon[98639]: pgmap v280: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:01 smithi186 ceph-mon[135253]: pgmap v280: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:01 smithi152 ceph-mon[134767]: pgmap v280: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.932+0000 7f1c94810700 1 -- 172.21.15.70:0/967235668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068190 msgr2=0x7f1c90068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.932+0000 7f1c94810700 1 --2- 172.21.15.70:0/967235668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068190 0x7f1c90068570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1c800098f0 tx=0x7f1c80009c00 comp rx=0 tx=0).stop 2023-10-18T12:25:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 -- 172.21.15.70:0/967235668 shutdown_connections 2023-10-18T12:25:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 --2- 172.21.15.70:0/967235668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c9010d300 0x7f1c9010f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 --2- 172.21.15.70:0/967235668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068190 0x7f1c90068570 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 --2- 172.21.15.70:0/967235668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c90068ab0 0x7f1c9010cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 -- 172.21.15.70:0/967235668 >> 172.21.15.70:0/967235668 conn(0x7f1c90075790 msgr2=0x7f1c90075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 -- 172.21.15.70:0/967235668 shutdown_connections 2023-10-18T12:25:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.933+0000 7f1c94810700 1 -- 172.21.15.70:0/967235668 wait complete. 2023-10-18T12:25:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.934+0000 7f1c94810700 1 Processor -- start 2023-10-18T12:25:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.934+0000 7f1c94810700 1 -- start start 2023-10-18T12:25:02.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.934+0000 7f1c94810700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c90068190 0x7f1c90198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c94810700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068ab0 0x7f1c90199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c94810700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9010d300 0x7f1c9019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c94810700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9019dd20 con 0x7f1c90068ab0 2023-10-18T12:25:02.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c94810700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9019de90 con 0x7f1c9010d300 2023-10-18T12:25:02.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c94810700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9019e050 con 0x7f1c90068190 2023-10-18T12:25:02.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068ab0 0x7f1c90199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068ab0 0x7f1c90199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33046/0 (socket says 172.21.15.70:33046) 2023-10-18T12:25:02.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8dd9b700 1 -- 172.21.15.70:0/528301704 learned_addr learned my addr 172.21.15.70:0/528301704 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:02.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8e59c700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c90068190 0x7f1c90198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8ed9d700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9010d300 0x7f1c9019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8e59c700 1 -- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9010d300 msgr2=0x7f1c9019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8e59c700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9010d300 0x7f1c9019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8e59c700 1 -- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068ab0 msgr2=0x7f1c90199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8e59c700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c90068ab0 0x7f1c90199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.935+0000 7f1c8e59c700 1 -- 172.21.15.70:0/528301704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c80009520 con 0x7f1c90068190 2023-10-18T12:25:02.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.936+0000 7f1c8e59c700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c90068190 0x7f1c90198e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009230 tx=0x7f1c8000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.936+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001c070 con 0x7f1c90068190 2023-10-18T12:25:02.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.936+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8000ba20 con 0x7f1c90068190 2023-10-18T12:25:02.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.936+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9019e2d0 con 0x7f1c90068190 2023-10-18T12:25:02.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.936+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800038e0 con 0x7f1c90068190 2023-10-18T12:25:02.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.936+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c901a5b90 con 0x7f1c90068190 2023-10-18T12:25:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.937+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c90112bf0 con 0x7f1c90068190 2023-10-18T12:25:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.939+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1c80020030 con 0x7f1c90068190 2023-10-18T12:25:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c7f7fe700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c780a1560 0x7f1c780a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f1c800ce180 con 0x7f1c90068190 2023-10-18T12:25:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c8dd9b700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c780a1560 0x7f1c780a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c7f7fe700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] conn(0x7f1c780ac270 0x7f1c780ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 --> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c780aed80 con 0x7f1c780ac270 2023-10-18T12:25:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c800ce4f0 con 0x7f1c90068190 2023-10-18T12:25:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.941+0000 7f1c8dd9b700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c780a1560 0x7f1c780a3a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1c9019a380 tx=0x7f1c84009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.942+0000 7f1c8ed9d700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] conn(0x7f1c780ac270 0x7f1c780ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.942+0000 7f1c8ed9d700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] conn(0x7f1c780ac270 0x7f1c780ae6b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.943+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== osd.1 v2:172.21.15.152:6800/1167081028 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1c780aed80 con 0x7f1c780ac270 2023-10-18T12:25:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.968+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 --> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c9019e5a0 con 0x7f1c780ac270 2023-10-18T12:25:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.969+0000 7f1c7f7fe700 1 -- 172.21.15.70:0/528301704 <== osd.1 v2:172.21.15.152:6800/1167081028 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c8800f040 con 0x7f1c780ac270 2023-10-18T12:25:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] conn(0x7f1c780ac270 msgr2=0x7f1c780ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] conn(0x7f1c780ac270 0x7f1c780ae6b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c780a1560 msgr2=0x7f1c780a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c780a1560 0x7f1c780a3a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1c9019a380 tx=0x7f1c84009310 comp rx=0 tx=0).stop 2023-10-18T12:25:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c90068190 msgr2=0x7f1c90198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 --2- 172.21.15.70:0/528301704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c90068190 0x7f1c90198e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009230 tx=0x7f1c8000fa00 comp rx=0 tx=0).stop 2023-10-18T12:25:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c8ed9d700 1 -- 172.21.15.70:0/528301704 reap_dead start 2023-10-18T12:25:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 shutdown_connections 2023-10-18T12:25:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 >> 172.21.15.70:0/528301704 conn(0x7f1c90075790 msgr2=0x7f1c9010ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 shutdown_connections 2023-10-18T12:25:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.970+0000 7f1c94810700 1 -- 172.21.15.70:0/528301704 wait complete. 2023-10-18T12:25:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 -- 172.21.15.70:0/99580958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01057c0 msgr2=0x7f3db0105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 --2- 172.21.15.70:0/99580958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01057c0 0x7f3db0105ba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009990 tx=0x7f3da0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 -- 172.21.15.70:0/99580958 shutdown_connections 2023-10-18T12:25:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 --2- 172.21.15.70:0/99580958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db0101260 0x7f3db010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 --2- 172.21.15.70:0/99580958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01057c0 0x7f3db0105ba0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 --2- 172.21.15.70:0/99580958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3db01008a0 0x7f3db0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 -- 172.21.15.70:0/99580958 >> 172.21.15.70:0/99580958 conn(0x7f3db00779e0 msgr2=0x7f3db0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 -- 172.21.15.70:0/99580958 shutdown_connections 2023-10-18T12:25:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.259+0000 7f3db518d700 1 -- 172.21.15.70:0/99580958 wait complete. 2023-10-18T12:25:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.260+0000 7f3db518d700 1 Processor -- start 2023-10-18T12:25:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.260+0000 7f3db518d700 1 -- start start 2023-10-18T12:25:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3db518d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01008a0 0x7f3db0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3db518d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db0101260 0x7f3db0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3db518d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3db01057c0 0x7f3db0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3db518d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db01a4fe0 con 0x7f3db01008a0 2023-10-18T12:25:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3db518d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db01a5150 con 0x7f3db0101260 2023-10-18T12:25:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3db518d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db01a5310 con 0x7f3db01057c0 2023-10-18T12:25:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01008a0 0x7f3db0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01008a0 0x7f3db0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33064/0 (socket says 172.21.15.70:33064) 2023-10-18T12:25:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 -- 172.21.15.70:0/2357104071 learned_addr learned my addr 172.21.15.70:0/2357104071 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daf59e700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3db01057c0 0x7f3db0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3dae59c700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db0101260 0x7f3db0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 -- 172.21.15.70:0/2357104071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3db01057c0 msgr2=0x7f3db0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3db01057c0 0x7f3db0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 -- 172.21.15.70:0/2357104071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db0101260 msgr2=0x7f3db0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db0101260 0x7f3db0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.261+0000 7f3daed9d700 1 -- 172.21.15.70:0/2357104071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00095f0 con 0x7f3db01008a0 2023-10-18T12:25:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.262+0000 7f3daed9d700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01008a0 0x7f3db0072af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3da0000c00 tx=0x7f3da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.262+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da001d070 con 0x7f3db01008a0 2023-10-18T12:25:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.262+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3da0022470 con 0x7f3db01008a0 2023-10-18T12:25:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.262+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db01a5590 con 0x7f3db01008a0 2023-10-18T12:25:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.262+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0017650 con 0x7f3db01008a0 2023-10-18T12:25:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.262+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db01a5a00 con 0x7f3db01008a0 2023-10-18T12:25:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.263+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3db01109d0 con 0x7f3db01008a0 2023-10-18T12:25:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.265+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3da00225e0 con 0x7f3db01008a0 2023-10-18T12:25:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.266+0000 7f3d97fff700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3d980a15e0 0x7f3d980a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.266+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f3da00ceb10 con 0x7f3db01008a0 2023-10-18T12:25:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.266+0000 7f3dae59c700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3d980a15e0 0x7f3d980a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.266+0000 7f3d97fff700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] conn(0x7f3d980ac2f0 0x7f3d980ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.266+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 --> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d980aee00 con 0x7f3d980ac2f0 2023-10-18T12:25:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.267+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da00cee80 con 0x7f3db01008a0 2023-10-18T12:25:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.267+0000 7f3daf59e700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] conn(0x7f3d980ac2f0 0x7f3d980ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.267+0000 7f3dae59c700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3d980a15e0 0x7f3d980a3aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3db00741d0 tx=0x7f3da4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.267+0000 7f3daf59e700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] conn(0x7f3d980ac2f0 0x7f3d980ae730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.268+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== osd.6 v2:172.21.15.70:6818/3765589692 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f3d980aee00 con 0x7f3d980ac2f0 2023-10-18T12:25:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.285+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 --> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3db0101f30 con 0x7f3d980ac2f0 2023-10-18T12:25:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.285+0000 7f3d97fff700 1 -- 172.21.15.70:0/2357104071 <== osd.6 v2:172.21.15.70:6818/3765589692 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3db0101f30 con 0x7f3d980ac2f0 2023-10-18T12:25:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.285+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] conn(0x7f3d980ac2f0 msgr2=0x7f3d980ae730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.285+0000 7f3db518d700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] conn(0x7f3d980ac2f0 0x7f3d980ae730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3d980a15e0 msgr2=0x7f3d980a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3d980a15e0 0x7f3d980a3aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3db00741d0 tx=0x7f3da4009260 comp rx=0 tx=0).stop 2023-10-18T12:25:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01008a0 msgr2=0x7f3db0072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 --2- 172.21.15.70:0/2357104071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3db01008a0 0x7f3db0072af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3da0000c00 tx=0x7f3da0004730 comp rx=0 tx=0).stop 2023-10-18T12:25:02.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3daf59e700 1 -- 172.21.15.70:0/2357104071 reap_dead start 2023-10-18T12:25:02.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 shutdown_connections 2023-10-18T12:25:02.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 >> 172.21.15.70:0/2357104071 conn(0x7f3db00779e0 msgr2=0x7f3db00fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 shutdown_connections 2023-10-18T12:25:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.286+0000 7f3db518d700 1 -- 172.21.15.70:0/2357104071 wait complete. 2023-10-18T12:25:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 -- 172.21.15.70:0/3151230708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 msgr2=0x7f362c072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 --2- 172.21.15.70:0/3151230708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c072a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f361c009990 tx=0x7f361c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 -- 172.21.15.70:0/3151230708 shutdown_connections 2023-10-18T12:25:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 --2- 172.21.15.70:0/3151230708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c111570 0x7f362c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 --2- 172.21.15.70:0/3151230708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c072a20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 --2- 172.21.15.70:0/3151230708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f362c072f60 0x7f362c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.845+0000 7f3631ab0700 1 -- 172.21.15.70:0/3151230708 >> 172.21.15.70:0/3151230708 conn(0x7f362c0779e0 msgr2=0x7f362c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.846+0000 7f3631ab0700 1 -- 172.21.15.70:0/3151230708 shutdown_connections 2023-10-18T12:25:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.846+0000 7f3631ab0700 1 -- 172.21.15.70:0/3151230708 wait complete. 2023-10-18T12:25:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 Processor -- start 2023-10-18T12:25:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 -- start start 2023-10-18T12:25:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f362c072f60 0x7f362c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c111570 0x7f362c1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1a2270 con 0x7f362c074770 2023-10-18T12:25:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1a23e0 con 0x7f362c111570 2023-10-18T12:25:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.847+0000 7f3631ab0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1a25a0 con 0x7f362c072f60 2023-10-18T12:25:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33078/0 (socket says 172.21.15.70:33078) 2023-10-18T12:25:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 -- 172.21.15.70:0/873599316 learned_addr learned my addr 172.21.15.70:0/873599316 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362b7fe700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f362c072f60 0x7f362c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362bfff700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c111570 0x7f362c1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 -- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f362c072f60 msgr2=0x7f362c19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f362c072f60 0x7f362c19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 -- 172.21.15.70:0/873599316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c111570 msgr2=0x7f362c1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c111570 0x7f362c1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 -- 172.21.15.70:0/873599316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0095f0 con 0x7f362c074770 2023-10-18T12:25:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.848+0000 7f362affd700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c19d8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f362000e9a0 tx=0x7f362000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.849+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620009790 con 0x7f362c074770 2023-10-18T12:25:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.849+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3620010430 con 0x7f362c074770 2023-10-18T12:25:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.849+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c1a2880 con 0x7f362c074770 2023-10-18T12:25:02.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.849+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620018680 con 0x7f362c074770 2023-10-18T12:25:02.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.849+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c1aa0e0 con 0x7f362c074770 2023-10-18T12:25:02.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.850+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f362c117140 con 0x7f362c074770 2023-10-18T12:25:02.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.852+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f36200105a0 con 0x7f362c074770 2023-10-18T12:25:02.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.854+0000 7f3628ff9700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36140a1560 0x7f36140a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f3620014070 con 0x7f362c074770 2023-10-18T12:25:02.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f362b7fe700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36140a1560 0x7f36140a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f3628ff9700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] conn(0x7f36140ac270 0x7f36140ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:02.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 --> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36140aed80 con 0x7f36140ac270 2023-10-18T12:25:02.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36200cd9f0 con 0x7f362c074770 2023-10-18T12:25:02.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f362b7fe700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36140a1560 0x7f36140a3a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b2e0 tx=0x7f361c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.855+0000 7f362bfff700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] conn(0x7f36140ac270 0x7f36140ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:02.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.856+0000 7f362bfff700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] conn(0x7f36140ac270 0x7f36140ae6b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:02.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.857+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== osd.5 v2:172.21.15.186:6808/443683147 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f36140aed80 con 0x7f36140ac270 2023-10-18T12:25:02.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.875+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 --> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f362c110b10 con 0x7f36140ac270 2023-10-18T12:25:02.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.876+0000 7f3628ff9700 1 -- 172.21.15.70:0/873599316 <== osd.5 v2:172.21.15.186:6808/443683147 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f362c110b10 con 0x7f36140ac270 2023-10-18T12:25:02.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.876+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] conn(0x7f36140ac270 msgr2=0x7f36140ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.876+0000 7f3631ab0700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] conn(0x7f36140ac270 0x7f36140ae6b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36140a1560 msgr2=0x7f36140a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36140a1560 0x7f36140a3a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b2e0 tx=0x7f361c005fd0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 msgr2=0x7f362c19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:02.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 --2- 172.21.15.70:0/873599316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c074770 0x7f362c19d8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f362000e9a0 tx=0x7f362000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:02.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f362bfff700 1 -- 172.21.15.70:0/873599316 reap_dead start 2023-10-18T12:25:02.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 shutdown_connections 2023-10-18T12:25:02.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 >> 172.21.15.70:0/873599316 conn(0x7f362c0779e0 msgr2=0x7f362c1101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:02.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 shutdown_connections 2023-10-18T12:25:02.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:02.877+0000 7f3631ab0700 1 -- 172.21.15.70:0/873599316 wait complete. 2023-10-18T12:25:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:03 smithi070 ceph-mon[98639]: pgmap v281: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:25:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:03 smithi186 ceph-mon[135253]: pgmap v281: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:25:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:03 smithi152 ceph-mon[134767]: pgmap v281: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:25:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 -- 172.21.15.70:0/2160642032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff58101260 msgr2=0x7eff5810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 --2- 172.21.15.70:0/2160642032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff58101260 0x7eff5810d650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff5000b0a0 tx=0x7eff5000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 -- 172.21.15.70:0/2160642032 shutdown_connections 2023-10-18T12:25:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 --2- 172.21.15.70:0/2160642032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff58101260 0x7eff5810d650 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 --2- 172.21.15.70:0/2160642032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff581057c0 0x7eff58105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 --2- 172.21.15.70:0/2160642032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff581008a0 0x7eff58100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.497+0000 7eff5e824700 1 -- 172.21.15.70:0/2160642032 >> 172.21.15.70:0/2160642032 conn(0x7eff580779e0 msgr2=0x7eff58077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.498+0000 7eff5e824700 1 -- 172.21.15.70:0/2160642032 shutdown_connections 2023-10-18T12:25:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.498+0000 7eff5e824700 1 -- 172.21.15.70:0/2160642032 wait complete. 2023-10-18T12:25:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 Processor -- start 2023-10-18T12:25:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 -- start start 2023-10-18T12:25:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff581008a0 0x7eff58198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff58101260 0x7eff58199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff581057c0 0x7eff5819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5819de40 con 0x7eff581008a0 2023-10-18T12:25:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5819dfb0 con 0x7eff58101260 2023-10-18T12:25:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5e824700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5819e170 con 0x7eff581057c0 2023-10-18T12:25:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff581008a0 0x7eff58198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff581008a0 0x7eff58198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33092/0 (socket says 172.21.15.70:33092) 2023-10-18T12:25:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff57fff700 1 -- 172.21.15.70:0/3013803170 learned_addr learned my addr 172.21.15.70:0/3013803170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff5cdc1700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff581057c0 0x7eff5819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.499+0000 7eff577fe700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff58101260 0x7eff58199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff57fff700 1 -- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff581057c0 msgr2=0x7eff5819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff57fff700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff581057c0 0x7eff5819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff57fff700 1 -- 172.21.15.70:0/3013803170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff58101260 msgr2=0x7eff58199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff57fff700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff58101260 0x7eff58199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff57fff700 1 -- 172.21.15.70:0/3013803170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff50009d00 con 0x7eff581008a0 2023-10-18T12:25:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff577fe700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff58101260 0x7eff58199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff57fff700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff581008a0 0x7eff58198f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e970 tx=0x7eff4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48009790 con 0x7eff581008a0 2023-10-18T12:25:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff4800eec0 con 0x7eff581008a0 2023-10-18T12:25:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5819e450 con 0x7eff581008a0 2023-10-18T12:25:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48003cf0 con 0x7eff581008a0 2023-10-18T12:25:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.500+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff581a5cb0 con 0x7eff581008a0 2023-10-18T12:25:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.501+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff58110a00 con 0x7eff581008a0 2023-10-18T12:25:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.503+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7eff4800ccd0 con 0x7eff581008a0 2023-10-18T12:25:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.504+0000 7eff557fa700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7eff400a15e0 0x7eff400a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7eff48014070 con 0x7eff581008a0 2023-10-18T12:25:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff577fe700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7eff400a15e0 0x7eff400a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff557fa700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] conn(0x7eff400ac2f0 0x7eff400ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 --> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff400aee00 con 0x7eff400ac2f0 2023-10-18T12:25:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff480cd840 con 0x7eff581008a0 2023-10-18T12:25:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff577fe700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7eff400a15e0 0x7eff400a3aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff5819a4a0 tx=0x7eff4c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.505+0000 7eff5cdc1700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] conn(0x7eff400ac2f0 0x7eff400ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.506+0000 7eff5cdc1700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] conn(0x7eff400ac2f0 0x7eff400ae730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.507+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== osd.8 v2:172.21.15.186:6816/2454566106 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7eff400aee00 con 0x7eff400ac2f0 2023-10-18T12:25:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.523+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 --> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff58101f30 con 0x7eff400ac2f0 2023-10-18T12:25:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff557fa700 1 -- 172.21.15.70:0/3013803170 <== osd.8 v2:172.21.15.186:6816/2454566106 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7eff58101f30 con 0x7eff400ac2f0 2023-10-18T12:25:03.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] conn(0x7eff400ac2f0 msgr2=0x7eff400ae730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:03.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5e824700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] conn(0x7eff400ac2f0 0x7eff400ae730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:03.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7eff400a15e0 msgr2=0x7eff400a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:03.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5e824700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7eff400a15e0 0x7eff400a3aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff5819a4a0 tx=0x7eff4c009260 comp rx=0 tx=0).stop 2023-10-18T12:25:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff581008a0 msgr2=0x7eff58198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5e824700 1 --2- 172.21.15.70:0/3013803170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff581008a0 0x7eff58198f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e970 tx=0x7eff4800ec80 comp rx=0 tx=0).stop 2023-10-18T12:25:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.524+0000 7eff5cdc1700 1 -- 172.21.15.70:0/3013803170 reap_dead start 2023-10-18T12:25:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.525+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 shutdown_connections 2023-10-18T12:25:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.525+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 >> 172.21.15.70:0/3013803170 conn(0x7eff580779e0 msgr2=0x7eff5810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.525+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 shutdown_connections 2023-10-18T12:25:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:03.525+0000 7eff5e824700 1 -- 172.21.15.70:0/3013803170 wait complete. 2023-10-18T12:25:03.909 INFO:teuthology.orchestra.run.smithi070.stdout:60129542210 2023-10-18T12:25:03.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-18T12:25:04.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/2644214350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08072f60 msgr2=0x7fcd080733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/2644214350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08072f60 0x7fcd080733e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009990 tx=0x7fccfc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/2644214350 shutdown_connections 2023-10-18T12:25:04.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/2644214350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd08111570 0x7fcd081139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/2644214350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd08074770 0x7fcd08072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/2644214350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08072f60 0x7fcd080733e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/2644214350 >> 172.21.15.70:0/2644214350 conn(0x7fcd080779e0 msgr2=0x7fcd08077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:04.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/2644214350 shutdown_connections 2023-10-18T12:25:04.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.256+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/2644214350 wait complete. 2023-10-18T12:25:04.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.257+0000 7fcd0d0e8700 1 Processor -- start 2023-10-18T12:25:04.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.257+0000 7fcd0d0e8700 1 -- start start 2023-10-18T12:25:04.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0d0e8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd08072f60 0x7fcd0819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0d0e8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd08074770 0x7fcd0819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0d0e8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08111570 0x7fcd081a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0d0e8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081a21e0 con 0x7fcd08111570 2023-10-18T12:25:04.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0d0e8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081a2350 con 0x7fcd08074770 2023-10-18T12:25:04.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0d0e8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081a2510 con 0x7fcd08072f60 2023-10-18T12:25:04.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08111570 0x7fcd081a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08111570 0x7fcd081a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33030/0 (socket says 172.21.15.70:33030) 2023-10-18T12:25:04.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd08074770 0x7fcd0819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0759e700 1 -- 172.21.15.70:0/3425758483 learned_addr learned my addr 172.21.15.70:0/3425758483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:04.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd06d9d700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd08072f60 0x7fcd0819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.258+0000 7fcd0659c700 1 -- 172.21.15.70:0/3425758483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd08072f60 msgr2=0x7fcd0819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0659c700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd08072f60 0x7fcd0819d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0659c700 1 -- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08111570 msgr2=0x7fcd081a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0659c700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08111570 0x7fcd081a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0659c700 1 -- 172.21.15.70:0/3425758483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc0095f0 con 0x7fcd08074770 2023-10-18T12:25:04.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0759e700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd08111570 0x7fcd081a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:04.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd06d9d700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd08072f60 0x7fcd0819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:04.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0659c700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd08074770 0x7fcd0819d820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000c00 tx=0x7fccfc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:04.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc01d070 con 0x7fcd08074770 2023-10-18T12:25:04.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd081a2790 con 0x7fcd08074770 2023-10-18T12:25:04.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccfc021410 con 0x7fcd08074770 2023-10-18T12:25:04.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc017650 con 0x7fcd08074770 2023-10-18T12:25:04.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.259+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd081aa050 con 0x7fcd08074770 2023-10-18T12:25:04.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.260+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd081170f0 con 0x7fcd08074770 2023-10-18T12:25:04.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.263+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fccfc021580 con 0x7fcd08074770 2023-10-18T12:25:04.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.265+0000 7fcceffff700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fccf00a15e0 0x7fccf00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.265+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fccfc0ce600 con 0x7fcd08074770 2023-10-18T12:25:04.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.265+0000 7fcd06d9d700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fccf00a15e0 0x7fccf00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.265+0000 7fcceffff700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] conn(0x7fccf00ac2f0 0x7fccf00ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.265+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 --> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccf00aee00 con 0x7fccf00ac2f0 2023-10-18T12:25:04.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.265+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccfc0ce9b0 con 0x7fcd08074770 2023-10-18T12:25:04.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.266+0000 7fcd0759e700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] conn(0x7fccf00ac2f0 0x7fccf00ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.266+0000 7fcd06d9d700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fccf00a15e0 0x7fccf00a3aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011490 tx=0x7fccf8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:04.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.266+0000 7fcd0759e700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] conn(0x7fccf00ac2f0 0x7fccf00ae730 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:04.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.267+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== osd.4 v2:172.21.15.152:6808/934272440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fccf00aee00 con 0x7fccf00ac2f0 2023-10-18T12:25:04.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.284+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 --> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd08110a90 con 0x7fccf00ac2f0 2023-10-18T12:25:04.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.285+0000 7fcceffff700 1 -- 172.21.15.70:0/3425758483 <== osd.4 v2:172.21.15.152:6808/934272440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcd08110a90 con 0x7fccf00ac2f0 2023-10-18T12:25:04.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.285+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] conn(0x7fccf00ac2f0 msgr2=0x7fccf00ae730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.285+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] conn(0x7fccf00ac2f0 0x7fccf00ae730 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fccf00a15e0 msgr2=0x7fccf00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fccf00a15e0 0x7fccf00a3aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011490 tx=0x7fccf8009260 comp rx=0 tx=0).stop 2023-10-18T12:25:04.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd08074770 msgr2=0x7fcd0819d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 --2- 172.21.15.70:0/3425758483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd08074770 0x7fcd0819d820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000c00 tx=0x7fccfc004730 comp rx=0 tx=0).stop 2023-10-18T12:25:04.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0759e700 1 -- 172.21.15.70:0/3425758483 reap_dead start 2023-10-18T12:25:04.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 shutdown_connections 2023-10-18T12:25:04.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 >> 172.21.15.70:0/3425758483 conn(0x7fcd080779e0 msgr2=0x7fcd08110140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:04.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 shutdown_connections 2023-10-18T12:25:04.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:01.286+0000 7fcd0d0e8700 1 -- 172.21.15.70:0/3425758483 wait complete. 2023-10-18T12:25:04.279 INFO:teuthology.orchestra.run.smithi070.stdout:201863462955 2023-10-18T12:25:04.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-18T12:25:04.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 -- 172.21.15.70:0/998584938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc0ff750 msgr2=0x7fa7bc0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 --2- 172.21.15.70:0/998584938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc0ffbd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009990 tx=0x7fa7b0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 -- 172.21.15.70:0/998584938 shutdown_connections 2023-10-18T12:25:04.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 --2- 172.21.15.70:0/998584938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc100110 0x7fa7bc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 --2- 172.21.15.70:0/998584938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7bc1056b0 0x7fa7bc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 --2- 172.21.15.70:0/998584938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc0ffbd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 -- 172.21.15.70:0/998584938 >> 172.21.15.70:0/998584938 conn(0x7fa7bc075790 msgr2=0x7fa7bc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:04.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 -- 172.21.15.70:0/998584938 shutdown_connections 2023-10-18T12:25:04.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.878+0000 7fa7c166f700 1 -- 172.21.15.70:0/998584938 wait complete. 2023-10-18T12:25:04.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 Processor -- start 2023-10-18T12:25:04.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 -- start start 2023-10-18T12:25:04.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc100110 0x7fa7bc1a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7bc1056b0 0x7fa7bc10b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc10bc20 con 0x7fa7bc100110 2023-10-18T12:25:04.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc10bd90 con 0x7fa7bc0ff750 2023-10-18T12:25:04.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.880+0000 7fa7c166f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc10bf50 con 0x7fa7bc1056b0 2023-10-18T12:25:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc1a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56982/0 (socket says 172.21.15.70:56982) 2023-10-18T12:25:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 -- 172.21.15.70:0/1084347686 learned_addr learned my addr 172.21.15.70:0/1084347686 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7bb7fe700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7bc1056b0 0x7fa7bc10b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7ba7fc700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc100110 0x7fa7bc1a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 -- 172.21.15.70:0/1084347686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7bc1056b0 msgr2=0x7fa7bc10b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7bc1056b0 0x7fa7bc10b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 -- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc100110 msgr2=0x7fa7bc1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc100110 0x7fa7bc1a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7baffd700 1 -- 172.21.15.70:0/1084347686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b00095f0 con 0x7fa7bc0ff750 2023-10-18T12:25:04.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.881+0000 7fa7bb7fe700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7bc1056b0 0x7fa7bc10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:04.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7ba7fc700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7bc100110 0x7fa7bc1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:04.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7baffd700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc1a1e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00e9a0 tx=0x7fa7ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:04.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00cbd0 con 0x7fa7bc0ff750 2023-10-18T12:25:04.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ac00cd30 con 0x7fa7bc0ff750 2023-10-18T12:25:04.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac018680 con 0x7fa7bc0ff750 2023-10-18T12:25:04.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc10c230 con 0x7fa7bc0ff750 2023-10-18T12:25:04.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.882+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc1a9f80 con 0x7fa7bc0ff750 2023-10-18T12:25:04.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.883+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7bc110a50 con 0x7fa7bc0ff750 2023-10-18T12:25:04.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.887+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fa7ac0187e0 con 0x7fa7bc0ff750 2023-10-18T12:25:04.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.889+0000 7fa7a3fff700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7a40a1660 0x7fa7a40a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.889+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fa7ac014070 con 0x7fa7bc0ff750 2023-10-18T12:25:04.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.889+0000 7fa7ba7fc700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7a40a1660 0x7fa7a40a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.889+0000 7fa7a3fff700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] conn(0x7fa7a40ac370 0x7fa7a40ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.889+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 --> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7a40aee80 con 0x7fa7a40ac370 2023-10-18T12:25:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.889+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ac0cda60 con 0x7fa7bc0ff750 2023-10-18T12:25:04.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.890+0000 7fa7bb7fe700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] conn(0x7fa7a40ac370 0x7fa7a40ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:04.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.890+0000 7fa7ba7fc700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7a40a1660 0x7fa7a40a3b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000b2e0 tx=0x7fa7b0005fb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:04.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.891+0000 7fa7bb7fe700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] conn(0x7fa7a40ac370 0x7fa7a40ae7b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:04.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.892+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== osd.3 v2:172.21.15.70:6810/3863911164 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fa7a40aee80 con 0x7fa7a40ac370 2023-10-18T12:25:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.911+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 --> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7bc0fdfb0 con 0x7fa7a40ac370 2023-10-18T12:25:04.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.911+0000 7fa7a3fff700 1 -- 172.21.15.70:0/1084347686 <== osd.3 v2:172.21.15.70:6810/3863911164 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa7bc0fdfb0 con 0x7fa7a40ac370 2023-10-18T12:25:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] conn(0x7fa7a40ac370 msgr2=0x7fa7a40ae7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] conn(0x7fa7a40ac370 0x7fa7a40ae7b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7a40a1660 msgr2=0x7fa7a40a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7a40a1660 0x7fa7a40a3b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000b2e0 tx=0x7fa7b0005fb0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc0ff750 msgr2=0x7fa7bc1a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 --2- 172.21.15.70:0/1084347686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7bc0ff750 0x7fa7bc1a1e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00e9a0 tx=0x7fa7ac00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7bb7fe700 1 -- 172.21.15.70:0/1084347686 reap_dead start 2023-10-18T12:25:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 shutdown_connections 2023-10-18T12:25:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 >> 172.21.15.70:0/1084347686 conn(0x7fa7bc075790 msgr2=0x7fa7bc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 shutdown_connections 2023-10-18T12:25:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:04.912+0000 7fa7c166f700 1 -- 172.21.15.70:0/1084347686 wait complete. 2023-10-18T12:25:05.188 INFO:teuthology.orchestra.run.smithi070.stdout:146028888118 2023-10-18T12:25:05.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-18T12:25:05.311 INFO:teuthology.orchestra.run.smithi070.stdout:257698037795 2023-10-18T12:25:05.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-18T12:25:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:05 smithi186 ceph-mon[135253]: pgmap v282: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:05.510 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:05 smithi070 ceph-mon[98639]: pgmap v282: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:05 smithi152 ceph-mon[134767]: pgmap v282: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:06.246 INFO:teuthology.orchestra.run.smithi070.stdout:111669149754 2023-10-18T12:25:06.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-18T12:25:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 -- 172.21.15.70:0/3599970621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 msgr2=0x7f1e7c10ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/3599970621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 0x7f1e7c10ccb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009990 tx=0x7f1e64009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 -- 172.21.15.70:0/3599970621 shutdown_connections 2023-10-18T12:25:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/3599970621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e7c10d1f0 0x7f1e7c10f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/3599970621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 0x7f1e7c072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/3599970621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 0x7f1e7c10ccb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.251+0000 7f1e82d7f700 1 -- 172.21.15.70:0/3599970621 >> 172.21.15.70:0/3599970621 conn(0x7f1e7c077950 msgr2=0x7f1e7c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.252+0000 7f1e82d7f700 1 -- 172.21.15.70:0/3599970621 shutdown_connections 2023-10-18T12:25:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.252+0000 7f1e82d7f700 1 -- 172.21.15.70:0/3599970621 wait complete. 2023-10-18T12:25:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.253+0000 7f1e82d7f700 1 Processor -- start 2023-10-18T12:25:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.253+0000 7f1e82d7f700 1 -- start start 2023-10-18T12:25:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e82d7f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 0x7f1e7c198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e82d7f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 0x7f1e7c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e82d7f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e7c10d1f0 0x7f1e7c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e82d7f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c19dca0 con 0x7f1e7c073340 2023-10-18T12:25:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e82d7f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c19de10 con 0x7f1e7c10d1f0 2023-10-18T12:25:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e82d7f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c19dfd0 con 0x7f1e7c072a20 2023-10-18T12:25:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e80b1b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 0x7f1e7c198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e7bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 0x7f1e7c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e80b1b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 0x7f1e7c198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:55314/0 (socket says 172.21.15.70:55314) 2023-10-18T12:25:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.254+0000 7f1e80b1b700 1 -- 172.21.15.70:0/992455717 learned_addr learned my addr 172.21.15.70:0/992455717 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e8131c700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e7c10d1f0 0x7f1e7c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e80b1b700 1 -- 172.21.15.70:0/992455717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e7c10d1f0 msgr2=0x7f1e7c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e80b1b700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e7c10d1f0 0x7f1e7c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e80b1b700 1 -- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 msgr2=0x7f1e7c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e80b1b700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c073340 0x7f1e7c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e80b1b700 1 -- 172.21.15.70:0/992455717 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e640095f0 con 0x7f1e7c072a20 2023-10-18T12:25:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e8131c700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e7c10d1f0 0x7f1e7c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:25:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e80b1b700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 0x7f1e7c198e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400d980 tx=0x7f1e7400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.255+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74004220 con 0x7f1e7c072a20 2023-10-18T12:25:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.256+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c19e2b0 con 0x7f1e7c072a20 2023-10-18T12:25:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.256+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c1a5b10 con 0x7f1e7c072a20 2023-10-18T12:25:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.256+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e74004cf0 con 0x7f1e7c072a20 2023-10-18T12:25:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.256+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400b750 con 0x7f1e7c072a20 2023-10-18T12:25:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.257+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e5c000fc0 con 0x7f1e7c072a20 2023-10-18T12:25:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.260+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1e74004830 con 0x7f1e7c072a20 2023-10-18T12:25:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.262+0000 7f1e79ffb700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e680a15e0 0x7f1e680a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.262+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f1e740ccef0 con 0x7f1e7c072a20 2023-10-18T12:25:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.262+0000 7f1e7bfff700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e680a15e0 0x7f1e680a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.263+0000 7f1e79ffb700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] conn(0x7f1e680ac2f0 0x7f1e680ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.263+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 --> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e680aee00 con 0x7f1e680ac2f0 2023-10-18T12:25:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.263+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e740cf4d0 con 0x7f1e7c072a20 2023-10-18T12:25:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.263+0000 7f1e7bfff700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e680a15e0 0x7f1e680a3aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c10eeb0 tx=0x7f1e6400b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.263+0000 7f1e8131c700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] conn(0x7f1e680ac2f0 0x7f1e680ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.264+0000 7f1e8131c700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] conn(0x7f1e680ac2f0 0x7f1e680ae730 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.265+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== osd.11 v2:172.21.15.186:6824/1072168944 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1e680aee00 con 0x7f1e680ac2f0 2023-10-18T12:25:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.288+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 --> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e5c002d40 con 0x7f1e680ac2f0 2023-10-18T12:25:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.288+0000 7f1e79ffb700 1 -- 172.21.15.70:0/992455717 <== osd.11 v2:172.21.15.186:6824/1072168944 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1e5c002d40 con 0x7f1e680ac2f0 2023-10-18T12:25:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] conn(0x7f1e680ac2f0 msgr2=0x7f1e680ae730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] conn(0x7f1e680ac2f0 0x7f1e680ae730 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e680a15e0 msgr2=0x7f1e680a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1e680a15e0 0x7f1e680a3aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c10eeb0 tx=0x7f1e6400b540 comp rx=0 tx=0).stop 2023-10-18T12:25:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 msgr2=0x7f1e7c198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 --2- 172.21.15.70:0/992455717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e7c072a20 0x7f1e7c198e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400d980 tx=0x7f1e7400dc90 comp rx=0 tx=0).stop 2023-10-18T12:25:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e8131c700 1 -- 172.21.15.70:0/992455717 reap_dead start 2023-10-18T12:25:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 shutdown_connections 2023-10-18T12:25:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.289+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 >> 172.21.15.70:0/992455717 conn(0x7f1e7c077950 msgr2=0x7f1e7c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.290+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 shutdown_connections 2023-10-18T12:25:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.290+0000 7f1e82d7f700 1 -- 172.21.15.70:0/992455717 wait complete. 2023-10-18T12:25:06.465 INFO:teuthology.orchestra.run.smithi070.stdout:167503724594 2023-10-18T12:25:06.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-18T12:25:06.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 -- 172.21.15.70:0/1399323992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01029c0 msgr2=0x7efef0102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 --2- 172.21.15.70:0/1399323992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01029c0 0x7efef0102e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efee0009990 tx=0x7efee0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 -- 172.21.15.70:0/1399323992 shutdown_connections 2023-10-18T12:25:06.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 --2- 172.21.15.70:0/1399323992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efef0103380 0x7efef010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 --2- 172.21.15.70:0/1399323992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef01089c0 0x7efef0108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 --2- 172.21.15.70:0/1399323992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01029c0 0x7efef0102e40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.598+0000 7efef87a3700 1 -- 172.21.15.70:0/1399323992 >> 172.21.15.70:0/1399323992 conn(0x7efef00fe4b0 msgr2=0x7efef01008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:06.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.599+0000 7efef87a3700 1 -- 172.21.15.70:0/1399323992 shutdown_connections 2023-10-18T12:25:06.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.599+0000 7efef87a3700 1 -- 172.21.15.70:0/1399323992 wait complete. 2023-10-18T12:25:06.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.600+0000 7efef87a3700 1 Processor -- start 2023-10-18T12:25:06.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.600+0000 7efef87a3700 1 -- start start 2023-10-18T12:25:06.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.601+0000 7efef87a3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efef01029c0 0x7efef0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.601+0000 7efef87a3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef0103380 0x7efef0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.601+0000 7efef87a3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01089c0 0x7efef019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.601+0000 7efef87a3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef019dd10 con 0x7efef01089c0 2023-10-18T12:25:06.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.601+0000 7efef87a3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef019de80 con 0x7efef0103380 2023-10-18T12:25:06.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.601+0000 7efef87a3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef019e040 con 0x7efef01029c0 2023-10-18T12:25:06.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef0103380 0x7efef0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef6d40700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01089c0 0x7efef019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef0103380 0x7efef0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57024/0 (socket says 172.21.15.70:57024) 2023-10-18T12:25:06.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 -- 172.21.15.70:0/2211805514 learned_addr learned my addr 172.21.15.70:0/2211805514 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:06.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef653f700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efef01029c0 0x7efef0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 -- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efef01029c0 msgr2=0x7efef0198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efef01029c0 0x7efef0198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 -- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01089c0 msgr2=0x7efef019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.602+0000 7efef5d3e700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01089c0 0x7efef019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.603+0000 7efef5d3e700 1 -- 172.21.15.70:0/2211805514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00095f0 con 0x7efef0103380 2023-10-18T12:25:06.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.603+0000 7efef6d40700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efef01089c0 0x7efef019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:06.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.603+0000 7efef5d3e700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef0103380 0x7efef0199350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efee0000c00 tx=0x7efee0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.603+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee001d070 con 0x7efef0103380 2023-10-18T12:25:06.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.604+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef019e2c0 con 0x7efef0103380 2023-10-18T12:25:06.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.604+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee0022470 con 0x7efef0103380 2023-10-18T12:25:06.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.604+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0017650 con 0x7efef0103380 2023-10-18T12:25:06.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.604+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01a5b80 con 0x7efef0103380 2023-10-18T12:25:06.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.605+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efef0112cc0 con 0x7efef0103380 2023-10-18T12:25:06.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.609+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7efee00225e0 con 0x7efef0103380 2023-10-18T12:25:06.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.612+0000 7efee77fe700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efedc0a1560 0x7efedc0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.612+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7efee00cee50 con 0x7efef0103380 2023-10-18T12:25:06.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.612+0000 7efef653f700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efedc0a1560 0x7efedc0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.613+0000 7efee77fe700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] conn(0x7efedc0ac270 0x7efedc0ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.613+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 --> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efedc0aed80 con 0x7efedc0ac270 2023-10-18T12:25:06.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.613+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efee00cf260 con 0x7efef0103380 2023-10-18T12:25:06.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.613+0000 7efef653f700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efedc0a1560 0x7efedc0a3a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efeec009be0 tx=0x7efeec009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.613+0000 7efef6d40700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] conn(0x7efedc0ac270 0x7efedc0ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.614+0000 7efef6d40700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] conn(0x7efedc0ac270 0x7efedc0ae6b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.615+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== osd.2 v2:172.21.15.186:6800/1687717134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7efedc0aed80 con 0x7efedc0ac270 2023-10-18T12:25:06.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.634+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 --> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efef0060ef0 con 0x7efedc0ac270 2023-10-18T12:25:06.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.635+0000 7efee77fe700 1 -- 172.21.15.70:0/2211805514 <== osd.2 v2:172.21.15.186:6800/1687717134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efef0060ef0 con 0x7efedc0ac270 2023-10-18T12:25:06.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.635+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] conn(0x7efedc0ac270 msgr2=0x7efedc0ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.635+0000 7efef87a3700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] conn(0x7efedc0ac270 0x7efedc0ae6b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.635+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efedc0a1560 msgr2=0x7efedc0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.635+0000 7efef87a3700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efedc0a1560 0x7efedc0a3a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efeec009be0 tx=0x7efeec009260 comp rx=0 tx=0).stop 2023-10-18T12:25:06.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.635+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef0103380 msgr2=0x7efef0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.636+0000 7efef87a3700 1 --2- 172.21.15.70:0/2211805514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef0103380 0x7efef0199350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efee0000c00 tx=0x7efee0005740 comp rx=0 tx=0).stop 2023-10-18T12:25:06.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.636+0000 7efef6d40700 1 -- 172.21.15.70:0/2211805514 reap_dead start 2023-10-18T12:25:06.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.636+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 shutdown_connections 2023-10-18T12:25:06.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.636+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 >> 172.21.15.70:0/2211805514 conn(0x7efef00fe4b0 msgr2=0x7efef00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:06.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.636+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 shutdown_connections 2023-10-18T12:25:06.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.636+0000 7efef87a3700 1 -- 172.21.15.70:0/2211805514 wait complete. 2023-10-18T12:25:06.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 -- 172.21.15.70:0/2290778643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c1067a0 msgr2=0x7f5a2c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 --2- 172.21.15.70:0/2290778643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c106b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009990 tx=0x7f5a28009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 -- 172.21.15.70:0/2290778643 shutdown_connections 2023-10-18T12:25:06.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 --2- 172.21.15.70:0/2290778643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a2c068b30 0x7f5a2c10d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 --2- 172.21.15.70:0/2290778643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c106b80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 --2- 172.21.15.70:0/2290778643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c068190 0x7f5a2c0685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 -- 172.21.15.70:0/2290778643 >> 172.21.15.70:0/2290778643 conn(0x7f5a2c075790 msgr2=0x7f5a2c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:06.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.660+0000 7f5a33b09700 1 -- 172.21.15.70:0/2290778643 shutdown_connections 2023-10-18T12:25:06.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.661+0000 7f5a33b09700 1 -- 172.21.15.70:0/2290778643 wait complete. 2023-10-18T12:25:06.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.662+0000 7f5a33b09700 1 Processor -- start 2023-10-18T12:25:06.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.662+0000 7f5a33b09700 1 -- start start 2023-10-18T12:25:06.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a33b09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c068190 0x7f5a2c0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a33b09700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a2c068b30 0x7f5a2c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a33b09700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a33b09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c101790 con 0x7f5a2c068190 2023-10-18T12:25:06.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a33b09700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c1a9680 con 0x7f5a2c068b30 2023-10-18T12:25:06.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a33b09700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c1a97f0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a320a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a320a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c1002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:55352/0 (socket says 172.21.15.70:55352) 2023-10-18T12:25:06.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.663+0000 7f5a320a6700 1 -- 172.21.15.70:0/1142098048 learned_addr learned my addr 172.21.15.70:0/1142098048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:06.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a318a5700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c068190 0x7f5a2c0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a310a4700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a2c068b30 0x7f5a2c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a320a6700 1 -- 172.21.15.70:0/1142098048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a2c068b30 msgr2=0x7f5a2c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a320a6700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a2c068b30 0x7f5a2c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a320a6700 1 -- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c068190 msgr2=0x7f5a2c0ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a320a6700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c068190 0x7f5a2c0ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.664+0000 7f5a320a6700 1 -- 172.21.15.70:0/1142098048 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280095f0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.665+0000 7f5a320a6700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c1002a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400b970 tx=0x7f5a2400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.665+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2400c950 con 0x7f5a2c1067a0 2023-10-18T12:25:06.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.665+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a24014440 con 0x7f5a2c1067a0 2023-10-18T12:25:06.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.665+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a240075d0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.665+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c1a9ad0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.665+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c1a9f80 con 0x7f5a2c1067a0 2023-10-18T12:25:06.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.667+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a2c101ee0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.670+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5a240145b0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.671+0000 7f5a22ffd700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5a180a1560 0x7f5a180a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.671+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f5a24006e40 con 0x7f5a2c1067a0 2023-10-18T12:25:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.672+0000 7f5a22ffd700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] conn(0x7f5a180ac270 0x7f5a180ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.672+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 --> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a180aed80 con 0x7f5a180ac270 2023-10-18T12:25:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.672+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a240cecc0 con 0x7f5a2c1067a0 2023-10-18T12:25:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.672+0000 7f5a318a5700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5a180a1560 0x7f5a180a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.673+0000 7f5a310a4700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] conn(0x7f5a180ac270 0x7f5a180ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.673+0000 7f5a318a5700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5a180a1560 0x7f5a180a3a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800b2e0 tx=0x7f5a28005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.674+0000 7f5a310a4700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] conn(0x7f5a180ac270 0x7f5a180ae6b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:06.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.675+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== osd.9 v2:172.21.15.70:6826/4034457990 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f5a180aed80 con 0x7f5a180ac270 2023-10-18T12:25:06.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.707+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 --> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5a2c110a00 con 0x7f5a180ac270 2023-10-18T12:25:06.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.709+0000 7f5a22ffd700 1 -- 172.21.15.70:0/1142098048 <== osd.9 v2:172.21.15.70:6826/4034457990 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5a2c110a00 con 0x7f5a180ac270 2023-10-18T12:25:06.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.711+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] conn(0x7f5a180ac270 msgr2=0x7f5a180ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.711+0000 7f5a33b09700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] conn(0x7f5a180ac270 0x7f5a180ae6b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.711+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5a180a1560 msgr2=0x7f5a180a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.711+0000 7f5a33b09700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5a180a1560 0x7f5a180a3a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800b2e0 tx=0x7f5a28005fd0 comp rx=0 tx=0).stop 2023-10-18T12:25:06.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.711+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c1067a0 msgr2=0x7f5a2c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:06.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.711+0000 7f5a33b09700 1 --2- 172.21.15.70:0/1142098048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a2c1067a0 0x7f5a2c1002a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400b970 tx=0x7f5a2400bd30 comp rx=0 tx=0).stop 2023-10-18T12:25:06.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.712+0000 7f5a320a6700 1 -- 172.21.15.70:0/1142098048 reap_dead start 2023-10-18T12:25:06.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.712+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 shutdown_connections 2023-10-18T12:25:06.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.712+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 >> 172.21.15.70:0/1142098048 conn(0x7f5a2c075790 msgr2=0x7f5a2c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:06.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.712+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 shutdown_connections 2023-10-18T12:25:06.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:06.712+0000 7f5a33b09700 1 -- 172.21.15.70:0/1142098048 wait complete. 2023-10-18T12:25:07.099 INFO:teuthology.orchestra.run.smithi070.stdout:343597383703 2023-10-18T12:25:07.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-18T12:25:07.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:07 smithi070 ceph-mon[98639]: pgmap v283: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 -- 172.21.15.70:0/2383086384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03381067c0 msgr2=0x7f0338106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 --2- 172.21.15.70:0/2383086384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03381067c0 0x7f0338106ba0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0334009990 tx=0x7f0334009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 -- 172.21.15.70:0/2383086384 shutdown_connections 2023-10-18T12:25:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 --2- 172.21.15.70:0/2383086384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0338068b30 0x7f033810d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 --2- 172.21.15.70:0/2383086384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03381067c0 0x7f0338106ba0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 --2- 172.21.15.70:0/2383086384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 0x7f03380685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 -- 172.21.15.70:0/2383086384 >> 172.21.15.70:0/2383086384 conn(0x7f0338075790 msgr2=0x7f0338075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 -- 172.21.15.70:0/2383086384 shutdown_connections 2023-10-18T12:25:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.234+0000 7f03401ce700 1 -- 172.21.15.70:0/2383086384 wait complete. 2023-10-18T12:25:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.236+0000 7f03401ce700 1 Processor -- start 2023-10-18T12:25:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.236+0000 7f03401ce700 1 -- start start 2023-10-18T12:25:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.237+0000 7f03401ce700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 0x7f0338194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.237+0000 7f03401ce700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338068b30 0x7f0338194fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.237+0000 7f03401ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03381067c0 0x7f0338199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.237+0000 7f03401ce700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338199940 con 0x7f0338068b30 2023-10-18T12:25:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f03401ce700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338199ab0 con 0x7f03381067c0 2023-10-18T12:25:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f03401ce700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338199c70 con 0x7f0338068190 2023-10-18T12:25:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 0x7f0338194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 0x7f0338194aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:55360/0 (socket says 172.21.15.70:55360) 2023-10-18T12:25:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 -- 172.21.15.70:0/1003486354 learned_addr learned my addr 172.21.15.70:0/1003486354 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033e76b700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03381067c0 0x7f0338199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033d769700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338068b30 0x7f0338194fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 -- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03381067c0 msgr2=0x7f0338199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03381067c0 0x7f0338199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 -- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338068b30 msgr2=0x7f0338194fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338068b30 0x7f0338194fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033df6a700 1 -- 172.21.15.70:0/1003486354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330009c40 con 0x7f0338068190 2023-10-18T12:25:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.238+0000 7f033d769700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338068b30 0x7f0338194fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.239+0000 7f033df6a700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 0x7f0338194aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0334005910 tx=0x7f0334003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.239+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334003d50 con 0x7f0338068190 2023-10-18T12:25:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.239+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03340095f0 con 0x7f0338068190 2023-10-18T12:25:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.239+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0334005700 con 0x7f0338068190 2023-10-18T12:25:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.239+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334022870 con 0x7f0338068190 2023-10-18T12:25:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.239+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03381a1620 con 0x7f0338068190 2023-10-18T12:25:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.240+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f033810e8e0 con 0x7f0338068190 2023-10-18T12:25:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.244+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f033400bb20 con 0x7f0338068190 2023-10-18T12:25:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.246+0000 7f032effd700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03240a15c0 0x7f03240a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.246+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f0334011e40 con 0x7f0338068190 2023-10-18T12:25:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.246+0000 7f032effd700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] conn(0x7f03240ac2d0 0x7f03240ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.246+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 --> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03240aede0 con 0x7f03240ac2d0 2023-10-18T12:25:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.246+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0334022870 con 0x7f0338068190 2023-10-18T12:25:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.247+0000 7f033d769700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03240a15c0 0x7f03240a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.247+0000 7f033e76b700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] conn(0x7f03240ac2d0 0x7f03240ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:07.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.247+0000 7f033e76b700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] conn(0x7f03240ac2d0 0x7f03240ae710 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:07.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.248+0000 7f033d769700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03240a15c0 0x7f03240a3a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0338195fb0 tx=0x7f032800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:07.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.248+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== osd.10 v2:172.21.15.152:6824/4238500329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f03240aede0 con 0x7f03240ac2d0 2023-10-18T12:25:07.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.270+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 --> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0338101f00 con 0x7f03240ac2d0 2023-10-18T12:25:07.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.272+0000 7f032effd700 1 -- 172.21.15.70:0/1003486354 <== osd.10 v2:172.21.15.152:6824/4238500329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0338101f00 con 0x7f03240ac2d0 2023-10-18T12:25:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.272+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] conn(0x7f03240ac2d0 msgr2=0x7f03240ae710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.272+0000 7f03401ce700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] conn(0x7f03240ac2d0 0x7f03240ae710 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03240a15c0 msgr2=0x7f03240a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03240a15c0 0x7f03240a3a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0338195fb0 tx=0x7f032800b410 comp rx=0 tx=0).stop 2023-10-18T12:25:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 msgr2=0x7f0338194aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 --2- 172.21.15.70:0/1003486354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0338068190 0x7f0338194aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0334005910 tx=0x7f0334003c30 comp rx=0 tx=0).stop 2023-10-18T12:25:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f033e76b700 1 -- 172.21.15.70:0/1003486354 reap_dead start 2023-10-18T12:25:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 shutdown_connections 2023-10-18T12:25:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 >> 172.21.15.70:0/1003486354 conn(0x7f0338075790 msgr2=0x7f03380fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 shutdown_connections 2023-10-18T12:25:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:07.275+0000 7f03401ce700 1 -- 172.21.15.70:0/1003486354 wait complete. 2023-10-18T12:25:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:07 smithi186 ceph-mon[135253]: pgmap v283: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:07.597 INFO:teuthology.orchestra.run.smithi070.stdout:81604378689 2023-10-18T12:25:07.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-18T12:25:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:07 smithi152 ceph-mon[134767]: pgmap v283: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:25:08.034 INFO:teuthology.orchestra.run.smithi070.stdout:292057776161 2023-10-18T12:25:08.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-18T12:25:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:08.318 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:08.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 -- 172.21.15.70:0/2683092442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1057c0 msgr2=0x7f0a0c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:08.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 --2- 172.21.15.70:0/2683092442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1057c0 0x7f0a0c105ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009990 tx=0x7f09fc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 -- 172.21.15.70:0/2683092442 shutdown_connections 2023-10-18T12:25:08.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 --2- 172.21.15.70:0/2683092442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a0c101260 0x7f0a0c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 --2- 172.21.15.70:0/2683092442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1057c0 0x7f0a0c105ba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 --2- 172.21.15.70:0/2683092442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a0c1008a0 0x7f0a0c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 -- 172.21.15.70:0/2683092442 >> 172.21.15.70:0/2683092442 conn(0x7f0a0c0779e0 msgr2=0x7f0a0c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:08.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 -- 172.21.15.70:0/2683092442 shutdown_connections 2023-10-18T12:25:08.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.427+0000 7f0a10b37700 1 -- 172.21.15.70:0/2683092442 wait complete. 2023-10-18T12:25:08.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 Processor -- start 2023-10-18T12:25:08.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 -- start start 2023-10-18T12:25:08.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1008a0 0x7f0a0c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:08.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a0c101260 0x7f0a0c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:08.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a0c1057c0 0x7f0a0c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:08.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c19dd70 con 0x7f0a0c1057c0 2023-10-18T12:25:08.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c19dee0 con 0x7f0a0c1008a0 2023-10-18T12:25:08.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.428+0000 7f0a10b37700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c19e0a0 con 0x7f0a0c101260 2023-10-18T12:25:08.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1008a0 0x7f0a0c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:08.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1008a0 0x7f0a0c198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34022/0 (socket says 172.21.15.70:34022) 2023-10-18T12:25:08.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a0c1057c0 0x7f0a0c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:08.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 -- 172.21.15.70:0/153815503 learned_addr learned my addr 172.21.15.70:0/153815503 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:08.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a09d9b700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a0c101260 0x7f0a0c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:08.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 -- 172.21.15.70:0/153815503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a0c101260 msgr2=0x7f0a0c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:08.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a0c101260 0x7f0a0c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 -- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a0c1057c0 msgr2=0x7f0a0c19d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:08.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a0c1057c0 0x7f0a0c19d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0a59c700 1 -- 172.21.15.70:0/153815503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0095f0 con 0x7f0a0c1008a0 2023-10-18T12:25:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.429+0000 7f0a0ad9d700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a0c1057c0 0x7f0a0c19d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.430+0000 7f0a0a59c700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1008a0 0x7f0a0c198e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.430+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01d070 con 0x7f0a0c1008a0 2023-10-18T12:25:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.430+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc022470 con 0x7f0a0c1008a0 2023-10-18T12:25:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.430+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c19e320 con 0x7f0a0c1008a0 2023-10-18T12:25:08.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.430+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc017650 con 0x7f0a0c1008a0 2023-10-18T12:25:08.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.430+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1a5be0 con 0x7f0a0c1008a0 2023-10-18T12:25:08.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.431+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a0c110a20 con 0x7f0a0c1008a0 2023-10-18T12:25:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.433+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f09fc00fad0 con 0x7f0a0c1008a0 2023-10-18T12:25:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.434+0000 7f09fb7fe700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09f40a1560 0x7f09f40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.434+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f09fc0cee80 con 0x7f0a0c1008a0 2023-10-18T12:25:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.434+0000 7f0a09d9b700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09f40a1560 0x7f09f40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.435+0000 7f09fb7fe700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] conn(0x7f09f40ac1f0 0x7f09f40ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.435+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 --> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09f40aed00 con 0x7f09f40ac1f0 2023-10-18T12:25:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.435+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09fc0cf1f0 con 0x7f0a0c1008a0 2023-10-18T12:25:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.435+0000 7f0a09d9b700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09f40a1560 0x7f09f40a3a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c19a3d0 tx=0x7f0a00009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.435+0000 7f0a0ad9d700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] conn(0x7f09f40ac1f0 0x7f09f40ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.436+0000 7f0a0ad9d700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] conn(0x7f09f40ac1f0 0x7f09f40ae630 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.436+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== osd.7 v2:172.21.15.152:6816/2054103643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f09f40aed00 con 0x7f09f40ac1f0 2023-10-18T12:25:08.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.453+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 --> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a0c19e5f0 con 0x7f09f40ac1f0 2023-10-18T12:25:08.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.454+0000 7f09fb7fe700 1 -- 172.21.15.70:0/153815503 <== osd.7 v2:172.21.15.152:6816/2054103643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0a0400f040 con 0x7f09f40ac1f0 2023-10-18T12:25:08.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.454+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] conn(0x7f09f40ac1f0 msgr2=0x7f09f40ae630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:08.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.454+0000 7f0a10b37700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] conn(0x7f09f40ac1f0 0x7f09f40ae630 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:08.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.454+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09f40a1560 msgr2=0x7f09f40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:08.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f09f40a1560 0x7f09f40a3a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c19a3d0 tx=0x7f0a00009260 comp rx=0 tx=0).stop 2023-10-18T12:25:08.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1008a0 msgr2=0x7f0a0c198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:08.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 --2- 172.21.15.70:0/153815503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a0c1008a0 0x7f0a0c198e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc005740 comp rx=0 tx=0).stop 2023-10-18T12:25:08.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a0ad9d700 1 -- 172.21.15.70:0/153815503 reap_dead start 2023-10-18T12:25:08.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 shutdown_connections 2023-10-18T12:25:08.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 >> 172.21.15.70:0/153815503 conn(0x7f0a0c0779e0 msgr2=0x7f0a0c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:08.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 shutdown_connections 2023-10-18T12:25:08.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:08.455+0000 7f0a10b37700 1 -- 172.21.15.70:0/153815503 wait complete. 2023-10-18T12:25:08.469 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:08.561 INFO:teuthology.orchestra.run.smithi070.stdout:313532612635 2023-10-18T12:25:08.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-18T12:25:09.009 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:09.048 INFO:teuthology.orchestra.run.smithi070.stdout:223338299433 2023-10-18T12:25:09.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-18T12:25:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:09 smithi070 ceph-mon[98639]: pgmap v284: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:25:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:09 smithi186 ceph-mon[135253]: pgmap v284: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:25:09.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:09 smithi152 ceph-mon[134767]: pgmap v284: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:25:10.596 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:10.597 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 -- 172.21.15.70:0/3140063297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c4101260 msgr2=0x7f03c410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/3140063297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c4101260 0x7f03c410d650 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00b0a0 tx=0x7f03bc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 -- 172.21.15.70:0/3140063297 shutdown_connections 2023-10-18T12:25:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/3140063297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c4101260 0x7f03c410d650 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/3140063297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c41057c0 0x7f03c4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/3140063297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03c41008a0 0x7f03c4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.114+0000 7f03c9bf0700 1 -- 172.21.15.70:0/3140063297 >> 172.21.15.70:0/3140063297 conn(0x7f03c40779e0 msgr2=0x7f03c4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.115+0000 7f03c9bf0700 1 -- 172.21.15.70:0/3140063297 shutdown_connections 2023-10-18T12:25:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.115+0000 7f03c9bf0700 1 -- 172.21.15.70:0/3140063297 wait complete. 2023-10-18T12:25:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 Processor -- start 2023-10-18T12:25:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 -- start start 2023-10-18T12:25:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 0x7f03c4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c4101260 0x7f03c4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03c41057c0 0x7f03c419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c419de40 con 0x7f03c41008a0 2023-10-18T12:25:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c419dfb0 con 0x7f03c4101260 2023-10-18T12:25:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c9bf0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c419e170 con 0x7f03c41057c0 2023-10-18T12:25:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 0x7f03c4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 0x7f03c4198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34776/0 (socket says 172.21.15.70:34776) 2023-10-18T12:25:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.116+0000 7f03c37fe700 1 -- 172.21.15.70:0/389979212 learned_addr learned my addr 172.21.15.70:0/389979212 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c2ffd700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c4101260 0x7f03c4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c3fff700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03c41057c0 0x7f03c419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c37fe700 1 -- 172.21.15.70:0/389979212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03c41057c0 msgr2=0x7f03c419d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c37fe700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03c41057c0 0x7f03c419d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c37fe700 1 -- 172.21.15.70:0/389979212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c4101260 msgr2=0x7f03c4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c37fe700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c4101260 0x7f03c4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c37fe700 1 -- 172.21.15.70:0/389979212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc009d00 con 0x7f03c41008a0 2023-10-18T12:25:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c37fe700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 0x7f03c4198f40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e970 tx=0x7f03b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4009790 con 0x7f03c41008a0 2023-10-18T12:25:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b400c7e0 con 0x7f03c41008a0 2023-10-18T12:25:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c419e450 con 0x7f03c41008a0 2023-10-18T12:25:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.117+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4003c70 con 0x7f03c41008a0 2023-10-18T12:25:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.118+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c41a5cb0 con 0x7f03c41008a0 2023-10-18T12:25:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.119+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c4101f30 con 0x7f03c41008a0 2023-10-18T12:25:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.121+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f03b400cc50 con 0x7f03c41008a0 2023-10-18T12:25:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.123+0000 7f03c0ff9700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03ac0a15e0 0x7f03ac0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.123+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f03b4014070 con 0x7f03c41008a0 2023-10-18T12:25:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.124+0000 7f03c2ffd700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03ac0a15e0 0x7f03ac0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.124+0000 7f03c2ffd700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03ac0a15e0 0x7f03ac0a3aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03c419a4a0 tx=0x7f03b8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.127+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f03b40964a0 con 0x7f03c41008a0 2023-10-18T12:25:11.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.349+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f03c404e4c0 con 0x7f03c41008a0 2023-10-18T12:25:11.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.350+0000 7f03c0ff9700 1 -- 172.21.15.70:0/389979212 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03b4003dd0 con 0x7f03c41008a0 2023-10-18T12:25:11.353 INFO:teuthology.orchestra.run.smithi070.stdout:201863462957 2023-10-18T12:25:11.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03ac0a15e0 msgr2=0x7f03ac0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03ac0a15e0 0x7f03ac0a3aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03c419a4a0 tx=0x7f03b8009260 comp rx=0 tx=0).stop 2023-10-18T12:25:11.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 msgr2=0x7f03c4198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 0x7f03c4198f40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e970 tx=0x7f03b400ec80 comp rx=0 tx=0).stop 2023-10-18T12:25:11.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 shutdown_connections 2023-10-18T12:25:11.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03c41057c0 0x7f03c419d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c41008a0 0x7f03c4198f40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03ac0a15e0 0x7f03ac0a3aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 --2- 172.21.15.70:0/389979212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03c4101260 0x7f03c4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 >> 172.21.15.70:0/389979212 conn(0x7f03c40779e0 msgr2=0x7f03c410ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:11.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 shutdown_connections 2023-10-18T12:25:11.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.352+0000 7f03c9bf0700 1 -- 172.21.15.70:0/389979212 wait complete. 2023-10-18T12:25:11.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:11 smithi186 ceph-mon[135253]: pgmap v285: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-18T12:25:11.522 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:11.524 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:11.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:11 smithi070 ceph-mon[98639]: pgmap v285: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-18T12:25:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 -- 172.21.15.70:0/1544558089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd841008a0 msgr2=0x7efd84100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 --2- 172.21.15.70:0/1544558089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd841008a0 0x7efd84100d20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efd78009990 tx=0x7efd78009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 -- 172.21.15.70:0/1544558089 shutdown_connections 2023-10-18T12:25:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 --2- 172.21.15.70:0/1544558089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84101260 0x7efd8410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 --2- 172.21.15.70:0/1544558089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841057c0 0x7efd84105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 --2- 172.21.15.70:0/1544558089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd841008a0 0x7efd84100d20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 -- 172.21.15.70:0/1544558089 >> 172.21.15.70:0/1544558089 conn(0x7efd840779e0 msgr2=0x7efd84077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 -- 172.21.15.70:0/1544558089 shutdown_connections 2023-10-18T12:25:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.647+0000 7efd89046700 1 -- 172.21.15.70:0/1544558089 wait complete. 2023-10-18T12:25:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.648+0000 7efd89046700 1 Processor -- start 2023-10-18T12:25:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 -- start start 2023-10-18T12:25:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 0x7efd8419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd84101260 0x7efd8419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd841057c0 0x7efd84196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd84197100 con 0x7efd841057c0 2023-10-18T12:25:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd84197270 con 0x7efd84101260 2023-10-18T12:25:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd89046700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd84197430 con 0x7efd841008a0 2023-10-18T12:25:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd82d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 0x7efd8419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd82d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 0x7efd8419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:45644/0 (socket says 172.21.15.70:45644) 2023-10-18T12:25:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd8359e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd841057c0 0x7efd84196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.649+0000 7efd82d9d700 1 -- 172.21.15.70:0/378239007 learned_addr learned my addr 172.21.15.70:0/378239007 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd8259c700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd84101260 0x7efd8419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd82d9d700 1 -- 172.21.15.70:0/378239007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd84101260 msgr2=0x7efd8419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd82d9d700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd84101260 0x7efd8419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd82d9d700 1 -- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd841057c0 msgr2=0x7efd84196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd82d9d700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd841057c0 0x7efd84196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd82d9d700 1 -- 172.21.15.70:0/378239007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74009520 con 0x7efd841008a0 2023-10-18T12:25:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd82d9d700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 0x7efd8419c910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd7400ea50 tx=0x7efd7400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400cbd0 con 0x7efd841008a0 2023-10-18T12:25:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd780095f0 con 0x7efd841008a0 2023-10-18T12:25:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.650+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd7400cd30 con 0x7efd841008a0 2023-10-18T12:25:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.651+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74018680 con 0x7efd841008a0 2023-10-18T12:25:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.651+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd84197900 con 0x7efd841008a0 2023-10-18T12:25:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.652+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd84108da0 con 0x7efd841008a0 2023-10-18T12:25:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.654+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7efd74010430 con 0x7efd841008a0 2023-10-18T12:25:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.655+0000 7efd6bfff700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd6c0a1560 0x7efd6c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.655+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7efd74014070 con 0x7efd841008a0 2023-10-18T12:25:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.655+0000 7efd8259c700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd6c0a1560 0x7efd6c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.656+0000 7efd8259c700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd6c0a1560 0x7efd6c0a3a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd84197c10 tx=0x7efd7800b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.657+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd740969d0 con 0x7efd841008a0 2023-10-18T12:25:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:11 smithi152 ceph-mon[134767]: pgmap v285: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-18T12:25:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.891+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7efd84197f10 con 0x7efd841008a0 2023-10-18T12:25:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.892+0000 7efd6bfff700 1 -- 172.21.15.70:0/378239007 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd74099ff0 con 0x7efd841008a0 2023-10-18T12:25:11.895 INFO:teuthology.orchestra.run.smithi070.stdout:257698037797 2023-10-18T12:25:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.894+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd6c0a1560 msgr2=0x7efd6c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.894+0000 7efd89046700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd6c0a1560 0x7efd6c0a3a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd84197c10 tx=0x7efd7800b260 comp rx=0 tx=0).stop 2023-10-18T12:25:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.894+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 msgr2=0x7efd8419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.894+0000 7efd89046700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 0x7efd8419c910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd7400ea50 tx=0x7efd7400ed60 comp rx=0 tx=0).stop 2023-10-18T12:25:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 shutdown_connections 2023-10-18T12:25:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd841057c0 0x7efd84196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd841008a0 0x7efd8419c910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd6c0a1560 0x7efd6c0a3a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 --2- 172.21.15.70:0/378239007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd84101260 0x7efd8419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 >> 172.21.15.70:0/378239007 conn(0x7efd840779e0 msgr2=0x7efd840fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:11.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 shutdown_connections 2023-10-18T12:25:11.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:11.895+0000 7efd89046700 1 -- 172.21.15.70:0/378239007 wait complete. 2023-10-18T12:25:12.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:12 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/389979212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-18T12:25:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:12 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/378239007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-18T12:25:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:12 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:25:12.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/389979212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-18T12:25:12.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/378239007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-18T12:25:12.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:12 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:25:12.541 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462955 got 201863462957 for osd.6 2023-10-18T12:25:12.541 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:12.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:12 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/389979212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-18T12:25:12.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:12 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/378239007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-18T12:25:12.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:12 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:25:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2014321857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8681057c0 msgr2=0x7fd868105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2014321857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8681057c0 0x7fd868105ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd858009990 tx=0x7fd858009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2014321857 shutdown_connections 2023-10-18T12:25:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2014321857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd868101260 0x7fd86810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2014321857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8681057c0 0x7fd868105ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2014321857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8681008a0 0x7fd868100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.067+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2014321857 >> 172.21.15.70:0/2014321857 conn(0x7fd8680779e0 msgr2=0x7fd868077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:13.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.068+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2014321857 shutdown_connections 2023-10-18T12:25:13.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.068+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2014321857 wait complete. 2023-10-18T12:25:13.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.069+0000 7fd86e6f1700 1 Processor -- start 2023-10-18T12:25:13.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.069+0000 7fd86e6f1700 1 -- start start 2023-10-18T12:25:13.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86e6f1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8681008a0 0x7fd868198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86e6f1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 0x7fd8681994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86e6f1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8681057c0 0x7fd86819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86e6f1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86819de10 con 0x7fd8681057c0 2023-10-18T12:25:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86e6f1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86819df80 con 0x7fd868101260 2023-10-18T12:25:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86e6f1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86819e140 con 0x7fd8681008a0 2023-10-18T12:25:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd8677fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 0x7fd8681994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd8677fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 0x7fd8681994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34062/0 (socket says 172.21.15.70:34062) 2023-10-18T12:25:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd8677fe700 1 -- 172.21.15.70:0/2591861212 learned_addr learned my addr 172.21.15.70:0/2591861212 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.070+0000 7fd86cc8e700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8681057c0 0x7fd86819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd867fff700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8681008a0 0x7fd868198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8677fe700 1 -- 172.21.15.70:0/2591861212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8681008a0 msgr2=0x7fd868198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8677fe700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8681008a0 0x7fd868198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8677fe700 1 -- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8681057c0 msgr2=0x7fd86819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8677fe700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8681057c0 0x7fd86819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8677fe700 1 -- 172.21.15.70:0/2591861212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580095f0 con 0x7fd868101260 2023-10-18T12:25:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd86cc8e700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8681057c0 0x7fd86819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8677fe700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 0x7fd8681994a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e9c0 tx=0x7fd85c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.071+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0097b0 con 0x7fd868101260 2023-10-18T12:25:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.072+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd85c004cf0 con 0x7fd868101260 2023-10-18T12:25:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.072+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86819e420 con 0x7fd868101260 2023-10-18T12:25:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.072+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c005600 con 0x7fd868101260 2023-10-18T12:25:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.072+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8681a5c80 con 0x7fd868101260 2023-10-18T12:25:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.073+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd868101f30 con 0x7fd868101260 2023-10-18T12:25:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.076+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd85c00cc60 con 0x7fd868101260 2023-10-18T12:25:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.077+0000 7fd8657fa700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd8500a15e0 0x7fd8500a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.077+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fd85c014070 con 0x7fd868101260 2023-10-18T12:25:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.077+0000 7fd867fff700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd8500a15e0 0x7fd8500a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.078+0000 7fd867fff700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd8500a15e0 0x7fd8500a3aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd85800b2e0 tx=0x7fd858005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:13.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.081+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd85c096630 con 0x7fd868101260 2023-10-18T12:25:13.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 -- 172.21.15.70:0/85649333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 msgr2=0x7f2b7c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 --2- 172.21.15.70:0/85649333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c0733e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009990 tx=0x7f2b68009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 -- 172.21.15.70:0/85649333 shutdown_connections 2023-10-18T12:25:13.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 --2- 172.21.15.70:0/85649333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 0x7f2b7c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 --2- 172.21.15.70:0/85649333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b7c074770 0x7f2b7c072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 --2- 172.21.15.70:0/85649333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c0733e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 -- 172.21.15.70:0/85649333 >> 172.21.15.70:0/85649333 conn(0x7f2b7c0779e0 msgr2=0x7f2b7c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:13.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 -- 172.21.15.70:0/85649333 shutdown_connections 2023-10-18T12:25:13.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.250+0000 7f2b82e38700 1 -- 172.21.15.70:0/85649333 wait complete. 2023-10-18T12:25:13.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.251+0000 7f2b82e38700 1 Processor -- start 2023-10-18T12:25:13.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 -- start start 2023-10-18T12:25:13.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b7c074770 0x7f2b7c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 0x7f2b7c1a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1a21d0 con 0x7f2b7c074770 2023-10-18T12:25:13.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1a2340 con 0x7f2b7c072f60 2023-10-18T12:25:13.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b82e38700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1a2500 con 0x7f2b7c111570 2023-10-18T12:25:13.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.252+0000 7f2b80bd4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b80bd4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c19d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34080/0 (socket says 172.21.15.70:34080) 2023-10-18T12:25:13.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b7bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b7c074770 0x7f2b7c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b80bd4700 1 -- 172.21.15.70:0/2746159534 learned_addr learned my addr 172.21.15.70:0/2746159534 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:13.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 0x7f2b7c1a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 -- 172.21.15.70:0/2746159534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 msgr2=0x7f2b7c19d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c19d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 -- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b7c074770 msgr2=0x7f2b7c19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b7c074770 0x7f2b7c19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 -- 172.21.15.70:0/2746159534 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680095f0 con 0x7f2b7c111570 2023-10-18T12:25:13.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b80bd4700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:13.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b813d5700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 0x7f2b7c1a1bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400ef90 tx=0x7f2b7400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:13.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.253+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74018070 con 0x7f2b7c111570 2023-10-18T12:25:13.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.254+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c1a2780 con 0x7f2b7c111570 2023-10-18T12:25:13.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.254+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b7400f040 con 0x7f2b7c111570 2023-10-18T12:25:13.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.254+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400cb80 con 0x7f2b7c111570 2023-10-18T12:25:13.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.254+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c1aa040 con 0x7f2b7c111570 2023-10-18T12:25:13.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.255+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c0740b0 con 0x7f2b7c111570 2023-10-18T12:25:13.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.258+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f2b74003a00 con 0x7f2b7c111570 2023-10-18T12:25:13.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.259+0000 7f2b79ffb700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b6c0a1560 0x7f2b6c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:13.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.260+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2b740ccc50 con 0x7f2b7c111570 2023-10-18T12:25:13.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.260+0000 7f2b80bd4700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b6c0a1560 0x7f2b6c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:13.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.260+0000 7f2b80bd4700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b6c0a1560 0x7f2b6c0a3a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b70011550 tx=0x7f2b70009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:13.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.264+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2b74095f10 con 0x7f2b7c111570 2023-10-18T12:25:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.299+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd86804e4c0 con 0x7fd868101260 2023-10-18T12:25:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.300+0000 7fd8657fa700 1 -- 172.21.15.70:0/2591861212 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd85c099c50 con 0x7fd868101260 2023-10-18T12:25:13.303 INFO:teuthology.orchestra.run.smithi070.stdout:60129542212 2023-10-18T12:25:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd8500a15e0 msgr2=0x7fd8500a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd8500a15e0 0x7fd8500a3aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd85800b2e0 tx=0x7fd858005fd0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 msgr2=0x7fd8681994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 0x7fd8681994a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e9c0 tx=0x7fd85c00ecd0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 shutdown_connections 2023-10-18T12:25:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8681057c0 0x7fd86819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8681008a0 0x7fd868198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd8500a15e0 0x7fd8500a3aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 --2- 172.21.15.70:0/2591861212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd868101260 0x7fd8681994a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 >> 172.21.15.70:0/2591861212 conn(0x7fd8680779e0 msgr2=0x7fd86810bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 shutdown_connections 2023-10-18T12:25:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.302+0000 7fd86e6f1700 1 -- 172.21.15.70:0/2591861212 wait complete. 2023-10-18T12:25:13.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:13 smithi070 ceph-mon[98639]: pgmap v286: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:13.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:13 smithi186 ceph-mon[135253]: pgmap v286: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:13.489 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:13.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.509+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b7c19e5e0 con 0x7f2b7c111570 2023-10-18T12:25:13.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.510+0000 7f2b79ffb700 1 -- 172.21.15.70:0/2746159534 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b74099530 con 0x7f2b7c111570 2023-10-18T12:25:13.513 INFO:teuthology.orchestra.run.smithi070.stdout:38654705737 2023-10-18T12:25:13.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.513+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b6c0a1560 msgr2=0x7f2b6c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.513+0000 7f2b82e38700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b6c0a1560 0x7f2b6c0a3a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b70011550 tx=0x7f2b70009260 comp rx=0 tx=0).stop 2023-10-18T12:25:13.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.513+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 msgr2=0x7f2b7c1a1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:13.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.513+0000 7f2b82e38700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 0x7f2b7c1a1bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400ef90 tx=0x7f2b7400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 shutdown_connections 2023-10-18T12:25:13.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c111570 0x7f2b7c1a1bc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b6c0a1560 0x7f2b6c0a3a20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b7c072f60 0x7f2b7c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 --2- 172.21.15.70:0/2746159534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b7c074770 0x7f2b7c19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 >> 172.21.15.70:0/2746159534 conn(0x7f2b7c0779e0 msgr2=0x7f2b7c1101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:13.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 shutdown_connections 2023-10-18T12:25:13.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:13.514+0000 7f2b82e38700 1 -- 172.21.15.70:0/2746159534 wait complete. 2023-10-18T12:25:13.519 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:13 smithi152 ceph-mon[134767]: pgmap v286: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:14.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037795 got 257698037797 for osd.8 2023-10-18T12:25:14.371 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2591861212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-18T12:25:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2746159534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-18T12:25:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-18T12:25:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-18T12:25:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-18T12:25:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-18T12:25:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-10-18T12:25:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-10-18T12:25:14.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-10-18T12:25:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1]}]: dispatch 2023-10-18T12:25:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-18T12:25:14.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:14 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-10-18T12:25:14.530 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2591861212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-18T12:25:14.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2746159534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-18T12:25:14.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-18T12:25:14.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-18T12:25:14.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-18T12:25:14.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-18T12:25:14.532 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-10-18T12:25:14.532 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-10-18T12:25:14.532 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-10-18T12:25:14.532 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1]}]: dispatch 2023-10-18T12:25:14.532 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-18T12:25:14.533 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:14 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-10-18T12:25:14.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705734 got 38654705737 for osd.0 2023-10-18T12:25:14.534 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:14.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2591861212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-18T12:25:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2746159534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-18T12:25:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-18T12:25:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-18T12:25:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-18T12:25:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-18T12:25:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-10-18T12:25:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-10-18T12:25:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-10-18T12:25:14.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1]}]: dispatch 2023-10-18T12:25:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-18T12:25:14.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:14 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-10-18T12:25:14.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542210 got 60129542212 for osd.1 2023-10-18T12:25:14.897 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: pgmap v287: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-18T12:25:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-18T12:25:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-18T12:25:15.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-18T12:25:15.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-10-18T12:25:15.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-10-18T12:25:15.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-10-18T12:25:15.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1]}]': finished 2023-10-18T12:25:15.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-18T12:25:15.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-10-18T12:25:15.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: osdmap e90: 12 total, 12 up, 12 in 2023-10-18T12:25:15.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: pgmap v287: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-18T12:25:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-18T12:25:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-18T12:25:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-18T12:25:15.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-10-18T12:25:15.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-10-18T12:25:15.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-10-18T12:25:15.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1]}]': finished 2023-10-18T12:25:15.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-18T12:25:15.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-10-18T12:25:15.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: osdmap e90: 12 total, 12 up, 12 in 2023-10-18T12:25:15.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: pgmap v287: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-18T12:25:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-18T12:25:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-18T12:25:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-18T12:25:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-10-18T12:25:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-10-18T12:25:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-10-18T12:25:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1]}]': finished 2023-10-18T12:25:15.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-18T12:25:15.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-10-18T12:25:15.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: osdmap e90: 12 total, 12 up, 12 in 2023-10-18T12:25:15.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:15.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:16.009 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:16.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:16 smithi070 ceph-mon[98639]: osdmap e91: 12 total, 12 up, 12 in 2023-10-18T12:25:16.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:16 smithi070 ceph-mon[98639]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2023-10-18T12:25:16.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:16 smithi152 ceph-mon[134767]: osdmap e91: 12 total, 12 up, 12 in 2023-10-18T12:25:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:16 smithi152 ceph-mon[134767]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2023-10-18T12:25:16.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:16 smithi186 ceph-mon[135253]: osdmap e91: 12 total, 12 up, 12 in 2023-10-18T12:25:16.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:16 smithi186 ceph-mon[135253]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2023-10-18T12:25:16.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 -- 172.21.15.70:0/833133397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c881008a0 msgr2=0x7f7c88100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:16.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/833133397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c881008a0 0x7f7c88100d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009990 tx=0x7f7c7c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:16.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 -- 172.21.15.70:0/833133397 shutdown_connections 2023-10-18T12:25:16.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/833133397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c88101260 0x7f7c8810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:16.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/833133397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 0x7f7c88105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:16.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/833133397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c881008a0 0x7f7c88100d20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:16.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 -- 172.21.15.70:0/833133397 >> 172.21.15.70:0/833133397 conn(0x7f7c880779e0 msgr2=0x7f7c88077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:16.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 -- 172.21.15.70:0/833133397 shutdown_connections 2023-10-18T12:25:16.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.776+0000 7f7c8d96a700 1 -- 172.21.15.70:0/833133397 wait complete. 2023-10-18T12:25:16.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.777+0000 7f7c8d96a700 1 Processor -- start 2023-10-18T12:25:16.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.777+0000 7f7c8d96a700 1 -- start start 2023-10-18T12:25:16.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c8d96a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c881008a0 0x7f7c88198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:16.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c8d96a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 0x7f7c88199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:16.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c8d96a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 0x7f7c8819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:16.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c8d96a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8819dd00 con 0x7f7c88101260 2023-10-18T12:25:16.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c8d96a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8819de70 con 0x7f7c881057c0 2023-10-18T12:25:16.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c8d96a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8819e030 con 0x7f7c881008a0 2023-10-18T12:25:16.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c867fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 0x7f7c88199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:16.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c86ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c881008a0 0x7f7c88198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:16.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c867fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 0x7f7c88199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34856/0 (socket says 172.21.15.70:34856) 2023-10-18T12:25:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c867fc700 1 -- 172.21.15.70:0/3398157239 learned_addr learned my addr 172.21.15.70:0/3398157239 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.778+0000 7f7c877fe700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 0x7f7c8819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c86ffd700 1 -- 172.21.15.70:0/3398157239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 msgr2=0x7f7c8819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c86ffd700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 0x7f7c8819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c86ffd700 1 -- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 msgr2=0x7f7c88199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c86ffd700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 0x7f7c88199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c86ffd700 1 -- 172.21.15.70:0/3398157239 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0095f0 con 0x7f7c881008a0 2023-10-18T12:25:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c877fe700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 0x7f7c8819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c867fc700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 0x7f7c88199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:16.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c86ffd700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c881008a0 0x7f7c88198e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9c0 tx=0x7f7c7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:16.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800cbd0 con 0x7f7c881008a0 2023-10-18T12:25:16.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8819e310 con 0x7f7c881008a0 2023-10-18T12:25:16.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c7800eeb0 con 0x7f7c881008a0 2023-10-18T12:25:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78003c70 con 0x7f7c881008a0 2023-10-18T12:25:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.779+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c881a5b70 con 0x7f7c881008a0 2023-10-18T12:25:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.780+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c88101f30 con 0x7f7c881008a0 2023-10-18T12:25:16.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.782+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f7c78003e70 con 0x7f7c881008a0 2023-10-18T12:25:16.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.784+0000 7f7c8c968700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7c700a15e0 0x7f7c700a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:16.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.784+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f7c78014070 con 0x7f7c881008a0 2023-10-18T12:25:16.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.784+0000 7f7c867fc700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7c700a15e0 0x7f7c700a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:16.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.784+0000 7f7c867fc700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7c700a15e0 0x7f7c700a3aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c8819a360 tx=0x7f7c7c005bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:16.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:16.786+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7c78096700 con 0x7f7c881008a0 2023-10-18T12:25:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.017+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c8819a1b0 con 0x7f7c881008a0 2023-10-18T12:25:17.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.018+0000 7f7c8c968700 1 -- 172.21.15.70:0/3398157239 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c78099d20 con 0x7f7c881008a0 2023-10-18T12:25:17.021 INFO:teuthology.orchestra.run.smithi070.stdout:81604378691 2023-10-18T12:25:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7c700a15e0 msgr2=0x7f7c700a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7c700a15e0 0x7f7c700a3aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c8819a360 tx=0x7f7c7c005bc0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c881008a0 msgr2=0x7f7c88198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c881008a0 0x7f7c88198e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9c0 tx=0x7f7c7800ecd0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 shutdown_connections 2023-10-18T12:25:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7c700a15e0 0x7f7c700a3aa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c881057c0 0x7f7c8819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c881008a0 0x7f7c88198e00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 --2- 172.21.15.70:0/3398157239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c88101260 0x7f7c88199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 >> 172.21.15.70:0/3398157239 conn(0x7f7c880779e0 msgr2=0x7f7c8810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 shutdown_connections 2023-10-18T12:25:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.020+0000 7f7c8d96a700 1 -- 172.21.15.70:0/3398157239 wait complete. 2023-10-18T12:25:17.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 -- 172.21.15.70:0/3323585806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1008a0 msgr2=0x7f67dc100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3323585806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1008a0 0x7f67dc100d20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009990 tx=0x7f67d0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 -- 172.21.15.70:0/3323585806 shutdown_connections 2023-10-18T12:25:17.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3323585806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3323585806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67dc1057c0 0x7f67dc105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3323585806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1008a0 0x7f67dc100d20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 -- 172.21.15.70:0/3323585806 >> 172.21.15.70:0/3323585806 conn(0x7f67dc0779e0 msgr2=0x7f67dc077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 -- 172.21.15.70:0/3323585806 shutdown_connections 2023-10-18T12:25:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.447+0000 7f67e29a6700 1 -- 172.21.15.70:0/3323585806 wait complete. 2023-10-18T12:25:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.448+0000 7f67e29a6700 1 Processor -- start 2023-10-18T12:25:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 -- start start 2023-10-18T12:25:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67dc1008a0 0x7f67dc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 0x7f67dc19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc19dd30 con 0x7f67dc1057c0 2023-10-18T12:25:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc19dea0 con 0x7f67dc1008a0 2023-10-18T12:25:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e29a6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc19e060 con 0x7f67dc101260 2023-10-18T12:25:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e0f43700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 0x7f67dc19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67db7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e0f43700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 0x7f67dc19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34882/0 (socket says 172.21.15.70:34882) 2023-10-18T12:25:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67db7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:45706/0 (socket says 172.21.15.70:45706) 2023-10-18T12:25:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67e0f43700 1 -- 172.21.15.70:0/3116834937 learned_addr learned my addr 172.21.15.70:0/3116834937 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.449+0000 7f67dbfff700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67dc1008a0 0x7f67dc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67db7fe700 1 -- 172.21.15.70:0/3116834937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67dc1008a0 msgr2=0x7f67dc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67db7fe700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67dc1008a0 0x7f67dc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67db7fe700 1 -- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 msgr2=0x7f67dc19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67db7fe700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 0x7f67dc19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67db7fe700 1 -- 172.21.15.70:0/3116834937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00095f0 con 0x7f67dc101260 2023-10-18T12:25:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67e0f43700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 0x7f67dc19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67db7fe700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc199370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098e0 tx=0x7f67d0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d001d070 con 0x7f67dc101260 2023-10-18T12:25:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.450+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc19e2e0 con 0x7f67dc101260 2023-10-18T12:25:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.451+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc1a5ba0 con 0x7f67dc101260 2023-10-18T12:25:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.451+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d002b3b0 con 0x7f67dc101260 2023-10-18T12:25:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.451+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000e640 con 0x7f67dc101260 2023-10-18T12:25:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.452+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c80052f0 con 0x7f67dc101260 2023-10-18T12:25:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.453+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f67d000fb10 con 0x7f67dc101260 2023-10-18T12:25:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.455+0000 7f67d97fa700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67c40a15e0 0x7f67c40a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.455+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f67d00ce410 con 0x7f67dc101260 2023-10-18T12:25:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.455+0000 7f67dbfff700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67c40a15e0 0x7f67c40a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.456+0000 7f67dbfff700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67c40a15e0 0x7f67c40a3aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f67dc101930 tx=0x7f67cc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.457+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f67d0097670 con 0x7f67dc101260 2023-10-18T12:25:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:17 smithi070 ceph-mon[98639]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-10-18T12:25:17.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:17 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3398157239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-18T12:25:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.666+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f67c8005160 con 0x7f67dc101260 2023-10-18T12:25:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.666+0000 7f67d97fa700 1 -- 172.21.15.70:0/3116834937 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67d0027090 con 0x7f67dc101260 2023-10-18T12:25:17.670 INFO:teuthology.orchestra.run.smithi070.stdout:146028888121 2023-10-18T12:25:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:17 smithi152 ceph-mon[134767]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-10-18T12:25:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:17 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3398157239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-18T12:25:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67c40a15e0 msgr2=0x7f67c40a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67c40a15e0 0x7f67c40a3aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f67dc101930 tx=0x7f67cc009260 comp rx=0 tx=0).stop 2023-10-18T12:25:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 msgr2=0x7f67dc199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc199370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098e0 tx=0x7f67d0005740 comp rx=0 tx=0).stop 2023-10-18T12:25:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 shutdown_connections 2023-10-18T12:25:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc1057c0 0x7f67dc19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f67c40a15e0 0x7f67c40a3aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67dc1008a0 0x7f67dc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 --2- 172.21.15.70:0/3116834937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc101260 0x7f67dc199370 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 >> 172.21.15.70:0/3116834937 conn(0x7f67dc0779e0 msgr2=0x7f67dc10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 shutdown_connections 2023-10-18T12:25:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:17.669+0000 7f67e29a6700 1 -- 172.21.15.70:0/3116834937 wait complete. 2023-10-18T12:25:17.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:17 smithi186 ceph-mon[135253]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-10-18T12:25:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:17 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3398157239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-18T12:25:17.971 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378689 got 81604378691 for osd.2 2023-10-18T12:25:17.972 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 -- 172.21.15.70:0/1290771193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01057c0 msgr2=0x7f93f0105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 --2- 172.21.15.70:0/1290771193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01057c0 0x7f93f0105ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009990 tx=0x7f93e0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 -- 172.21.15.70:0/1290771193 shutdown_connections 2023-10-18T12:25:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 --2- 172.21.15.70:0/1290771193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93f0101260 0x7f93f010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 --2- 172.21.15.70:0/1290771193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01057c0 0x7f93f0105ba0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 --2- 172.21.15.70:0/1290771193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01008a0 0x7f93f0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 -- 172.21.15.70:0/1290771193 >> 172.21.15.70:0/1290771193 conn(0x7f93f00779e0 msgr2=0x7f93f0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.098+0000 7f93f6813700 1 -- 172.21.15.70:0/1290771193 shutdown_connections 2023-10-18T12:25:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.099+0000 7f93f6813700 1 -- 172.21.15.70:0/1290771193 wait complete. 2023-10-18T12:25:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.099+0000 7f93f6813700 1 Processor -- start 2023-10-18T12:25:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 -- start start 2023-10-18T12:25:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 0x7f93f0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93f0101260 0x7f93f0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01057c0 0x7f93f0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f01a4f90 con 0x7f93f01008a0 2023-10-18T12:25:18.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f01a5100 con 0x7f93f0101260 2023-10-18T12:25:18.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93f6813700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f01a52c0 con 0x7f93f01057c0 2023-10-18T12:25:18.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93effff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 0x7f93f0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93effff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 0x7f93f0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37066/0 (socket says 172.21.15.70:37066) 2023-10-18T12:25:18.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.100+0000 7f93effff700 1 -- 172.21.15.70:0/3983955566 learned_addr learned my addr 172.21.15.70:0/3983955566 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:18.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93f4db0700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01057c0 0x7f93f0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93ef7fe700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93f0101260 0x7f93f0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93effff700 1 -- 172.21.15.70:0/3983955566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01057c0 msgr2=0x7f93f0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93effff700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01057c0 0x7f93f0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93effff700 1 -- 172.21.15.70:0/3983955566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93f0101260 msgr2=0x7f93f0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93effff700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93f0101260 0x7f93f0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93effff700 1 -- 172.21.15.70:0/3983955566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e00095f0 con 0x7f93f01008a0 2023-10-18T12:25:18.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93f4db0700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01057c0 0x7f93f0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:18.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93effff700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 0x7f93f0072af0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:18.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e001d070 con 0x7f93f01008a0 2023-10-18T12:25:18.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e0022470 con 0x7f93f01008a0 2023-10-18T12:25:18.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f01a5540 con 0x7f93f01008a0 2023-10-18T12:25:18.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.101+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0017650 con 0x7f93f01008a0 2023-10-18T12:25:18.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.102+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f01a5950 con 0x7f93f01008a0 2023-10-18T12:25:18.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.103+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f0108d80 con 0x7f93f01008a0 2023-10-18T12:25:18.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.105+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f93e000fad0 con 0x7f93f01008a0 2023-10-18T12:25:18.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.106+0000 7f93ed7fa700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93d80a1560 0x7f93d80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.107+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f93e00ceda0 con 0x7f93f01008a0 2023-10-18T12:25:18.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.107+0000 7f93ef7fe700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93d80a1560 0x7f93d80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.107+0000 7f93ef7fe700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93d80a1560 0x7f93d80a3a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f93f0074220 tx=0x7f93e4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:18.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.109+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f93e0097ee0 con 0x7f93f01008a0 2023-10-18T12:25:18.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 -- 172.21.15.70:0/1967123652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c1057c0 msgr2=0x7fcd1c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 --2- 172.21.15.70:0/1967123652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c1057c0 0x7fcd1c105ba0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009990 tx=0x7fcd0c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 -- 172.21.15.70:0/1967123652 shutdown_connections 2023-10-18T12:25:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 --2- 172.21.15.70:0/1967123652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd1c101260 0x7fcd1c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 --2- 172.21.15.70:0/1967123652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c1057c0 0x7fcd1c105ba0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 --2- 172.21.15.70:0/1967123652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1c1008a0 0x7fcd1c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 -- 172.21.15.70:0/1967123652 >> 172.21.15.70:0/1967123652 conn(0x7fcd1c0779e0 msgr2=0x7fcd1c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 -- 172.21.15.70:0/1967123652 shutdown_connections 2023-10-18T12:25:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.264+0000 7fcd21d61700 1 -- 172.21.15.70:0/1967123652 wait complete. 2023-10-18T12:25:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.265+0000 7fcd21d61700 1 Processor -- start 2023-10-18T12:25:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 -- start start 2023-10-18T12:25:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1c1008a0 0x7fcd1c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 0x7fcd1c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd1c1057c0 0x7fcd1c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1970e0 con 0x7fcd1c101260 2023-10-18T12:25:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c197250 con 0x7fcd1c1057c0 2023-10-18T12:25:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd21d61700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c197410 con 0x7fcd1c1008a0 2023-10-18T12:25:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 0x7fcd1c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 0x7fcd1c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37086/0 (socket says 172.21.15.70:37086) 2023-10-18T12:25:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.266+0000 7fcd1affd700 1 -- 172.21.15.70:0/3339920928 learned_addr learned my addr 172.21.15.70:0/3339920928 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1bfff700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd1c1057c0 0x7fcd1c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1b7fe700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1c1008a0 0x7fcd1c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1affd700 1 -- 172.21.15.70:0/3339920928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1c1008a0 msgr2=0x7fcd1c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1affd700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1c1008a0 0x7fcd1c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1affd700 1 -- 172.21.15.70:0/3339920928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd1c1057c0 msgr2=0x7fcd1c196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1affd700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd1c1057c0 0x7fcd1c196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1affd700 1 -- 172.21.15.70:0/3339920928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0095f0 con 0x7fcd1c101260 2023-10-18T12:25:18.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.267+0000 7fcd1affd700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 0x7fcd1c19ce30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e9a0 tx=0x7fcd1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:18.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.268+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10009790 con 0x7fcd1c101260 2023-10-18T12:25:18.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.268+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c1976f0 con 0x7fcd1c101260 2023-10-18T12:25:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.268+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd10010430 con 0x7fcd1c101260 2023-10-18T12:25:18.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.268+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10018680 con 0x7fcd1c101260 2023-10-18T12:25:18.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.268+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c1a38e0 con 0x7fcd1c101260 2023-10-18T12:25:18.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.269+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c108d80 con 0x7fcd1c101260 2023-10-18T12:25:18.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.271+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fcd100105a0 con 0x7fcd1c101260 2023-10-18T12:25:18.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.272+0000 7fcd18ff9700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd040a15e0 0x7fcd040a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:18.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.273+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fcd10014070 con 0x7fcd1c101260 2023-10-18T12:25:18.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.273+0000 7fcd1b7fe700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd040a15e0 0x7fcd040a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:18.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.274+0000 7fcd1b7fe700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd040a15e0 0x7fcd040a3aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c006010 tx=0x7fcd0c00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:18.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.277+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcd10096540 con 0x7fcd1c101260 2023-10-18T12:25:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.347+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f93f0073fd0 con 0x7f93f01008a0 2023-10-18T12:25:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.347+0000 7f93ed7fa700 1 -- 172.21.15.70:0/3983955566 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f93e009b500 con 0x7f93f01008a0 2023-10-18T12:25:18.351 INFO:teuthology.orchestra.run.smithi070.stdout:343597383705 2023-10-18T12:25:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93d80a1560 msgr2=0x7f93d80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93d80a1560 0x7f93d80a3a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f93f0074220 tx=0x7f93e4009260 comp rx=0 tx=0).stop 2023-10-18T12:25:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 msgr2=0x7f93f0072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 0x7f93f0072af0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0005740 comp rx=0 tx=0).stop 2023-10-18T12:25:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 shutdown_connections 2023-10-18T12:25:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93f01057c0 0x7f93f0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93f01008a0 0x7f93f0072af0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93d80a1560 0x7f93d80a3a20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 --2- 172.21.15.70:0/3983955566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93f0101260 0x7f93f0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 >> 172.21.15.70:0/3983955566 conn(0x7f93f00779e0 msgr2=0x7f93f00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 shutdown_connections 2023-10-18T12:25:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.350+0000 7f93f6813700 1 -- 172.21.15.70:0/3983955566 wait complete. 2023-10-18T12:25:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.524+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcd1c101f30 con 0x7fcd1c101260 2023-10-18T12:25:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.525+0000 7fcd18ff9700 1 -- 172.21.15.70:0/3339920928 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd10099b60 con 0x7fcd1c101260 2023-10-18T12:25:18.528 INFO:teuthology.orchestra.run.smithi070.stdout:111669149756 2023-10-18T12:25:18.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd040a15e0 msgr2=0x7fcd040a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd040a15e0 0x7fcd040a3aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c006010 tx=0x7fcd0c00b260 comp rx=0 tx=0).stop 2023-10-18T12:25:18.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 msgr2=0x7fcd1c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:18.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 0x7fcd1c19ce30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e9a0 tx=0x7fcd1000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 shutdown_connections 2023-10-18T12:25:18.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd040a15e0 0x7fcd040a3aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd1c1057c0 0x7fcd1c196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1c1008a0 0x7fcd1c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 --2- 172.21.15.70:0/3339920928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c101260 0x7fcd1c19ce30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:18.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 >> 172.21.15.70:0/3339920928 conn(0x7fcd1c0779e0 msgr2=0x7fcd1c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:18.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.527+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 shutdown_connections 2023-10-18T12:25:18.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:18.528+0000 7fcd21d61700 1 -- 172.21.15.70:0/3339920928 wait complete. 2023-10-18T12:25:18.552 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:18 smithi070 ceph-mon[98639]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2023-10-18T12:25:18.552 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:18 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3116834937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-18T12:25:18.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:18 smithi152 ceph-mon[134767]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2023-10-18T12:25:18.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:18 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3116834937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-18T12:25:18.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:18 smithi186 ceph-mon[135253]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2023-10-18T12:25:18.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:18 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3116834937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-18T12:25:18.980 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888118 got 146028888121 for osd.4 2023-10-18T12:25:18.980 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:19.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 -- 172.21.15.70:0/1054122655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204072c20 msgr2=0x7f6204073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:19.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 --2- 172.21.15.70:0/1054122655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204072c20 0x7f6204073000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f62000098f0 tx=0x7f6200009c00 comp rx=0 tx=0).stop 2023-10-18T12:25:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 -- 172.21.15.70:0/1054122655 shutdown_connections 2023-10-18T12:25:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 --2- 172.21.15.70:0/1054122655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f620410ce30 0x7f620410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 --2- 172.21.15.70:0/1054122655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204072c20 0x7f6204073000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 --2- 172.21.15.70:0/1054122655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204073540 0x7f620410c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 -- 172.21.15.70:0/1054122655 >> 172.21.15.70:0/1054122655 conn(0x7f62040fc120 msgr2=0x7f62040fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 -- 172.21.15.70:0/1054122655 shutdown_connections 2023-10-18T12:25:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.056+0000 7f620ab17700 1 -- 172.21.15.70:0/1054122655 wait complete. 2023-10-18T12:25:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.057+0000 7f620ab17700 1 Processor -- start 2023-10-18T12:25:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.057+0000 7f620ab17700 1 -- start start 2023-10-18T12:25:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620ab17700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204072c20 0x7f6204198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620ab17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 0x7f6204199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620ab17700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f620410ce30 0x7f620419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620ab17700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419cb30 con 0x7f6204073540 2023-10-18T12:25:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620ab17700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419cca0 con 0x7f6204072c20 2023-10-18T12:25:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620ab17700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419ce60 con 0x7f620410ce30 2023-10-18T12:25:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f6209314700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 0x7f6204199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f6209314700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 0x7f6204199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37096/0 (socket says 172.21.15.70:37096) 2023-10-18T12:25:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f6209314700 1 -- 172.21.15.70:0/2716918824 learned_addr learned my addr 172.21.15.70:0/2716918824 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f6209b15700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204072c20 0x7f6204198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.058+0000 7f620a316700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f620410ce30 0x7f620419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209b15700 1 -- 172.21.15.70:0/2716918824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f620410ce30 msgr2=0x7f620419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209b15700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f620410ce30 0x7f620419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209b15700 1 -- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 msgr2=0x7f6204199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209b15700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 0x7f6204199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209b15700 1 -- 172.21.15.70:0/2716918824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6200009520 con 0x7f6204072c20 2023-10-18T12:25:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209314700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 0x7f6204199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f6209b15700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204072c20 0x7f6204198d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6200009fd0 tx=0x7f620000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620001c070 con 0x7f6204072c20 2023-10-18T12:25:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620419d0e0 con 0x7f6204072c20 2023-10-18T12:25:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620000baa0 con 0x7f6204072c20 2023-10-18T12:25:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62000038e0 con 0x7f6204072c20 2023-10-18T12:25:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.059+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62041a5a90 con 0x7f6204072c20 2023-10-18T12:25:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.060+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204108040 con 0x7f6204072c20 2023-10-18T12:25:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.064+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6200020030 con 0x7f6204072c20 2023-10-18T12:25:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.066+0000 7f61faffd700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61f00a1560 0x7f61f00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.066+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f62000ce370 con 0x7f6204072c20 2023-10-18T12:25:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.066+0000 7f6209314700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61f00a1560 0x7f61f00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.066+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f62000974b0 con 0x7f6204072c20 2023-10-18T12:25:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.067+0000 7f6209314700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61f00a1560 0x7f61f00a3a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f620419a210 tx=0x7f61f4009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:19.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.314+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6204199fc0 con 0x7f6204072c20 2023-10-18T12:25:19.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.314+0000 7f61faffd700 1 -- 172.21.15.70:0/2716918824 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f620009aad0 con 0x7f6204072c20 2023-10-18T12:25:19.318 INFO:teuthology.orchestra.run.smithi070.stdout:167503724597 2023-10-18T12:25:19.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61f00a1560 msgr2=0x7f61f00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:19.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61f00a1560 0x7f61f00a3a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f620419a210 tx=0x7f61f4009310 comp rx=0 tx=0).stop 2023-10-18T12:25:19.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204072c20 msgr2=0x7f6204198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:19.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204072c20 0x7f6204198d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6200009fd0 tx=0x7f620000fa00 comp rx=0 tx=0).stop 2023-10-18T12:25:19.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 shutdown_connections 2023-10-18T12:25:19.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f620410ce30 0x7f620419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f61f00a1560 0x7f61f00a3a20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6204072c20 0x7f6204198d40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 --2- 172.21.15.70:0/2716918824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6204073540 0x7f6204199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:19.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 >> 172.21.15.70:0/2716918824 conn(0x7f62040fc120 msgr2=0x7f62040fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:19.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 shutdown_connections 2023-10-18T12:25:19.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:19.317+0000 7f620ab17700 1 -- 172.21.15.70:0/2716918824 wait complete. 2023-10-18T12:25:19.481 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:19 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3983955566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-18T12:25:19.482 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:19 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3339920928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-18T12:25:19.484 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149754 got 111669149756 for osd.3 2023-10-18T12:25:19.484 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:19.641 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:19 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3983955566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-18T12:25:19.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:19 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3339920928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-18T12:25:19.717 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383705 for osd.11 2023-10-18T12:25:19.718 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:19 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3983955566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-18T12:25:19.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:19 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3339920928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-18T12:25:20.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 -- 172.21.15.70:0/4137692669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 msgr2=0x7f780c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 --2- 172.21.15.70:0/4137692669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 0x7f780c105ba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009990 tx=0x7f77fc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 -- 172.21.15.70:0/4137692669 shutdown_connections 2023-10-18T12:25:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 --2- 172.21.15.70:0/4137692669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 0x7f780c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 --2- 172.21.15.70:0/4137692669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 0x7f780c105ba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 --2- 172.21.15.70:0/4137692669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.175+0000 7f7812826700 1 -- 172.21.15.70:0/4137692669 >> 172.21.15.70:0/4137692669 conn(0x7f780c0779e0 msgr2=0x7f780c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.176+0000 7f7812826700 1 -- 172.21.15.70:0/4137692669 shutdown_connections 2023-10-18T12:25:20.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.176+0000 7f7812826700 1 -- 172.21.15.70:0/4137692669 wait complete. 2023-10-18T12:25:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 Processor -- start 2023-10-18T12:25:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 -- start start 2023-10-18T12:25:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 0x7f780c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 0x7f780c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a5130 con 0x7f780c101260 2023-10-18T12:25:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a52a0 con 0x7f780c1008a0 2023-10-18T12:25:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.177+0000 7f7812826700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a5460 con 0x7f780c1057c0 2023-10-18T12:25:20.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 0x7f780c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35678/0 (socket says 172.21.15.70:35678) 2023-10-18T12:25:20.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 -- 172.21.15.70:0/561732808 learned_addr learned my addr 172.21.15.70:0/561732808 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f7810dc3700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 0x7f780c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 -- 172.21.15.70:0/561732808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 msgr2=0x7f780c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 0x7f780c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 -- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 msgr2=0x7f780c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 0x7f780c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 -- 172.21.15.70:0/561732808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0095f0 con 0x7f780c1008a0 2023-10-18T12:25:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780b7fe700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 0x7f780c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.178+0000 7f780bfff700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c072af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77fc000c00 tx=0x7f77fc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.179+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc01d070 con 0x7f780c1008a0 2023-10-18T12:25:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.179+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c1a56e0 con 0x7f780c1008a0 2023-10-18T12:25:20.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.179+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c1a5b50 con 0x7f780c1008a0 2023-10-18T12:25:20.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.179+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77fc02b3b0 con 0x7f780c1008a0 2023-10-18T12:25:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.179+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00e640 con 0x7f780c1008a0 2023-10-18T12:25:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.180+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f80052f0 con 0x7f780c1008a0 2023-10-18T12:25:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.182+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f77fc00fad0 con 0x7f780c1008a0 2023-10-18T12:25:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.185+0000 7f78097fa700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77f40a1560 0x7f77f40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.185+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f77fc0ce480 con 0x7f780c1008a0 2023-10-18T12:25:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.185+0000 7f780b7fe700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77f40a1560 0x7f77f40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.187+0000 7f780b7fe700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77f40a1560 0x7f77f40a3a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f780c074220 tx=0x7f7800009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:20.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.188+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77fc0975c0 con 0x7f780c1008a0 2023-10-18T12:25:20.231 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724594 got 167503724597 for osd.5 2023-10-18T12:25:20.232 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.080+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/2984882290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068190 msgr2=0x7f4d340685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.080+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/2984882290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068190 0x7f4d340685f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009990 tx=0x7f4d24009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/2984882290 shutdown_connections 2023-10-18T12:25:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/2984882290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068b30 0x7f4d3410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/2984882290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d341067a0 0x7f4d34106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/2984882290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068190 0x7f4d340685f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/2984882290 >> 172.21.15.70:0/2984882290 conn(0x7f4d34075790 msgr2=0x7f4d34075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/2984882290 shutdown_connections 2023-10-18T12:25:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.081+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/2984882290 wait complete. 2023-10-18T12:25:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.082+0000 7f4d3bcf1700 1 Processor -- start 2023-10-18T12:25:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.082+0000 7f4d3bcf1700 1 -- start start 2023-10-18T12:25:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3bcf1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068190 0x7f4d34198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3bcf1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068b30 0x7f4d341993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3bcf1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d341067a0 0x7f4d3419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3bcf1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3419dd60 con 0x7f4d34068b30 2023-10-18T12:25:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3bcf1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3419ded0 con 0x7f4d341067a0 2023-10-18T12:25:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3bcf1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3419e090 con 0x7f4d34068190 2023-10-18T12:25:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3928c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068b30 0x7f4d341993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d39a8d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068190 0x7f4d34198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3928c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068b30 0x7f4d341993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37118/0 (socket says 172.21.15.70:37118) 2023-10-18T12:25:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3928c700 1 -- 172.21.15.70:0/4162871208 learned_addr learned my addr 172.21.15.70:0/4162871208 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.083+0000 7f4d3a28e700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d341067a0 0x7f4d3419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d39a8d700 1 -- 172.21.15.70:0/4162871208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d341067a0 msgr2=0x7f4d3419d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d39a8d700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d341067a0 0x7f4d3419d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d39a8d700 1 -- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068b30 msgr2=0x7f4d341993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d39a8d700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068b30 0x7f4d341993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d39a8d700 1 -- 172.21.15.70:0/4162871208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240095f0 con 0x7f4d34068190 2023-10-18T12:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d39a8d700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068190 0x7f4d34198e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e9f0 tx=0x7f4d3000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000c9c0 con 0x7f4d34068190 2023-10-18T12:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3419e370 con 0x7f4d34068190 2023-10-18T12:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d30010430 con 0x7f4d34068190 2023-10-18T12:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30018680 con 0x7f4d34068190 2023-10-18T12:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.084+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d341a5bd0 con 0x7f4d34068190 2023-10-18T12:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.086+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d340fed60 con 0x7f4d34068190 2023-10-18T12:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.087+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f4d30010860 con 0x7f4d34068190 2023-10-18T12:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.089+0000 7f4d2affd700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d200a1560 0x7f4d200a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:20.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.089+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f4d30014070 con 0x7f4d34068190 2023-10-18T12:25:20.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.089+0000 7f4d3928c700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d200a1560 0x7f4d200a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:20.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.090+0000 7f4d3928c700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d200a1560 0x7f4d200a3a20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d3419a3c0 tx=0x7f4d2400b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:20.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.091+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4d30096490 con 0x7f4d34068190 2023-10-18T12:25:20.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.318+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4d34101ee0 con 0x7f4d34068190 2023-10-18T12:25:20.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.320+0000 7f4d2affd700 1 -- 172.21.15.70:0/4162871208 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d30099ab0 con 0x7f4d34068190 2023-10-18T12:25:20.323 INFO:teuthology.orchestra.run.smithi070.stdout:223338299435 2023-10-18T12:25:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.322+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d200a1560 msgr2=0x7f4d200a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.322+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d200a1560 0x7f4d200a3a20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d3419a3c0 tx=0x7f4d2400b260 comp rx=0 tx=0).stop 2023-10-18T12:25:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.322+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068190 msgr2=0x7f4d34198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.322+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068190 0x7f4d34198e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e9f0 tx=0x7f4d3000ed00 comp rx=0 tx=0).stop 2023-10-18T12:25:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 shutdown_connections 2023-10-18T12:25:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d200a1560 0x7f4d200a3a20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d341067a0 0x7f4d3419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34068190 0x7f4d34198e60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 --2- 172.21.15.70:0/4162871208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d34068b30 0x7f4d341993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 >> 172.21.15.70:0/4162871208 conn(0x7f4d34075790 msgr2=0x7f4d3410bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 shutdown_connections 2023-10-18T12:25:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.323+0000 7f4d3bcf1700 1 -- 172.21.15.70:0/4162871208 wait complete. 2023-10-18T12:25:20.467 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:20 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2716918824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-18T12:25:20.467 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:20 smithi070 ceph-mon[98639]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:20.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.465+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f77f8005160 con 0x7f780c1008a0 2023-10-18T12:25:20.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.466+0000 7f78097fa700 1 -- 172.21.15.70:0/561732808 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77fc09abe0 con 0x7f780c1008a0 2023-10-18T12:25:20.469 INFO:teuthology.orchestra.run.smithi070.stdout:292057776163 2023-10-18T12:25:20.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77f40a1560 msgr2=0x7f77f40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77f40a1560 0x7f77f40a3a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f780c074220 tx=0x7f7800009260 comp rx=0 tx=0).stop 2023-10-18T12:25:20.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 msgr2=0x7f780c072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:20.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c072af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77fc000c00 tx=0x7f77fc005740 comp rx=0 tx=0).stop 2023-10-18T12:25:20.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 shutdown_connections 2023-10-18T12:25:20.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c1057c0 0x7f780c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77f40a1560 0x7f77f40a3a20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f780c1008a0 0x7f780c072af0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 --2- 172.21.15.70:0/561732808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780c101260 0x7f780c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:20.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 >> 172.21.15.70:0/561732808 conn(0x7f780c0779e0 msgr2=0x7f780c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:20.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 shutdown_connections 2023-10-18T12:25:20.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:20.469+0000 7f7812826700 1 -- 172.21.15.70:0/561732808 wait complete. 2023-10-18T12:25:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:20 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2716918824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-18T12:25:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:20 smithi152 ceph-mon[134767]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:20 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2716918824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-18T12:25:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:20 smithi186 ceph-mon[135253]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:21.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776163 for osd.9 2023-10-18T12:25:21.040 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:21.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:21 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4162871208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-18T12:25:21.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:21 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/561732808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-18T12:25:21.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299435 for osd.7 2023-10-18T12:25:21.483 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:21.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:21 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4162871208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-18T12:25:21.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:21 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/561732808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-18T12:25:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:21 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4162871208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-18T12:25:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:21 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/561732808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-18T12:25:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 -- 172.21.15.70:0/1549642187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 msgr2=0x7fab9c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 --2- 172.21.15.70:0/1549642187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c100d20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab90009990 tx=0x7fab90009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 -- 172.21.15.70:0/1549642187 shutdown_connections 2023-10-18T12:25:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 --2- 172.21.15.70:0/1549642187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c101260 0x7fab9c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 --2- 172.21.15.70:0/1549642187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c1057c0 0x7fab9c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 --2- 172.21.15.70:0/1549642187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c100d20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 -- 172.21.15.70:0/1549642187 >> 172.21.15.70:0/1549642187 conn(0x7fab9c0779e0 msgr2=0x7fab9c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:22.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 -- 172.21.15.70:0/1549642187 shutdown_connections 2023-10-18T12:25:22.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.109+0000 7faba22ed700 1 -- 172.21.15.70:0/1549642187 wait complete. 2023-10-18T12:25:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.110+0000 7faba22ed700 1 Processor -- start 2023-10-18T12:25:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.110+0000 7faba22ed700 1 -- start start 2023-10-18T12:25:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.110+0000 7faba22ed700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba22ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c101260 0x7fab9c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba22ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c1057c0 0x7fab9c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba22ed700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c197140 con 0x7fab9c1008a0 2023-10-18T12:25:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba22ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1972b0 con 0x7fab9c101260 2023-10-18T12:25:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba22ed700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c197470 con 0x7fab9c1057c0 2023-10-18T12:25:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c101260 0x7fab9c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c101260 0x7fab9c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35698/0 (socket says 172.21.15.70:35698) 2023-10-18T12:25:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9b7fe700 1 -- 172.21.15.70:0/3473919364 learned_addr learned my addr 172.21.15.70:0/3473919364 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37140/0 (socket says 172.21.15.70:37140) 2023-10-18T12:25:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba088a700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c1057c0 0x7fab9c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 -- 172.21.15.70:0/3473919364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c1057c0 msgr2=0x7fab9c196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c1057c0 0x7fab9c196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 -- 172.21.15.70:0/3473919364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c101260 msgr2=0x7fab9c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c101260 0x7fab9c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7fab9bfff700 1 -- 172.21.15.70:0/3473919364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900095f0 con 0x7fab9c1008a0 2023-10-18T12:25:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.111+0000 7faba088a700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c1057c0 0x7fab9c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:22.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.112+0000 7fab9bfff700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c19c950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00e9c0 tx=0x7fab8c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:22.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.112+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00cbd0 con 0x7fab9c1008a0 2023-10-18T12:25:22.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.112+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab8c00eeb0 con 0x7fab9c1008a0 2023-10-18T12:25:22.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.112+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c197750 con 0x7fab9c1008a0 2023-10-18T12:25:22.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.112+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c003c70 con 0x7fab9c1008a0 2023-10-18T12:25:22.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.112+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c1a3940 con 0x7fab9c1008a0 2023-10-18T12:25:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.113+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c108da0 con 0x7fab9c1008a0 2023-10-18T12:25:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.115+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fab8c003dd0 con 0x7fab9c1008a0 2023-10-18T12:25:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.116+0000 7fab997fa700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fab840a1560 0x7fab840a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.116+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fab8c014070 con 0x7fab9c1008a0 2023-10-18T12:25:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.117+0000 7fab9b7fe700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fab840a1560 0x7fab840a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.117+0000 7fab9b7fe700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fab840a1560 0x7fab840a3a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab9c197c50 tx=0x7fab9000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.120+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fab8c096670 con 0x7fab9c1008a0 2023-10-18T12:25:22.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.336+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fab9c04e4f0 con 0x7fab9c1008a0 2023-10-18T12:25:22.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.337+0000 7fab997fa700 1 -- 172.21.15.70:0/3473919364 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fab8c099c90 con 0x7fab9c1008a0 2023-10-18T12:25:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:313532612638 2023-10-18T12:25:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fab840a1560 msgr2=0x7fab840a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fab840a1560 0x7fab840a3a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab9c197c50 tx=0x7fab9000b260 comp rx=0 tx=0).stop 2023-10-18T12:25:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 msgr2=0x7fab9c19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c19c950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00e9c0 tx=0x7fab8c00ecd0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 shutdown_connections 2023-10-18T12:25:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab9c1057c0 0x7fab9c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9c1008a0 0x7fab9c19c950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fab840a1560 0x7fab840a3a20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 --2- 172.21.15.70:0/3473919364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab9c101260 0x7fab9c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:22.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 >> 172.21.15.70:0/3473919364 conn(0x7fab9c0779e0 msgr2=0x7fab9c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:22.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 shutdown_connections 2023-10-18T12:25:22.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:22.339+0000 7faba22ed700 1 -- 172.21.15.70:0/3473919364 wait complete. 2023-10-18T12:25:22.508 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:22 smithi070 ceph-mon[98639]: pgmap v293: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:22.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:22 smithi152 ceph-mon[134767]: pgmap v293: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:22.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:22 smithi186 ceph-mon[135253]: pgmap v293: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:22.802 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612638 for osd.10 2023-10-18T12:25:22.803 DEBUG:teuthology.parallel:result is None 2023-10-18T12:25:22.803 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-18T12:25:22.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-18T12:25:23.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:23 smithi070 ceph-mon[98639]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-10-18T12:25:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:23 smithi070 ceph-mon[98639]: Cluster is now healthy 2023-10-18T12:25:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:23 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3473919364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-18T12:25:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:23.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:23 smithi152 ceph-mon[134767]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-10-18T12:25:23.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:23 smithi152 ceph-mon[134767]: Cluster is now healthy 2023-10-18T12:25:23.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:23 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3473919364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-18T12:25:23.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:23 smithi186 ceph-mon[135253]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-10-18T12:25:23.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:23 smithi186 ceph-mon[135253]: Cluster is now healthy 2023-10-18T12:25:23.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:23 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3473919364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-18T12:25:25.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.201+0000 7fa3ada55700 1 -- 172.21.15.70:0/3938683555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 msgr2=0x7fa3a8108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:25.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.201+0000 7fa3ada55700 1 --2- 172.21.15.70:0/3938683555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 0x7fa3a8108d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa398009990 tx=0x7fa398009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 -- 172.21.15.70:0/3938683555 shutdown_connections 2023-10-18T12:25:25.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 --2- 172.21.15.70:0/3938683555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3a8103360 0x7fa3a810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 --2- 172.21.15.70:0/3938683555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 0x7fa3a8108d80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 --2- 172.21.15.70:0/3938683555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 -- 172.21.15.70:0/3938683555 >> 172.21.15.70:0/3938683555 conn(0x7fa3a80fe4b0 msgr2=0x7fa3a81008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:25.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 -- 172.21.15.70:0/3938683555 shutdown_connections 2023-10-18T12:25:25.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.202+0000 7fa3ada55700 1 -- 172.21.15.70:0/3938683555 wait complete. 2023-10-18T12:25:25.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 Processor -- start 2023-10-18T12:25:25.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 -- start start 2023-10-18T12:25:25.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:25.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3a8103360 0x7fa3a8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:25.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 0x7fa3a819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:25.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a819dd10 con 0x7fa3a81089a0 2023-10-18T12:25:25.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a819de80 con 0x7fa3a81029a0 2023-10-18T12:25:25.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3ada55700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a819e040 con 0x7fa3a8103360 2023-10-18T12:25:25.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3a6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:25.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3a6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35714/0 (socket says 172.21.15.70:35714) 2023-10-18T12:25:25.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.203+0000 7fa3a6ffd700 1 -- 172.21.15.70:0/2814957269 learned_addr learned my addr 172.21.15.70:0/2814957269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:25.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a77fe700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 0x7fa3a819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:25.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a67fc700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3a8103360 0x7fa3a8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:25.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a6ffd700 1 -- 172.21.15.70:0/2814957269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3a8103360 msgr2=0x7fa3a8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:25.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a6ffd700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3a8103360 0x7fa3a8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a6ffd700 1 -- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 msgr2=0x7fa3a819d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:25.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a6ffd700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 0x7fa3a819d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a6ffd700 1 -- 172.21.15.70:0/2814957269 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3980095f0 con 0x7fa3a81029a0 2023-10-18T12:25:25.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3a6ffd700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8198e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa398005910 tx=0x7fa398004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:25.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39801d070 con 0x7fa3a81029a0 2023-10-18T12:25:25.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.204+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a819e2c0 con 0x7fa3a81029a0 2023-10-18T12:25:25.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.205+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa398022470 con 0x7fa3a81029a0 2023-10-18T12:25:25.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.205+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a81a5b80 con 0x7fa3a81029a0 2023-10-18T12:25:25.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.205+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398017650 con 0x7fa3a81029a0 2023-10-18T12:25:25.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.205+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a810afa0 con 0x7fa3a81029a0 2023-10-18T12:25:25.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.208+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fa3980225e0 con 0x7fa3a81029a0 2023-10-18T12:25:25.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.209+0000 7fa3aca53700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa3900a1560 0x7fa3900a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:25.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.209+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fa3980ce9b0 con 0x7fa3a81029a0 2023-10-18T12:25:25.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.209+0000 7fa3a67fc700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa3900a1560 0x7fa3900a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:25.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.210+0000 7fa3a67fc700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa3900a1560 0x7fa3900a3a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3a819a370 tx=0x7fa39c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:25.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.212+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa398097af0 con 0x7fa3a81029a0 2023-10-18T12:25:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:24 smithi186 ceph-mon[135253]: pgmap v294: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 979 B/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:25.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:25.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:24 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:24 smithi070 ceph-mon[98639]: pgmap v294: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 979 B/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:24 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.407+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3a804e530 con 0x7fa3900a1560 2023-10-18T12:25:25.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:24 smithi152 ceph-mon[134767]: pgmap v294: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 979 B/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:25.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:25.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:24 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:25.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.420+0000 7fa3aca53700 1 -- 172.21.15.70:0/2814957269 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427099 (secure 0 0 0) 0x7fa3a804e530 con 0x7fa3900a1560 2023-10-18T12:25:25.431 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:25.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa3900a1560 msgr2=0x7fa3900a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:25.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa3900a1560 0x7fa3900a3a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3a819a370 tx=0x7fa39c009260 comp rx=0 tx=0).stop 2023-10-18T12:25:25.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 msgr2=0x7fa3a8198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:25.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8198e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa398005910 tx=0x7fa398004730 comp rx=0 tx=0).stop 2023-10-18T12:25:25.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 shutdown_connections 2023-10-18T12:25:25.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a81089a0 0x7fa3a819d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa3900a1560 0x7fa3900a3a20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3a81029a0 0x7fa3a8198e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 --2- 172.21.15.70:0/2814957269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3a8103360 0x7fa3a8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 >> 172.21.15.70:0/2814957269 conn(0x7fa3a80fe4b0 msgr2=0x7fa3a80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 shutdown_connections 2023-10-18T12:25:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:25.426+0000 7fa3ada55700 1 -- 172.21.15.70:0/2814957269 wait complete. 2023-10-18T12:25:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:25:25.947 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-10-18T12:25:23.610295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652297,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":245,"num_read_kb":235,"num_write":255,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338260,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603564,"statfs":{"total":1151940427776,"available":1151594049536,"internally_reserved":0,"allocated":14315520,"data_stored":8096653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":9,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.404301"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215639+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:25:14.215639+0000","last_peered":"2023-10-18T12:25:14.215639+0000","last_clean":"2023-10-18T12:25:14.215639+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:25:14.215639+0000","last_undegraded":"2023-10-18T12:25:14.215639+0000","last_fullsized":"2023-10-18T12:25:14.215639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218812+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:25:14.218812+0000","last_peered":"2023-10-18T12:25:14.218812+0000","last_clean":"2023-10-18T12:25:14.218812+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:25:14.218812+0000","last_undegraded":"2023-10-18T12:25:14.218812+0000","last_fullsized":"2023-10-18T12:25:14.218812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218728+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:25:14.218728+0000","last_peered":"2023-10-18T12:25:14.218728+0000","last_clean":"2023-10-18T12:25:14.218728+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:25:14.218728+0000","last_undegraded":"2023-10-18T12:25:14.218728+0000","last_fullsized":"2023-10-18T12:25:14.218728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212709+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:25:14.212709+0000","last_peered":"2023-10-18T12:25:14.212709+0000","last_clean":"2023-10-18T12:25:14.212709+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:25:14.212709+0000","last_undegraded":"2023-10-18T12:25:14.212709+0000","last_fullsized":"2023-10-18T12:25:14.212709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216241+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:25:14.216241+0000","last_peered":"2023-10-18T12:25:14.216241+0000","last_clean":"2023-10-18T12:25:14.216241+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:25:14.216241+0000","last_undegraded":"2023-10-18T12:25:14.216241+0000","last_fullsized":"2023-10-18T12:25:14.216241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212255+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:25:14.212255+0000","last_peered":"2023-10-18T12:25:14.212255+0000","last_clean":"2023-10-18T12:25:14.212255+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:25:14.212255+0000","last_undegraded":"2023-10-18T12:25:14.212255+0000","last_fullsized":"2023-10-18T12:25:14.212255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212186+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:14.212186+0000","last_peered":"2023-10-18T12:25:14.212186+0000","last_clean":"2023-10-18T12:25:14.212186+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:14.212186+0000","last_undegraded":"2023-10-18T12:25:14.212186+0000","last_fullsized":"2023-10-18T12:25:14.212186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210824+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:25:14.210824+0000","last_peered":"2023-10-18T12:25:14.210824+0000","last_clean":"2023-10-18T12:25:14.210824+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:25:14.210824+0000","last_undegraded":"2023-10-18T12:25:14.210824+0000","last_fullsized":"2023-10-18T12:25:14.210824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215120+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:25:14.215120+0000","last_peered":"2023-10-18T12:25:14.215120+0000","last_clean":"2023-10-18T12:25:14.215120+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:25:14.215120+0000","last_undegraded":"2023-10-18T12:25:14.215120+0000","last_fullsized":"2023-10-18T12:25:14.215120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210943+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:25:14.210943+0000","last_peered":"2023-10-18T12:25:14.210943+0000","last_clean":"2023-10-18T12:25:14.210943+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:25:14.210943+0000","last_undegraded":"2023-10-18T12:25:14.210943+0000","last_fullsized":"2023-10-18T12:25:14.210943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210769+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:14.210769+0000","last_peered":"2023-10-18T12:25:14.210769+0000","last_clean":"2023-10-18T12:25:14.210769+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:14.210769+0000","last_undegraded":"2023-10-18T12:25:14.210769+0000","last_fullsized":"2023-10-18T12:25:14.210769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212634+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:14.212634+0000","last_peered":"2023-10-18T12:25:14.212634+0000","last_clean":"2023-10-18T12:25:14.212634+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:14.212634+0000","last_undegraded":"2023-10-18T12:25:14.212634+0000","last_fullsized":"2023-10-18T12:25:14.212634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208392+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:25:14.208392+0000","last_peered":"2023-10-18T12:25:14.208392+0000","last_clean":"2023-10-18T12:25:14.208392+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:25:14.208392+0000","last_undegraded":"2023-10-18T12:25:14.208392+0000","last_fullsized":"2023-10-18T12:25:14.208392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218307+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:25:14.218307+0000","last_peered":"2023-10-18T12:25:14.218307+0000","last_clean":"2023-10-18T12:25:14.218307+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:25:14.218307+0000","last_undegraded":"2023-10-18T12:25:14.218307+0000","last_fullsized":"2023-10-18T12:25:14.218307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218269+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:25:14.218269+0000","last_peered":"2023-10-18T12:25:14.218269+0000","last_clean":"2023-10-18T12:25:14.218269+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:25:14.218269+0000","last_undegraded":"2023-10-18T12:25:14.218269+0000","last_fullsized":"2023-10-18T12:25:14.218269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214462+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:25:14.214462+0000","last_peered":"2023-10-18T12:25:14.214462+0000","last_clean":"2023-10-18T12:25:14.214462+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:25:14.214462+0000","last_undegraded":"2023-10-18T12:25:14.214462+0000","last_fullsized":"2023-10-18T12:25:14.214462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215783+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:25:14.215783+0000","last_peered":"2023-10-18T12:25:14.215783+0000","last_clean":"2023-10-18T12:25:14.215783+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:25:14.215783+0000","last_undegraded":"2023-10-18T12:25:14.215783+0000","last_fullsized":"2023-10-18T12:25:14.215783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214795+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:25:14.214795+0000","last_peered":"2023-10-18T12:25:14.214795+0000","last_clean":"2023-10-18T12:25:14.214795+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:25:14.214795+0000","last_undegraded":"2023-10-18T12:25:14.214795+0000","last_fullsized":"2023-10-18T12:25:14.214795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213474+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:25:14.213474+0000","last_peered":"2023-10-18T12:25:14.213474+0000","last_clean":"2023-10-18T12:25:14.213474+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:25:14.213474+0000","last_undegraded":"2023-10-18T12:25:14.213474+0000","last_fullsized":"2023-10-18T12:25:14.213474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212928+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:25:14.212928+0000","last_peered":"2023-10-18T12:25:14.212928+0000","last_clean":"2023-10-18T12:25:14.212928+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:25:14.212928+0000","last_undegraded":"2023-10-18T12:25:14.212928+0000","last_fullsized":"2023-10-18T12:25:14.212928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214873+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:25:14.214873+0000","last_peered":"2023-10-18T12:25:14.214873+0000","last_clean":"2023-10-18T12:25:14.214873+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:25:14.214873+0000","last_undegraded":"2023-10-18T12:25:14.214873+0000","last_fullsized":"2023-10-18T12:25:14.214873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211210+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:25:14.211210+0000","last_peered":"2023-10-18T12:25:14.211210+0000","last_clean":"2023-10-18T12:25:14.211210+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:25:14.211210+0000","last_undegraded":"2023-10-18T12:25:14.211210+0000","last_fullsized":"2023-10-18T12:25:14.211210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210737+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:14.210737+0000","last_peered":"2023-10-18T12:25:14.210737+0000","last_clean":"2023-10-18T12:25:14.210737+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:14.210737+0000","last_undegraded":"2023-10-18T12:25:14.210737+0000","last_fullsized":"2023-10-18T12:25:14.210737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211356+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:25:14.211356+0000","last_peered":"2023-10-18T12:25:14.211356+0000","last_clean":"2023-10-18T12:25:14.211356+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:25:14.211356+0000","last_undegraded":"2023-10-18T12:25:14.211356+0000","last_fullsized":"2023-10-18T12:25:14.211356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218636+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:25:14.218636+0000","last_peered":"2023-10-18T12:25:14.218636+0000","last_clean":"2023-10-18T12:25:14.218636+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:25:14.218636+0000","last_undegraded":"2023-10-18T12:25:14.218636+0000","last_fullsized":"2023-10-18T12:25:14.218636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211459+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:14.211459+0000","last_peered":"2023-10-18T12:25:14.211459+0000","last_clean":"2023-10-18T12:25:14.211459+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:14.211459+0000","last_undegraded":"2023-10-18T12:25:14.211459+0000","last_fullsized":"2023-10-18T12:25:14.211459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215742+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:25:14.215742+0000","last_peered":"2023-10-18T12:25:14.215742+0000","last_clean":"2023-10-18T12:25:14.215742+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:25:14.215742+0000","last_undegraded":"2023-10-18T12:25:14.215742+0000","last_fullsized":"2023-10-18T12:25:14.215742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215091+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:25:14.215091+0000","last_peered":"2023-10-18T12:25:14.215091+0000","last_clean":"2023-10-18T12:25:14.215091+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:25:14.215091+0000","last_undegraded":"2023-10-18T12:25:14.215091+0000","last_fullsized":"2023-10-18T12:25:14.215091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210540+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:14.210540+0000","last_peered":"2023-10-18T12:25:14.210540+0000","last_clean":"2023-10-18T12:25:14.210540+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:14.210540+0000","last_undegraded":"2023-10-18T12:25:14.210540+0000","last_fullsized":"2023-10-18T12:25:14.210540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216726+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:25:14.216726+0000","last_peered":"2023-10-18T12:25:14.216726+0000","last_clean":"2023-10-18T12:25:14.216726+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:25:14.216726+0000","last_undegraded":"2023-10-18T12:25:14.216726+0000","last_fullsized":"2023-10-18T12:25:14.216726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211493+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:25:14.211493+0000","last_peered":"2023-10-18T12:25:14.211493+0000","last_clean":"2023-10-18T12:25:14.211493+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:25:14.211493+0000","last_undegraded":"2023-10-18T12:25:14.211493+0000","last_fullsized":"2023-10-18T12:25:14.211493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217322+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:25:15.217322+0000","last_peered":"2023-10-18T12:25:15.217322+0000","last_clean":"2023-10-18T12:25:15.217322+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:25:15.217322+0000","last_undegraded":"2023-10-18T12:25:15.217322+0000","last_fullsized":"2023-10-18T12:25:15.217322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211310+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:25:14.211310+0000","last_peered":"2023-10-18T12:25:14.211310+0000","last_clean":"2023-10-18T12:25:14.211310+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:25:14.211310+0000","last_undegraded":"2023-10-18T12:25:14.211310+0000","last_fullsized":"2023-10-18T12:25:14.211310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215778+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:25:14.215778+0000","last_peered":"2023-10-18T12:25:14.215778+0000","last_clean":"2023-10-18T12:25:14.215778+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:25:14.215778+0000","last_undegraded":"2023-10-18T12:25:14.215778+0000","last_fullsized":"2023-10-18T12:25:14.215778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211137+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:25:14.211137+0000","last_peered":"2023-10-18T12:25:14.211137+0000","last_clean":"2023-10-18T12:25:14.211137+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:25:14.211137+0000","last_undegraded":"2023-10-18T12:25:14.211137+0000","last_fullsized":"2023-10-18T12:25:14.211137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210401+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:25:14.210401+0000","last_peered":"2023-10-18T12:25:14.210401+0000","last_clean":"2023-10-18T12:25:14.210401+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:25:14.210401+0000","last_undegraded":"2023-10-18T12:25:14.210401+0000","last_fullsized":"2023-10-18T12:25:14.210401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211447+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:25:14.211447+0000","last_peered":"2023-10-18T12:25:14.211447+0000","last_clean":"2023-10-18T12:25:14.211447+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:25:14.211447+0000","last_undegraded":"2023-10-18T12:25:14.211447+0000","last_fullsized":"2023-10-18T12:25:14.211447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215355+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:25:14.215355+0000","last_peered":"2023-10-18T12:25:14.215355+0000","last_clean":"2023-10-18T12:25:14.215355+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:25:14.215355+0000","last_undegraded":"2023-10-18T12:25:14.215355+0000","last_fullsized":"2023-10-18T12:25:14.215355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207597+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:25:14.207597+0000","last_peered":"2023-10-18T12:25:14.207597+0000","last_clean":"2023-10-18T12:25:14.207597+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:25:14.207597+0000","last_undegraded":"2023-10-18T12:25:14.207597+0000","last_fullsized":"2023-10-18T12:25:14.207597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210728+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:25:14.210728+0000","last_peered":"2023-10-18T12:25:14.210728+0000","last_clean":"2023-10-18T12:25:14.210728+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:25:14.210728+0000","last_undegraded":"2023-10-18T12:25:14.210728+0000","last_fullsized":"2023-10-18T12:25:14.210728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218811+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:25:14.218811+0000","last_peered":"2023-10-18T12:25:14.218811+0000","last_clean":"2023-10-18T12:25:14.218811+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:25:14.218811+0000","last_undegraded":"2023-10-18T12:25:14.218811+0000","last_fullsized":"2023-10-18T12:25:14.218811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210651+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:25:14.210651+0000","last_peered":"2023-10-18T12:25:14.210651+0000","last_clean":"2023-10-18T12:25:14.210651+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:25:14.210651+0000","last_undegraded":"2023-10-18T12:25:14.210651+0000","last_fullsized":"2023-10-18T12:25:14.210651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213990+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:25:14.213990+0000","last_peered":"2023-10-18T12:25:14.213990+0000","last_clean":"2023-10-18T12:25:14.213990+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:25:14.213990+0000","last_undegraded":"2023-10-18T12:25:14.213990+0000","last_fullsized":"2023-10-18T12:25:14.213990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215384+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:25:14.215384+0000","last_peered":"2023-10-18T12:25:14.215384+0000","last_clean":"2023-10-18T12:25:14.215384+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:25:14.215384+0000","last_undegraded":"2023-10-18T12:25:14.215384+0000","last_fullsized":"2023-10-18T12:25:14.215384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213793+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:25:14.213793+0000","last_peered":"2023-10-18T12:25:14.213793+0000","last_clean":"2023-10-18T12:25:14.213793+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:25:14.213793+0000","last_undegraded":"2023-10-18T12:25:14.213793+0000","last_fullsized":"2023-10-18T12:25:14.213793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211053+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:14.211053+0000","last_peered":"2023-10-18T12:25:14.211053+0000","last_clean":"2023-10-18T12:25:14.211053+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:14.211053+0000","last_undegraded":"2023-10-18T12:25:14.211053+0000","last_fullsized":"2023-10-18T12:25:14.211053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218903+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:25:14.218903+0000","last_peered":"2023-10-18T12:25:14.218903+0000","last_clean":"2023-10-18T12:25:14.218903+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:25:14.218903+0000","last_undegraded":"2023-10-18T12:25:14.218903+0000","last_fullsized":"2023-10-18T12:25:14.218903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211621+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:14.211621+0000","last_peered":"2023-10-18T12:25:14.211621+0000","last_clean":"2023-10-18T12:25:14.211621+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:14.211621+0000","last_undegraded":"2023-10-18T12:25:14.211621+0000","last_fullsized":"2023-10-18T12:25:14.211621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211404+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:25:14.211404+0000","last_peered":"2023-10-18T12:25:14.211404+0000","last_clean":"2023-10-18T12:25:14.211404+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:25:14.211404+0000","last_undegraded":"2023-10-18T12:25:14.211404+0000","last_fullsized":"2023-10-18T12:25:14.211404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.216324+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:25:15.216324+0000","last_peered":"2023-10-18T12:25:15.216324+0000","last_clean":"2023-10-18T12:25:15.216324+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:25:15.216324+0000","last_undegraded":"2023-10-18T12:25:15.216324+0000","last_fullsized":"2023-10-18T12:25:15.216324+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213954+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:25:14.213954+0000","last_peered":"2023-10-18T12:25:14.213954+0000","last_clean":"2023-10-18T12:25:14.213954+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:25:14.213954+0000","last_undegraded":"2023-10-18T12:25:14.213954+0000","last_fullsized":"2023-10-18T12:25:14.213954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'29","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:21.235431+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:25:21.235431+0000","last_peered":"2023-10-18T12:25:21.235431+0000","last_clean":"2023-10-18T12:25:21.235431+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:25:21.235431+0000","last_undegraded":"2023-10-18T12:25:21.235431+0000","last_fullsized":"2023-10-18T12:25:21.235431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60885,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":104,"num_read_kb":111,"num_write":29,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215942+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:25:14.215942+0000","last_peered":"2023-10-18T12:25:14.215942+0000","last_clean":"2023-10-18T12:25:14.215942+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:25:14.215942+0000","last_undegraded":"2023-10-18T12:25:14.215942+0000","last_fullsized":"2023-10-18T12:25:14.215942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211801+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:25:14.211801+0000","last_peered":"2023-10-18T12:25:14.211801+0000","last_clean":"2023-10-18T12:25:14.211801+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:25:14.211801+0000","last_undegraded":"2023-10-18T12:25:14.211801+0000","last_fullsized":"2023-10-18T12:25:14.211801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211560+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:25:14.211560+0000","last_peered":"2023-10-18T12:25:14.211560+0000","last_clean":"2023-10-18T12:25:14.211560+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:25:14.211560+0000","last_undegraded":"2023-10-18T12:25:14.211560+0000","last_fullsized":"2023-10-18T12:25:14.211560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207775+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:25:14.207775+0000","last_peered":"2023-10-18T12:25:14.207775+0000","last_clean":"2023-10-18T12:25:14.207775+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:25:14.207775+0000","last_undegraded":"2023-10-18T12:25:14.207775+0000","last_fullsized":"2023-10-18T12:25:14.207775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214581+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:25:14.214581+0000","last_peered":"2023-10-18T12:25:14.214581+0000","last_clean":"2023-10-18T12:25:14.214581+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:25:14.214581+0000","last_undegraded":"2023-10-18T12:25:14.214581+0000","last_fullsized":"2023-10-18T12:25:14.214581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211113+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:25:14.211113+0000","last_peered":"2023-10-18T12:25:14.211113+0000","last_clean":"2023-10-18T12:25:14.211113+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:25:14.211113+0000","last_undegraded":"2023-10-18T12:25:14.211113+0000","last_fullsized":"2023-10-18T12:25:14.211113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.219532+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:15.219532+0000","last_peered":"2023-10-18T12:25:15.219532+0000","last_clean":"2023-10-18T12:25:15.219532+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:15.219532+0000","last_undegraded":"2023-10-18T12:25:15.219532+0000","last_fullsized":"2023-10-18T12:25:15.219532+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214765+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:25:14.214765+0000","last_peered":"2023-10-18T12:25:14.214765+0000","last_clean":"2023-10-18T12:25:14.214765+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:25:14.214765+0000","last_undegraded":"2023-10-18T12:25:14.214765+0000","last_fullsized":"2023-10-18T12:25:14.214765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210996+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:14.210996+0000","last_peered":"2023-10-18T12:25:14.210996+0000","last_clean":"2023-10-18T12:25:14.210996+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:14.210996+0000","last_undegraded":"2023-10-18T12:25:14.210996+0000","last_fullsized":"2023-10-18T12:25:14.210996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210833+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:14.210833+0000","last_peered":"2023-10-18T12:25:14.210833+0000","last_clean":"2023-10-18T12:25:14.210833+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:14.210833+0000","last_undegraded":"2023-10-18T12:25:14.210833+0000","last_fullsized":"2023-10-18T12:25:14.210833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208699+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:25:14.208699+0000","last_peered":"2023-10-18T12:25:14.208699+0000","last_clean":"2023-10-18T12:25:14.208699+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:25:14.208699+0000","last_undegraded":"2023-10-18T12:25:14.208699+0000","last_fullsized":"2023-10-18T12:25:14.208699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225793+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:25:15.225793+0000","last_peered":"2023-10-18T12:25:15.225793+0000","last_clean":"2023-10-18T12:25:15.225793+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:25:15.225793+0000","last_undegraded":"2023-10-18T12:25:15.225793+0000","last_fullsized":"2023-10-18T12:25:15.225793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211963+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:25:14.211963+0000","last_peered":"2023-10-18T12:25:14.211963+0000","last_clean":"2023-10-18T12:25:14.211963+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:25:14.211963+0000","last_undegraded":"2023-10-18T12:25:14.211963+0000","last_fullsized":"2023-10-18T12:25:14.211963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211886+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:25:14.211886+0000","last_peered":"2023-10-18T12:25:14.211886+0000","last_clean":"2023-10-18T12:25:14.211886+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:25:14.211886+0000","last_undegraded":"2023-10-18T12:25:14.211886+0000","last_fullsized":"2023-10-18T12:25:14.211886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.224004+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:25:15.224004+0000","last_peered":"2023-10-18T12:25:15.224004+0000","last_clean":"2023-10-18T12:25:15.224004+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:25:15.224004+0000","last_undegraded":"2023-10-18T12:25:15.224004+0000","last_fullsized":"2023-10-18T12:25:15.224004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215148+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:25:14.215148+0000","last_peered":"2023-10-18T12:25:14.215148+0000","last_clean":"2023-10-18T12:25:14.215148+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:25:14.215148+0000","last_undegraded":"2023-10-18T12:25:14.215148+0000","last_fullsized":"2023-10-18T12:25:14.215148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215695+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:25:14.215695+0000","last_peered":"2023-10-18T12:25:14.215695+0000","last_clean":"2023-10-18T12:25:14.215695+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:25:14.215695+0000","last_undegraded":"2023-10-18T12:25:14.215695+0000","last_fullsized":"2023-10-18T12:25:14.215695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214390+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:25:14.214390+0000","last_peered":"2023-10-18T12:25:14.214390+0000","last_clean":"2023-10-18T12:25:14.214390+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:25:14.214390+0000","last_undegraded":"2023-10-18T12:25:14.214390+0000","last_fullsized":"2023-10-18T12:25:14.214390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210689+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:25:14.210689+0000","last_peered":"2023-10-18T12:25:14.210689+0000","last_clean":"2023-10-18T12:25:14.210689+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:25:14.210689+0000","last_undegraded":"2023-10-18T12:25:14.210689+0000","last_fullsized":"2023-10-18T12:25:14.210689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208378+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:25:14.208378+0000","last_peered":"2023-10-18T12:25:14.208378+0000","last_clean":"2023-10-18T12:25:14.208378+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:25:14.208378+0000","last_undegraded":"2023-10-18T12:25:14.208378+0000","last_fullsized":"2023-10-18T12:25:14.208378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209917+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:25:14.209917+0000","last_peered":"2023-10-18T12:25:14.209917+0000","last_clean":"2023-10-18T12:25:14.209917+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:25:14.209917+0000","last_undegraded":"2023-10-18T12:25:14.209917+0000","last_fullsized":"2023-10-18T12:25:14.209917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211140+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:25:14.211140+0000","last_peered":"2023-10-18T12:25:14.211140+0000","last_clean":"2023-10-18T12:25:14.211140+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:25:14.211140+0000","last_undegraded":"2023-10-18T12:25:14.211140+0000","last_fullsized":"2023-10-18T12:25:14.211140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209020+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:25:14.209020+0000","last_peered":"2023-10-18T12:25:14.209020+0000","last_clean":"2023-10-18T12:25:14.209020+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:25:14.209020+0000","last_undegraded":"2023-10-18T12:25:14.209020+0000","last_fullsized":"2023-10-18T12:25:14.209020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210801+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:25:14.210801+0000","last_peered":"2023-10-18T12:25:14.210801+0000","last_clean":"2023-10-18T12:25:14.210801+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:25:14.210801+0000","last_undegraded":"2023-10-18T12:25:14.210801+0000","last_fullsized":"2023-10-18T12:25:14.210801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207506+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:25:14.207506+0000","last_peered":"2023-10-18T12:25:14.207506+0000","last_clean":"2023-10-18T12:25:14.207506+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:25:14.207506+0000","last_undegraded":"2023-10-18T12:25:14.207506+0000","last_fullsized":"2023-10-18T12:25:14.207506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210590+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:25:14.210590+0000","last_peered":"2023-10-18T12:25:14.210590+0000","last_clean":"2023-10-18T12:25:14.210590+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:25:14.210590+0000","last_undegraded":"2023-10-18T12:25:14.210590+0000","last_fullsized":"2023-10-18T12:25:14.210590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211226+0000","last_change":"2023-10-18T12:24:09.985663+0000","last_active":"2023-10-18T12:25:14.211226+0000","last_peered":"2023-10-18T12:25:14.211226+0000","last_clean":"2023-10-18T12:25:14.211226+0000","last_became_active":"2023-10-18T12:24:09.983659+0000","last_became_peered":"2023-10-18T12:24:09.983659+0000","last_unstale":"2023-10-18T12:25:14.211226+0000","last_undegraded":"2023-10-18T12:25:14.211226+0000","last_fullsized":"2023-10-18T12:25:14.211226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:39.706073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211540+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:25:14.211540+0000","last_peered":"2023-10-18T12:25:14.211540+0000","last_clean":"2023-10-18T12:25:14.211540+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:25:14.211540+0000","last_undegraded":"2023-10-18T12:25:14.211540+0000","last_fullsized":"2023-10-18T12:25:14.211540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210165+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:14.210165+0000","last_peered":"2023-10-18T12:25:14.210165+0000","last_clean":"2023-10-18T12:25:14.210165+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:14.210165+0000","last_undegraded":"2023-10-18T12:25:14.210165+0000","last_fullsized":"2023-10-18T12:25:14.210165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223137+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:25:15.223137+0000","last_peered":"2023-10-18T12:25:15.223137+0000","last_clean":"2023-10-18T12:25:15.223137+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:25:15.223137+0000","last_undegraded":"2023-10-18T12:25:15.223137+0000","last_fullsized":"2023-10-18T12:25:15.223137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207572+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:25:14.207572+0000","last_peered":"2023-10-18T12:25:14.207572+0000","last_clean":"2023-10-18T12:25:14.207572+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:25:14.207572+0000","last_undegraded":"2023-10-18T12:25:14.207572+0000","last_fullsized":"2023-10-18T12:25:14.207572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217523+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:25:15.217523+0000","last_peered":"2023-10-18T12:25:15.217523+0000","last_clean":"2023-10-18T12:25:15.217523+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:25:15.217523+0000","last_undegraded":"2023-10-18T12:25:15.217523+0000","last_fullsized":"2023-10-18T12:25:15.217523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214480+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:25:14.214480+0000","last_peered":"2023-10-18T12:25:14.214480+0000","last_clean":"2023-10-18T12:25:14.214480+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:25:14.214480+0000","last_undegraded":"2023-10-18T12:25:14.214480+0000","last_fullsized":"2023-10-18T12:25:14.214480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215870+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:25:14.215870+0000","last_peered":"2023-10-18T12:25:14.215870+0000","last_clean":"2023-10-18T12:25:14.215870+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:25:14.215870+0000","last_undegraded":"2023-10-18T12:25:14.215870+0000","last_fullsized":"2023-10-18T12:25:14.215870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208601+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:25:14.208601+0000","last_peered":"2023-10-18T12:25:14.208601+0000","last_clean":"2023-10-18T12:25:14.208601+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:25:14.208601+0000","last_undegraded":"2023-10-18T12:25:14.208601+0000","last_fullsized":"2023-10-18T12:25:14.208601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210763+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:25:14.210763+0000","last_peered":"2023-10-18T12:25:14.210763+0000","last_clean":"2023-10-18T12:25:14.210763+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:25:14.210763+0000","last_undegraded":"2023-10-18T12:25:14.210763+0000","last_fullsized":"2023-10-18T12:25:14.210763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214833+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:25:14.214833+0000","last_peered":"2023-10-18T12:25:14.214833+0000","last_clean":"2023-10-18T12:25:14.214833+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:25:14.214833+0000","last_undegraded":"2023-10-18T12:25:14.214833+0000","last_fullsized":"2023-10-18T12:25:14.214833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211642+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:25:14.211642+0000","last_peered":"2023-10-18T12:25:14.211642+0000","last_clean":"2023-10-18T12:25:14.211642+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:25:14.211642+0000","last_undegraded":"2023-10-18T12:25:14.211642+0000","last_fullsized":"2023-10-18T12:25:14.211642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212653+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:25:14.212653+0000","last_peered":"2023-10-18T12:25:14.212653+0000","last_clean":"2023-10-18T12:25:14.212653+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:25:14.212653+0000","last_undegraded":"2023-10-18T12:25:14.212653+0000","last_fullsized":"2023-10-18T12:25:14.212653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213075+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:14.213075+0000","last_peered":"2023-10-18T12:25:14.213075+0000","last_clean":"2023-10-18T12:25:14.213075+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:14.213075+0000","last_undegraded":"2023-10-18T12:25:14.213075+0000","last_fullsized":"2023-10-18T12:25:14.213075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218907+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:25:14.218907+0000","last_peered":"2023-10-18T12:25:14.218907+0000","last_clean":"2023-10-18T12:25:14.218907+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:25:14.218907+0000","last_undegraded":"2023-10-18T12:25:14.218907+0000","last_fullsized":"2023-10-18T12:25:14.218907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223250+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223250+0000","last_peered":"2023-10-18T12:25:15.223250+0000","last_clean":"2023-10-18T12:25:15.223250+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223250+0000","last_undegraded":"2023-10-18T12:25:15.223250+0000","last_fullsized":"2023-10-18T12:25:15.223250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214524+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:25:14.214524+0000","last_peered":"2023-10-18T12:25:14.214524+0000","last_clean":"2023-10-18T12:25:14.214524+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:25:14.214524+0000","last_undegraded":"2023-10-18T12:25:14.214524+0000","last_fullsized":"2023-10-18T12:25:14.214524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216149+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:25:14.216149+0000","last_peered":"2023-10-18T12:25:14.216149+0000","last_clean":"2023-10-18T12:25:14.216149+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:25:14.216149+0000","last_undegraded":"2023-10-18T12:25:14.216149+0000","last_fullsized":"2023-10-18T12:25:14.216149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.220890+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:25:15.220890+0000","last_peered":"2023-10-18T12:25:15.220890+0000","last_clean":"2023-10-18T12:25:15.220890+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:25:15.220890+0000","last_undegraded":"2023-10-18T12:25:15.220890+0000","last_fullsized":"2023-10-18T12:25:15.220890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210834+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:25:14.210834+0000","last_peered":"2023-10-18T12:25:14.210834+0000","last_clean":"2023-10-18T12:25:14.210834+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:25:14.210834+0000","last_undegraded":"2023-10-18T12:25:14.210834+0000","last_fullsized":"2023-10-18T12:25:14.210834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214117+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:25:14.214117+0000","last_peered":"2023-10-18T12:25:14.214117+0000","last_clean":"2023-10-18T12:25:14.214117+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:25:14.214117+0000","last_undegraded":"2023-10-18T12:25:14.214117+0000","last_fullsized":"2023-10-18T12:25:14.214117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211350+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:14.211350+0000","last_peered":"2023-10-18T12:25:14.211350+0000","last_clean":"2023-10-18T12:25:14.211350+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:14.211350+0000","last_undegraded":"2023-10-18T12:25:14.211350+0000","last_fullsized":"2023-10-18T12:25:14.211350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210884+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:25:14.210884+0000","last_peered":"2023-10-18T12:25:14.210884+0000","last_clean":"2023-10-18T12:25:14.210884+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:25:14.210884+0000","last_undegraded":"2023-10-18T12:25:14.210884+0000","last_fullsized":"2023-10-18T12:25:14.210884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211266+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:25:14.211266+0000","last_peered":"2023-10-18T12:25:14.211266+0000","last_clean":"2023-10-18T12:25:14.211266+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:25:14.211266+0000","last_undegraded":"2023-10-18T12:25:14.211266+0000","last_fullsized":"2023-10-18T12:25:14.211266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212507+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:14.212507+0000","last_peered":"2023-10-18T12:25:14.212507+0000","last_clean":"2023-10-18T12:25:14.212507+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:14.212507+0000","last_undegraded":"2023-10-18T12:25:14.212507+0000","last_fullsized":"2023-10-18T12:25:14.212507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216327+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:25:14.216327+0000","last_peered":"2023-10-18T12:25:14.216327+0000","last_clean":"2023-10-18T12:25:14.216327+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:25:14.216327+0000","last_undegraded":"2023-10-18T12:25:14.216327+0000","last_fullsized":"2023-10-18T12:25:14.216327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210067+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:25:14.210067+0000","last_peered":"2023-10-18T12:25:14.210067+0000","last_clean":"2023-10-18T12:25:14.210067+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:25:14.210067+0000","last_undegraded":"2023-10-18T12:25:14.210067+0000","last_fullsized":"2023-10-18T12:25:14.210067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214357+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:25:14.214357+0000","last_peered":"2023-10-18T12:25:14.214357+0000","last_clean":"2023-10-18T12:25:14.214357+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:25:14.214357+0000","last_undegraded":"2023-10-18T12:25:14.214357+0000","last_fullsized":"2023-10-18T12:25:14.214357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211315+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:25:14.211315+0000","last_peered":"2023-10-18T12:25:14.211315+0000","last_clean":"2023-10-18T12:25:14.211315+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:25:14.211315+0000","last_undegraded":"2023-10-18T12:25:14.211315+0000","last_fullsized":"2023-10-18T12:25:14.211315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210851+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:14.210851+0000","last_peered":"2023-10-18T12:25:14.210851+0000","last_clean":"2023-10-18T12:25:14.210851+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:14.210851+0000","last_undegraded":"2023-10-18T12:25:14.210851+0000","last_fullsized":"2023-10-18T12:25:14.210851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218308+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:25:14.218308+0000","last_peered":"2023-10-18T12:25:14.218308+0000","last_clean":"2023-10-18T12:25:14.218308+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:25:14.218308+0000","last_undegraded":"2023-10-18T12:25:14.218308+0000","last_fullsized":"2023-10-18T12:25:14.218308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211578+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:25:14.211578+0000","last_peered":"2023-10-18T12:25:14.211578+0000","last_clean":"2023-10-18T12:25:14.211578+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:25:14.211578+0000","last_undegraded":"2023-10-18T12:25:14.211578+0000","last_fullsized":"2023-10-18T12:25:14.211578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210274+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:14.210274+0000","last_peered":"2023-10-18T12:25:14.210274+0000","last_clean":"2023-10-18T12:25:14.210274+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:14.210274+0000","last_undegraded":"2023-10-18T12:25:14.210274+0000","last_fullsized":"2023-10-18T12:25:14.210274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210493+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:25:14.210493+0000","last_peered":"2023-10-18T12:25:14.210493+0000","last_clean":"2023-10-18T12:25:14.210493+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:25:14.210493+0000","last_undegraded":"2023-10-18T12:25:14.210493+0000","last_fullsized":"2023-10-18T12:25:14.210493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225441+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:25:15.225441+0000","last_peered":"2023-10-18T12:25:15.225441+0000","last_clean":"2023-10-18T12:25:15.225441+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:25:15.225441+0000","last_undegraded":"2023-10-18T12:25:15.225441+0000","last_fullsized":"2023-10-18T12:25:15.225441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211245+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:14.211245+0000","last_peered":"2023-10-18T12:25:14.211245+0000","last_clean":"2023-10-18T12:25:14.211245+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:14.211245+0000","last_undegraded":"2023-10-18T12:25:14.211245+0000","last_fullsized":"2023-10-18T12:25:14.211245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214225+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:25:14.214225+0000","last_peered":"2023-10-18T12:25:14.214225+0000","last_clean":"2023-10-18T12:25:14.214225+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:25:14.214225+0000","last_undegraded":"2023-10-18T12:25:14.214225+0000","last_fullsized":"2023-10-18T12:25:14.214225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212847+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:25:14.212847+0000","last_peered":"2023-10-18T12:25:14.212847+0000","last_clean":"2023-10-18T12:25:14.212847+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:25:14.212847+0000","last_undegraded":"2023-10-18T12:25:14.212847+0000","last_fullsized":"2023-10-18T12:25:14.212847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210805+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:25:14.210805+0000","last_peered":"2023-10-18T12:25:14.210805+0000","last_clean":"2023-10-18T12:25:14.210805+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:25:14.210805+0000","last_undegraded":"2023-10-18T12:25:14.210805+0000","last_fullsized":"2023-10-18T12:25:14.210805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218547+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:25:14.218547+0000","last_peered":"2023-10-18T12:25:14.218547+0000","last_clean":"2023-10-18T12:25:14.218547+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:25:14.218547+0000","last_undegraded":"2023-10-18T12:25:14.218547+0000","last_fullsized":"2023-10-18T12:25:14.218547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208516+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:25:14.208516+0000","last_peered":"2023-10-18T12:25:14.208516+0000","last_clean":"2023-10-18T12:25:14.208516+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:25:14.208516+0000","last_undegraded":"2023-10-18T12:25:14.208516+0000","last_fullsized":"2023-10-18T12:25:14.208516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214769+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:25:14.214769+0000","last_peered":"2023-10-18T12:25:14.214769+0000","last_clean":"2023-10-18T12:25:14.214769+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:25:14.214769+0000","last_undegraded":"2023-10-18T12:25:14.214769+0000","last_fullsized":"2023-10-18T12:25:14.214769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211339+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:25:14.211339+0000","last_peered":"2023-10-18T12:25:14.211339+0000","last_clean":"2023-10-18T12:25:14.211339+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:25:14.211339+0000","last_undegraded":"2023-10-18T12:25:14.211339+0000","last_fullsized":"2023-10-18T12:25:14.211339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211786+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:25:14.211786+0000","last_peered":"2023-10-18T12:25:14.211786+0000","last_clean":"2023-10-18T12:25:14.211786+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:25:14.211786+0000","last_undegraded":"2023-10-18T12:25:14.211786+0000","last_fullsized":"2023-10-18T12:25:14.211786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210728+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:25:14.210728+0000","last_peered":"2023-10-18T12:25:14.210728+0000","last_clean":"2023-10-18T12:25:14.210728+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:25:14.210728+0000","last_undegraded":"2023-10-18T12:25:14.210728+0000","last_fullsized":"2023-10-18T12:25:14.210728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211556+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:25:14.211556+0000","last_peered":"2023-10-18T12:25:14.211556+0000","last_clean":"2023-10-18T12:25:14.211556+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:25:14.211556+0000","last_undegraded":"2023-10-18T12:25:14.211556+0000","last_fullsized":"2023-10-18T12:25:14.211556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214616+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:25:14.214616+0000","last_peered":"2023-10-18T12:25:14.214616+0000","last_clean":"2023-10-18T12:25:14.214616+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:25:14.214616+0000","last_undegraded":"2023-10-18T12:25:14.214616+0000","last_fullsized":"2023-10-18T12:25:14.214616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208435+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:25:14.208435+0000","last_peered":"2023-10-18T12:25:14.208435+0000","last_clean":"2023-10-18T12:25:14.208435+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:25:14.208435+0000","last_undegraded":"2023-10-18T12:25:14.208435+0000","last_fullsized":"2023-10-18T12:25:14.208435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214670+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:25:14.214670+0000","last_peered":"2023-10-18T12:25:14.214670+0000","last_clean":"2023-10-18T12:25:14.214670+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:25:14.214670+0000","last_undegraded":"2023-10-18T12:25:14.214670+0000","last_fullsized":"2023-10-18T12:25:14.214670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61683,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":109,"num_read_kb":116,"num_write":72,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383707,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1,"5min":1,"15min":1},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85899999999999999}]},{"osd":9,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.754},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.90300000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.80000000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1160475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.63400000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80500000000000005,"5min":0.999,"15min":0.999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.03,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78000000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.439}]},{"osd":5,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.52700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.59699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.99399999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.752,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99299999999999999,"5min":1.099,"15min":1.099},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76900000000000002,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.754},{"interface":"front","average":{"1min":0.747,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61399999999999999}]},{"osd":9,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0269999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.82999999999999996,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.96299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73499999999999999}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.65100000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":517802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50600000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.56899999999999995,"15min":0.59099999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.60699999999999998,"5min":1.167,"15min":1.167},"last":0.39100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0109999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89200000000000002,"5min":1.155,"15min":1.155},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84199999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498}]},{"osd":3,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91600000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.69799999999999995,"15min":0.72299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97799999999999998,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":0.871},"min":{"1min":0.628,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.98099999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.80200000000000005,"15min":0.82599999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.85199999999999998,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.76100000000000001,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88100000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.89200000000000002,"15min":0.91400000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.90900000000000003,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.90000000000000002,"15min":0.92700000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.89400000000000002,"5min":1.556,"15min":1.556},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.85299999999999998,"15min":0.87},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.753}]},{"osd":8,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95599999999999996,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.996,"5min":1.357,"15min":1.357},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.89600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.92300000000000004,"15min":0.93999999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1459999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.94499999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.63},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":1.05}]}]},{"osd":8,"up_from":60,"seq":257698037799,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":513706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.44800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.436,"15min":0.435},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.54800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47299999999999998,"15min":0.47799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.54600000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.82499999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.85099999999999998,"15min":0.879},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.4399999999999999,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.76600000000000001,"15min":0.80300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69399999999999995,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.67200000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71999999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.752}]},{"osd":5,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":1.117,"15min":1.117},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66300000000000003,"5min":1.099,"15min":1.099},"last":0.86599999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94999999999999996,"5min":1.325,"15min":1.325},"last":0.99099999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91200000000000003,"5min":1.272,"15min":1.272},"last":1.03},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84999999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.91800000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96799999999999997,"5min":1.044,"15min":1.044},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0409999999999999,"5min":1.26,"15min":1.26},"last":1.0589999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.74199999999999999,"15min":0.71999999999999997},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.93999999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":1.012},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.90300000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1108170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.40699999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58399999999999996,"15min":0.56100000000000005},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.497,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89500000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.42899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.749,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.34100000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64400000000000002,"15min":0.61599999999999999},"min":{"1min":0.51400000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83599999999999997,"5min":0.999,"15min":0.999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97099999999999997,"5min":1.137,"15min":1.137},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.49299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99399999999999999,"5min":1.071,"15min":1.071},"last":0.54100000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.504}]},{"osd":9,"last update":"Wed Oct 18 12:24:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.61199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.70699999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79600000000000004,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98799999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.54300000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.622}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72499999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462960,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":509028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.59799999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79300000000000004,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.755},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59199999999999997,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98299999999999998,"5min":1.028,"15min":1.028},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0009999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86799999999999999,"5min":1.234,"15min":1.234},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.97599999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.03}]},{"osd":8,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.80300000000000005,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.004}]},{"osd":9,"last update":"Wed Oct 18 12:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.64200000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80800000000000005,"15min":0.81399999999999995},"min":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.89800000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":0.94999999999999996,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":7.9290000000000003,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.94699999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.077},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":1.046}]}]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":569913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":1.161,"15min":1.161},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79100000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89200000000000002,"5min":1.454,"15min":1.454},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.61099999999999999,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96399999999999997,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.64800000000000002,"15min":0.59099999999999997},"min":{"1min":0.63200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":1.298,"15min":1.298},"last":0.749}]},{"osd":6,"last update":"Wed Oct 18 12:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.55400000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.496},{"interface":"front","average":{"1min":0.625,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.623}]},{"osd":7,"last update":"Wed Oct 18 12:25:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.31,"15min":0.31},"max":{"1min":0.91500000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67900000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.103,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71699999999999997,"15min":0.69299999999999995},"min":{"1min":0.58299999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.53300000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72699999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.871}]}]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":509118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.53400000000000003},"min":{"1min":0.40500000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.071},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.081}]},{"osd":2,"last update":"Wed Oct 18 12:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.67900000000000005,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.379,"15min":0.379},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.67200000000000004,"15min":0.59599999999999997},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.024}]},{"osd":4,"last update":"Wed Oct 18 12:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60999999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99099999999999999,"5min":1.272,"15min":1.272},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.59799999999999998},"min":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.996}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.66200000000000003},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.65700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.879}]},{"osd":6,"last update":"Wed Oct 18 12:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70499999999999996,"15min":0.66900000000000004},"min":{"1min":0.56699999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.119}]},{"osd":7,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.92800000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":0.71799999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1639999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.92400000000000004,"15min":0.89500000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.996,"5min":0.91000000000000003,"15min":0.89600000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.5710000000000002,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.90900000000000003}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.04},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.0569999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378693,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":569674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55500000000000005,"15min":0.51700000000000002},"min":{"1min":0.47299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.499},{"interface":"front","average":{"1min":0.629,"5min":0.56000000000000005,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.749}]},{"osd":4,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.34000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73799999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.44},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.42299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.623}]},{"osd":7,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58999999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.747,"15min":0.746},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98199999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.083,"15min":1.083},"last":0.875}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79000000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149757,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":509132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69299999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78900000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.60499999999999998},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.01},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":1.645}]},{"osd":2,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93200000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.66200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.056}]},{"osd":4,"last update":"Wed Oct 18 12:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.876,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.67800000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.877,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.78400000000000003,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":1.093},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.60099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":0.90800000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.86099999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":1.3240000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.536},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.77600000000000002,"15min":0.73399999999999999},"min":{"1min":0.61299999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.84899999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97599999999999998,"5min":1.155,"15min":1.155},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.008,"5min":1.26,"15min":1.26},"last":0.92900000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.498},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.90600000000000003,"15min":0.89400000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":0.97499999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":4.8879999999999999,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":1.1339999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Oct 18 12:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.622},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57599999999999996,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.04,"15min":1.04},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed Oct 18 12:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.54600000000000004},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82099999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":1.22,"15min":1.22},"last":0.52700000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.73299999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.93899999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.879,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.75900000000000001}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.80800000000000005}]},{"osd":9,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80300000000000005,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.05,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.871},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.871,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.746}]},{"osd":10,"last update":"Wed Oct 18 12:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.67100000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.92400000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1103343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67100000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.442},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41299999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54500000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84699999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81599999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:25:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.48199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.64500000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.629},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46899999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86499999999999999,"15min":0.89500000000000002},"min":{"1min":0.55100000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":4.548,"15min":4.548},"last":0.69199999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:25:25.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-18T12:25:26.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:25:26.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:25:26.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:26.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:25:26.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:25:26.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:26.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:25:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:25:26.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:25:26.695 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:27.274 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:26 smithi070 ceph-mon[98639]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:25:27.274 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:26 smithi070 ceph-mon[98639]: pgmap v295: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 885 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:26 smithi152 ceph-mon[134767]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:25:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:26 smithi152 ceph-mon[134767]: pgmap v295: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 885 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:26 smithi186 ceph-mon[135253]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:25:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:26 smithi186 ceph-mon[135253]: pgmap v295: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 885 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2023-10-18T12:25:28.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3041489352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f01008a0 msgr2=0x7fb1f0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:28.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3041489352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f01008a0 0x7fb1f0100d20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009990 tx=0x7fb1dc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3041489352 shutdown_connections 2023-10-18T12:25:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3041489352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f0101260 0x7fb1f010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3041489352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f01057c0 0x7fb1f0105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3041489352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f01008a0 0x7fb1f0100d20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3041489352 >> 172.21.15.70:0/3041489352 conn(0x7fb1f00779e0 msgr2=0x7fb1f0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3041489352 shutdown_connections 2023-10-18T12:25:28.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.202+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3041489352 wait complete. 2023-10-18T12:25:28.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.203+0000 7fb1f6e93700 1 Processor -- start 2023-10-18T12:25:28.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.203+0000 7fb1f6e93700 1 -- start start 2023-10-18T12:25:28.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.203+0000 7fb1f6e93700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f01008a0 0x7fb1f0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:28.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f6e93700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 0x7fb1f0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:28.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f6e93700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f01057c0 0x7fb1f019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:28.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f6e93700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f019dde0 con 0x7fb1f0101260 2023-10-18T12:25:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f6e93700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f019df50 con 0x7fb1f01057c0 2023-10-18T12:25:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f6e93700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f019e110 con 0x7fb1f01008a0 2023-10-18T12:25:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 0x7fb1f0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f5430700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f01057c0 0x7fb1f019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 0x7fb1f0199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58806/0 (socket says 172.21.15.70:58806) 2023-10-18T12:25:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 -- 172.21.15.70:0/3754448981 learned_addr learned my addr 172.21.15.70:0/3754448981 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1f4c2f700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f01008a0 0x7fb1f0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 -- 172.21.15.70:0/3754448981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f01008a0 msgr2=0x7fb1f0198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f01008a0 0x7fb1f0198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 -- 172.21.15.70:0/3754448981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f01057c0 msgr2=0x7fb1f019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:28.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f01057c0 0x7fb1f019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 -- 172.21.15.70:0/3754448981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0095f0 con 0x7fb1f0101260 2023-10-18T12:25:28.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.204+0000 7fb1effff700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 0x7fb1f0199420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0095c0 tx=0x7fb1dc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:28.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.205+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc01d070 con 0x7fb1f0101260 2023-10-18T12:25:28.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.205+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1dc005050 con 0x7fb1f0101260 2023-10-18T12:25:28.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.205+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004b10 con 0x7fb1f0101260 2023-10-18T12:25:28.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.205+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f019e390 con 0x7fb1f0101260 2023-10-18T12:25:28.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.205+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f01a5a80 con 0x7fb1f0101260 2023-10-18T12:25:28.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.206+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f0101f30 con 0x7fb1f0101260 2023-10-18T12:25:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.208+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fb1dc0051c0 con 0x7fb1f0101260 2023-10-18T12:25:28.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.209+0000 7fb1edffb700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e00a1560 0x7fb1e00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:28.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.209+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fb1dc0ce530 con 0x7fb1f0101260 2023-10-18T12:25:28.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.209+0000 7fb1f4c2f700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e00a1560 0x7fb1e00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:28.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.210+0000 7fb1f4c2f700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e00a1560 0x7fb1e00a3a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4005950 tx=0x7fb1e40058e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:28.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.212+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb1dc097670 con 0x7fb1f0101260 2023-10-18T12:25:28.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.407+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1f019a260 con 0x7fb1e00a1560 2023-10-18T12:25:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.420+0000 7fb1edffb700 1 -- 172.21.15.70:0/3754448981 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427375 (secure 0 0 0) 0x7fb1f019a260 con 0x7fb1e00a1560 2023-10-18T12:25:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:28.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e00a1560 msgr2=0x7fb1e00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:28.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e00a1560 0x7fb1e00a3a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4005950 tx=0x7fb1e40058e0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 msgr2=0x7fb1f0199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 0x7fb1f0199420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0095c0 tx=0x7fb1dc00f690 comp rx=0 tx=0).stop 2023-10-18T12:25:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 shutdown_connections 2023-10-18T12:25:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1e00a1560 0x7fb1e00a3a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1f01057c0 0x7fb1f019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f01008a0 0x7fb1f0198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 --2- 172.21.15.70:0/3754448981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f0101260 0x7fb1f0199420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 >> 172.21.15.70:0/3754448981 conn(0x7fb1f00779e0 msgr2=0x7fb1f00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 shutdown_connections 2023-10-18T12:25:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:28.425+0000 7fb1f6e93700 1 -- 172.21.15.70:0/3754448981 wait complete. 2023-10-18T12:25:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:25:28.885 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-18T12:25:27.611520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652297,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":255,"num_read_kb":240,"num_write":255,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338260,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603564,"statfs":{"total":1151940427776,"available":1151594049536,"internally_reserved":0,"allocated":14315520,"data_stored":8096653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":14,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003762"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215639+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:25:14.215639+0000","last_peered":"2023-10-18T12:25:14.215639+0000","last_clean":"2023-10-18T12:25:14.215639+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:25:14.215639+0000","last_undegraded":"2023-10-18T12:25:14.215639+0000","last_fullsized":"2023-10-18T12:25:14.215639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218812+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:25:14.218812+0000","last_peered":"2023-10-18T12:25:14.218812+0000","last_clean":"2023-10-18T12:25:14.218812+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:25:14.218812+0000","last_undegraded":"2023-10-18T12:25:14.218812+0000","last_fullsized":"2023-10-18T12:25:14.218812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218728+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:25:14.218728+0000","last_peered":"2023-10-18T12:25:14.218728+0000","last_clean":"2023-10-18T12:25:14.218728+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:25:14.218728+0000","last_undegraded":"2023-10-18T12:25:14.218728+0000","last_fullsized":"2023-10-18T12:25:14.218728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212709+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:25:14.212709+0000","last_peered":"2023-10-18T12:25:14.212709+0000","last_clean":"2023-10-18T12:25:14.212709+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:25:14.212709+0000","last_undegraded":"2023-10-18T12:25:14.212709+0000","last_fullsized":"2023-10-18T12:25:14.212709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216241+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:25:14.216241+0000","last_peered":"2023-10-18T12:25:14.216241+0000","last_clean":"2023-10-18T12:25:14.216241+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:25:14.216241+0000","last_undegraded":"2023-10-18T12:25:14.216241+0000","last_fullsized":"2023-10-18T12:25:14.216241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212255+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:25:14.212255+0000","last_peered":"2023-10-18T12:25:14.212255+0000","last_clean":"2023-10-18T12:25:14.212255+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:25:14.212255+0000","last_undegraded":"2023-10-18T12:25:14.212255+0000","last_fullsized":"2023-10-18T12:25:14.212255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212186+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:14.212186+0000","last_peered":"2023-10-18T12:25:14.212186+0000","last_clean":"2023-10-18T12:25:14.212186+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:14.212186+0000","last_undegraded":"2023-10-18T12:25:14.212186+0000","last_fullsized":"2023-10-18T12:25:14.212186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210824+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:25:14.210824+0000","last_peered":"2023-10-18T12:25:14.210824+0000","last_clean":"2023-10-18T12:25:14.210824+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:25:14.210824+0000","last_undegraded":"2023-10-18T12:25:14.210824+0000","last_fullsized":"2023-10-18T12:25:14.210824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215120+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:25:14.215120+0000","last_peered":"2023-10-18T12:25:14.215120+0000","last_clean":"2023-10-18T12:25:14.215120+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:25:14.215120+0000","last_undegraded":"2023-10-18T12:25:14.215120+0000","last_fullsized":"2023-10-18T12:25:14.215120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210943+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:25:14.210943+0000","last_peered":"2023-10-18T12:25:14.210943+0000","last_clean":"2023-10-18T12:25:14.210943+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:25:14.210943+0000","last_undegraded":"2023-10-18T12:25:14.210943+0000","last_fullsized":"2023-10-18T12:25:14.210943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210769+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:14.210769+0000","last_peered":"2023-10-18T12:25:14.210769+0000","last_clean":"2023-10-18T12:25:14.210769+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:14.210769+0000","last_undegraded":"2023-10-18T12:25:14.210769+0000","last_fullsized":"2023-10-18T12:25:14.210769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212634+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:14.212634+0000","last_peered":"2023-10-18T12:25:14.212634+0000","last_clean":"2023-10-18T12:25:14.212634+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:14.212634+0000","last_undegraded":"2023-10-18T12:25:14.212634+0000","last_fullsized":"2023-10-18T12:25:14.212634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208392+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:25:14.208392+0000","last_peered":"2023-10-18T12:25:14.208392+0000","last_clean":"2023-10-18T12:25:14.208392+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:25:14.208392+0000","last_undegraded":"2023-10-18T12:25:14.208392+0000","last_fullsized":"2023-10-18T12:25:14.208392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218307+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:25:14.218307+0000","last_peered":"2023-10-18T12:25:14.218307+0000","last_clean":"2023-10-18T12:25:14.218307+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:25:14.218307+0000","last_undegraded":"2023-10-18T12:25:14.218307+0000","last_fullsized":"2023-10-18T12:25:14.218307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218269+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:25:14.218269+0000","last_peered":"2023-10-18T12:25:14.218269+0000","last_clean":"2023-10-18T12:25:14.218269+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:25:14.218269+0000","last_undegraded":"2023-10-18T12:25:14.218269+0000","last_fullsized":"2023-10-18T12:25:14.218269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214462+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:25:14.214462+0000","last_peered":"2023-10-18T12:25:14.214462+0000","last_clean":"2023-10-18T12:25:14.214462+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:25:14.214462+0000","last_undegraded":"2023-10-18T12:25:14.214462+0000","last_fullsized":"2023-10-18T12:25:14.214462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215783+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:25:14.215783+0000","last_peered":"2023-10-18T12:25:14.215783+0000","last_clean":"2023-10-18T12:25:14.215783+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:25:14.215783+0000","last_undegraded":"2023-10-18T12:25:14.215783+0000","last_fullsized":"2023-10-18T12:25:14.215783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214795+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:25:14.214795+0000","last_peered":"2023-10-18T12:25:14.214795+0000","last_clean":"2023-10-18T12:25:14.214795+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:25:14.214795+0000","last_undegraded":"2023-10-18T12:25:14.214795+0000","last_fullsized":"2023-10-18T12:25:14.214795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213474+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:25:14.213474+0000","last_peered":"2023-10-18T12:25:14.213474+0000","last_clean":"2023-10-18T12:25:14.213474+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:25:14.213474+0000","last_undegraded":"2023-10-18T12:25:14.213474+0000","last_fullsized":"2023-10-18T12:25:14.213474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212928+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:25:14.212928+0000","last_peered":"2023-10-18T12:25:14.212928+0000","last_clean":"2023-10-18T12:25:14.212928+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:25:14.212928+0000","last_undegraded":"2023-10-18T12:25:14.212928+0000","last_fullsized":"2023-10-18T12:25:14.212928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214873+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:25:14.214873+0000","last_peered":"2023-10-18T12:25:14.214873+0000","last_clean":"2023-10-18T12:25:14.214873+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:25:14.214873+0000","last_undegraded":"2023-10-18T12:25:14.214873+0000","last_fullsized":"2023-10-18T12:25:14.214873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211210+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:25:14.211210+0000","last_peered":"2023-10-18T12:25:14.211210+0000","last_clean":"2023-10-18T12:25:14.211210+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:25:14.211210+0000","last_undegraded":"2023-10-18T12:25:14.211210+0000","last_fullsized":"2023-10-18T12:25:14.211210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210737+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:14.210737+0000","last_peered":"2023-10-18T12:25:14.210737+0000","last_clean":"2023-10-18T12:25:14.210737+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:14.210737+0000","last_undegraded":"2023-10-18T12:25:14.210737+0000","last_fullsized":"2023-10-18T12:25:14.210737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211356+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:25:14.211356+0000","last_peered":"2023-10-18T12:25:14.211356+0000","last_clean":"2023-10-18T12:25:14.211356+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:25:14.211356+0000","last_undegraded":"2023-10-18T12:25:14.211356+0000","last_fullsized":"2023-10-18T12:25:14.211356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218636+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:25:14.218636+0000","last_peered":"2023-10-18T12:25:14.218636+0000","last_clean":"2023-10-18T12:25:14.218636+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:25:14.218636+0000","last_undegraded":"2023-10-18T12:25:14.218636+0000","last_fullsized":"2023-10-18T12:25:14.218636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211459+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:14.211459+0000","last_peered":"2023-10-18T12:25:14.211459+0000","last_clean":"2023-10-18T12:25:14.211459+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:14.211459+0000","last_undegraded":"2023-10-18T12:25:14.211459+0000","last_fullsized":"2023-10-18T12:25:14.211459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215742+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:25:14.215742+0000","last_peered":"2023-10-18T12:25:14.215742+0000","last_clean":"2023-10-18T12:25:14.215742+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:25:14.215742+0000","last_undegraded":"2023-10-18T12:25:14.215742+0000","last_fullsized":"2023-10-18T12:25:14.215742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215091+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:25:14.215091+0000","last_peered":"2023-10-18T12:25:14.215091+0000","last_clean":"2023-10-18T12:25:14.215091+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:25:14.215091+0000","last_undegraded":"2023-10-18T12:25:14.215091+0000","last_fullsized":"2023-10-18T12:25:14.215091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210540+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:14.210540+0000","last_peered":"2023-10-18T12:25:14.210540+0000","last_clean":"2023-10-18T12:25:14.210540+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:14.210540+0000","last_undegraded":"2023-10-18T12:25:14.210540+0000","last_fullsized":"2023-10-18T12:25:14.210540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216726+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:25:14.216726+0000","last_peered":"2023-10-18T12:25:14.216726+0000","last_clean":"2023-10-18T12:25:14.216726+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:25:14.216726+0000","last_undegraded":"2023-10-18T12:25:14.216726+0000","last_fullsized":"2023-10-18T12:25:14.216726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211493+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:25:14.211493+0000","last_peered":"2023-10-18T12:25:14.211493+0000","last_clean":"2023-10-18T12:25:14.211493+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:25:14.211493+0000","last_undegraded":"2023-10-18T12:25:14.211493+0000","last_fullsized":"2023-10-18T12:25:14.211493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217322+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:25:15.217322+0000","last_peered":"2023-10-18T12:25:15.217322+0000","last_clean":"2023-10-18T12:25:15.217322+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:25:15.217322+0000","last_undegraded":"2023-10-18T12:25:15.217322+0000","last_fullsized":"2023-10-18T12:25:15.217322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211310+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:25:14.211310+0000","last_peered":"2023-10-18T12:25:14.211310+0000","last_clean":"2023-10-18T12:25:14.211310+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:25:14.211310+0000","last_undegraded":"2023-10-18T12:25:14.211310+0000","last_fullsized":"2023-10-18T12:25:14.211310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215778+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:25:14.215778+0000","last_peered":"2023-10-18T12:25:14.215778+0000","last_clean":"2023-10-18T12:25:14.215778+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:25:14.215778+0000","last_undegraded":"2023-10-18T12:25:14.215778+0000","last_fullsized":"2023-10-18T12:25:14.215778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211137+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:25:14.211137+0000","last_peered":"2023-10-18T12:25:14.211137+0000","last_clean":"2023-10-18T12:25:14.211137+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:25:14.211137+0000","last_undegraded":"2023-10-18T12:25:14.211137+0000","last_fullsized":"2023-10-18T12:25:14.211137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210401+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:25:14.210401+0000","last_peered":"2023-10-18T12:25:14.210401+0000","last_clean":"2023-10-18T12:25:14.210401+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:25:14.210401+0000","last_undegraded":"2023-10-18T12:25:14.210401+0000","last_fullsized":"2023-10-18T12:25:14.210401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211447+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:25:14.211447+0000","last_peered":"2023-10-18T12:25:14.211447+0000","last_clean":"2023-10-18T12:25:14.211447+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:25:14.211447+0000","last_undegraded":"2023-10-18T12:25:14.211447+0000","last_fullsized":"2023-10-18T12:25:14.211447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215355+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:25:14.215355+0000","last_peered":"2023-10-18T12:25:14.215355+0000","last_clean":"2023-10-18T12:25:14.215355+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:25:14.215355+0000","last_undegraded":"2023-10-18T12:25:14.215355+0000","last_fullsized":"2023-10-18T12:25:14.215355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207597+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:25:14.207597+0000","last_peered":"2023-10-18T12:25:14.207597+0000","last_clean":"2023-10-18T12:25:14.207597+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:25:14.207597+0000","last_undegraded":"2023-10-18T12:25:14.207597+0000","last_fullsized":"2023-10-18T12:25:14.207597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210728+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:25:14.210728+0000","last_peered":"2023-10-18T12:25:14.210728+0000","last_clean":"2023-10-18T12:25:14.210728+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:25:14.210728+0000","last_undegraded":"2023-10-18T12:25:14.210728+0000","last_fullsized":"2023-10-18T12:25:14.210728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218811+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:25:14.218811+0000","last_peered":"2023-10-18T12:25:14.218811+0000","last_clean":"2023-10-18T12:25:14.218811+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:25:14.218811+0000","last_undegraded":"2023-10-18T12:25:14.218811+0000","last_fullsized":"2023-10-18T12:25:14.218811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210651+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:25:14.210651+0000","last_peered":"2023-10-18T12:25:14.210651+0000","last_clean":"2023-10-18T12:25:14.210651+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:25:14.210651+0000","last_undegraded":"2023-10-18T12:25:14.210651+0000","last_fullsized":"2023-10-18T12:25:14.210651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213990+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:25:14.213990+0000","last_peered":"2023-10-18T12:25:14.213990+0000","last_clean":"2023-10-18T12:25:14.213990+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:25:14.213990+0000","last_undegraded":"2023-10-18T12:25:14.213990+0000","last_fullsized":"2023-10-18T12:25:14.213990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215384+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:25:14.215384+0000","last_peered":"2023-10-18T12:25:14.215384+0000","last_clean":"2023-10-18T12:25:14.215384+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:25:14.215384+0000","last_undegraded":"2023-10-18T12:25:14.215384+0000","last_fullsized":"2023-10-18T12:25:14.215384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213793+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:25:14.213793+0000","last_peered":"2023-10-18T12:25:14.213793+0000","last_clean":"2023-10-18T12:25:14.213793+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:25:14.213793+0000","last_undegraded":"2023-10-18T12:25:14.213793+0000","last_fullsized":"2023-10-18T12:25:14.213793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211053+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:14.211053+0000","last_peered":"2023-10-18T12:25:14.211053+0000","last_clean":"2023-10-18T12:25:14.211053+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:14.211053+0000","last_undegraded":"2023-10-18T12:25:14.211053+0000","last_fullsized":"2023-10-18T12:25:14.211053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218903+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:25:14.218903+0000","last_peered":"2023-10-18T12:25:14.218903+0000","last_clean":"2023-10-18T12:25:14.218903+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:25:14.218903+0000","last_undegraded":"2023-10-18T12:25:14.218903+0000","last_fullsized":"2023-10-18T12:25:14.218903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211621+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:14.211621+0000","last_peered":"2023-10-18T12:25:14.211621+0000","last_clean":"2023-10-18T12:25:14.211621+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:14.211621+0000","last_undegraded":"2023-10-18T12:25:14.211621+0000","last_fullsized":"2023-10-18T12:25:14.211621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211404+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:25:14.211404+0000","last_peered":"2023-10-18T12:25:14.211404+0000","last_clean":"2023-10-18T12:25:14.211404+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:25:14.211404+0000","last_undegraded":"2023-10-18T12:25:14.211404+0000","last_fullsized":"2023-10-18T12:25:14.211404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.216324+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:25:15.216324+0000","last_peered":"2023-10-18T12:25:15.216324+0000","last_clean":"2023-10-18T12:25:15.216324+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:25:15.216324+0000","last_undegraded":"2023-10-18T12:25:15.216324+0000","last_fullsized":"2023-10-18T12:25:15.216324+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213954+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:25:14.213954+0000","last_peered":"2023-10-18T12:25:14.213954+0000","last_clean":"2023-10-18T12:25:14.213954+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:25:14.213954+0000","last_undegraded":"2023-10-18T12:25:14.213954+0000","last_fullsized":"2023-10-18T12:25:14.213954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'29","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:26.243870+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:25:26.243870+0000","last_peered":"2023-10-18T12:25:26.243870+0000","last_clean":"2023-10-18T12:25:26.243870+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:25:26.243870+0000","last_undegraded":"2023-10-18T12:25:26.243870+0000","last_fullsized":"2023-10-18T12:25:26.243870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60885,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":114,"num_read_kb":116,"num_write":29,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215942+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:25:14.215942+0000","last_peered":"2023-10-18T12:25:14.215942+0000","last_clean":"2023-10-18T12:25:14.215942+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:25:14.215942+0000","last_undegraded":"2023-10-18T12:25:14.215942+0000","last_fullsized":"2023-10-18T12:25:14.215942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211801+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:25:14.211801+0000","last_peered":"2023-10-18T12:25:14.211801+0000","last_clean":"2023-10-18T12:25:14.211801+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:25:14.211801+0000","last_undegraded":"2023-10-18T12:25:14.211801+0000","last_fullsized":"2023-10-18T12:25:14.211801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211560+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:25:14.211560+0000","last_peered":"2023-10-18T12:25:14.211560+0000","last_clean":"2023-10-18T12:25:14.211560+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:25:14.211560+0000","last_undegraded":"2023-10-18T12:25:14.211560+0000","last_fullsized":"2023-10-18T12:25:14.211560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207775+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:25:14.207775+0000","last_peered":"2023-10-18T12:25:14.207775+0000","last_clean":"2023-10-18T12:25:14.207775+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:25:14.207775+0000","last_undegraded":"2023-10-18T12:25:14.207775+0000","last_fullsized":"2023-10-18T12:25:14.207775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214581+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:25:14.214581+0000","last_peered":"2023-10-18T12:25:14.214581+0000","last_clean":"2023-10-18T12:25:14.214581+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:25:14.214581+0000","last_undegraded":"2023-10-18T12:25:14.214581+0000","last_fullsized":"2023-10-18T12:25:14.214581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211113+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:25:14.211113+0000","last_peered":"2023-10-18T12:25:14.211113+0000","last_clean":"2023-10-18T12:25:14.211113+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:25:14.211113+0000","last_undegraded":"2023-10-18T12:25:14.211113+0000","last_fullsized":"2023-10-18T12:25:14.211113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.219532+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:15.219532+0000","last_peered":"2023-10-18T12:25:15.219532+0000","last_clean":"2023-10-18T12:25:15.219532+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:15.219532+0000","last_undegraded":"2023-10-18T12:25:15.219532+0000","last_fullsized":"2023-10-18T12:25:15.219532+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214765+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:25:14.214765+0000","last_peered":"2023-10-18T12:25:14.214765+0000","last_clean":"2023-10-18T12:25:14.214765+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:25:14.214765+0000","last_undegraded":"2023-10-18T12:25:14.214765+0000","last_fullsized":"2023-10-18T12:25:14.214765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210996+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:14.210996+0000","last_peered":"2023-10-18T12:25:14.210996+0000","last_clean":"2023-10-18T12:25:14.210996+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:14.210996+0000","last_undegraded":"2023-10-18T12:25:14.210996+0000","last_fullsized":"2023-10-18T12:25:14.210996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210833+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:14.210833+0000","last_peered":"2023-10-18T12:25:14.210833+0000","last_clean":"2023-10-18T12:25:14.210833+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:14.210833+0000","last_undegraded":"2023-10-18T12:25:14.210833+0000","last_fullsized":"2023-10-18T12:25:14.210833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208699+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:25:14.208699+0000","last_peered":"2023-10-18T12:25:14.208699+0000","last_clean":"2023-10-18T12:25:14.208699+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:25:14.208699+0000","last_undegraded":"2023-10-18T12:25:14.208699+0000","last_fullsized":"2023-10-18T12:25:14.208699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225793+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:25:15.225793+0000","last_peered":"2023-10-18T12:25:15.225793+0000","last_clean":"2023-10-18T12:25:15.225793+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:25:15.225793+0000","last_undegraded":"2023-10-18T12:25:15.225793+0000","last_fullsized":"2023-10-18T12:25:15.225793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211963+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:25:14.211963+0000","last_peered":"2023-10-18T12:25:14.211963+0000","last_clean":"2023-10-18T12:25:14.211963+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:25:14.211963+0000","last_undegraded":"2023-10-18T12:25:14.211963+0000","last_fullsized":"2023-10-18T12:25:14.211963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211886+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:25:14.211886+0000","last_peered":"2023-10-18T12:25:14.211886+0000","last_clean":"2023-10-18T12:25:14.211886+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:25:14.211886+0000","last_undegraded":"2023-10-18T12:25:14.211886+0000","last_fullsized":"2023-10-18T12:25:14.211886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.224004+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:25:15.224004+0000","last_peered":"2023-10-18T12:25:15.224004+0000","last_clean":"2023-10-18T12:25:15.224004+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:25:15.224004+0000","last_undegraded":"2023-10-18T12:25:15.224004+0000","last_fullsized":"2023-10-18T12:25:15.224004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215148+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:25:14.215148+0000","last_peered":"2023-10-18T12:25:14.215148+0000","last_clean":"2023-10-18T12:25:14.215148+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:25:14.215148+0000","last_undegraded":"2023-10-18T12:25:14.215148+0000","last_fullsized":"2023-10-18T12:25:14.215148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215695+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:25:14.215695+0000","last_peered":"2023-10-18T12:25:14.215695+0000","last_clean":"2023-10-18T12:25:14.215695+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:25:14.215695+0000","last_undegraded":"2023-10-18T12:25:14.215695+0000","last_fullsized":"2023-10-18T12:25:14.215695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214390+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:25:14.214390+0000","last_peered":"2023-10-18T12:25:14.214390+0000","last_clean":"2023-10-18T12:25:14.214390+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:25:14.214390+0000","last_undegraded":"2023-10-18T12:25:14.214390+0000","last_fullsized":"2023-10-18T12:25:14.214390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210689+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:25:14.210689+0000","last_peered":"2023-10-18T12:25:14.210689+0000","last_clean":"2023-10-18T12:25:14.210689+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:25:14.210689+0000","last_undegraded":"2023-10-18T12:25:14.210689+0000","last_fullsized":"2023-10-18T12:25:14.210689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208378+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:25:14.208378+0000","last_peered":"2023-10-18T12:25:14.208378+0000","last_clean":"2023-10-18T12:25:14.208378+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:25:14.208378+0000","last_undegraded":"2023-10-18T12:25:14.208378+0000","last_fullsized":"2023-10-18T12:25:14.208378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209917+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:25:14.209917+0000","last_peered":"2023-10-18T12:25:14.209917+0000","last_clean":"2023-10-18T12:25:14.209917+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:25:14.209917+0000","last_undegraded":"2023-10-18T12:25:14.209917+0000","last_fullsized":"2023-10-18T12:25:14.209917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211140+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:25:14.211140+0000","last_peered":"2023-10-18T12:25:14.211140+0000","last_clean":"2023-10-18T12:25:14.211140+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:25:14.211140+0000","last_undegraded":"2023-10-18T12:25:14.211140+0000","last_fullsized":"2023-10-18T12:25:14.211140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209020+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:25:14.209020+0000","last_peered":"2023-10-18T12:25:14.209020+0000","last_clean":"2023-10-18T12:25:14.209020+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:25:14.209020+0000","last_undegraded":"2023-10-18T12:25:14.209020+0000","last_fullsized":"2023-10-18T12:25:14.209020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210801+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:25:14.210801+0000","last_peered":"2023-10-18T12:25:14.210801+0000","last_clean":"2023-10-18T12:25:14.210801+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:25:14.210801+0000","last_undegraded":"2023-10-18T12:25:14.210801+0000","last_fullsized":"2023-10-18T12:25:14.210801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207506+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:25:14.207506+0000","last_peered":"2023-10-18T12:25:14.207506+0000","last_clean":"2023-10-18T12:25:14.207506+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:25:14.207506+0000","last_undegraded":"2023-10-18T12:25:14.207506+0000","last_fullsized":"2023-10-18T12:25:14.207506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210590+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:25:14.210590+0000","last_peered":"2023-10-18T12:25:14.210590+0000","last_clean":"2023-10-18T12:25:14.210590+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:25:14.210590+0000","last_undegraded":"2023-10-18T12:25:14.210590+0000","last_fullsized":"2023-10-18T12:25:14.210590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211226+0000","last_change":"2023-10-18T12:24:09.985663+0000","last_active":"2023-10-18T12:25:14.211226+0000","last_peered":"2023-10-18T12:25:14.211226+0000","last_clean":"2023-10-18T12:25:14.211226+0000","last_became_active":"2023-10-18T12:24:09.983659+0000","last_became_peered":"2023-10-18T12:24:09.983659+0000","last_unstale":"2023-10-18T12:25:14.211226+0000","last_undegraded":"2023-10-18T12:25:14.211226+0000","last_fullsized":"2023-10-18T12:25:14.211226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:39.706073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211540+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:25:14.211540+0000","last_peered":"2023-10-18T12:25:14.211540+0000","last_clean":"2023-10-18T12:25:14.211540+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:25:14.211540+0000","last_undegraded":"2023-10-18T12:25:14.211540+0000","last_fullsized":"2023-10-18T12:25:14.211540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210165+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:14.210165+0000","last_peered":"2023-10-18T12:25:14.210165+0000","last_clean":"2023-10-18T12:25:14.210165+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:14.210165+0000","last_undegraded":"2023-10-18T12:25:14.210165+0000","last_fullsized":"2023-10-18T12:25:14.210165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223137+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:25:15.223137+0000","last_peered":"2023-10-18T12:25:15.223137+0000","last_clean":"2023-10-18T12:25:15.223137+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:25:15.223137+0000","last_undegraded":"2023-10-18T12:25:15.223137+0000","last_fullsized":"2023-10-18T12:25:15.223137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207572+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:25:14.207572+0000","last_peered":"2023-10-18T12:25:14.207572+0000","last_clean":"2023-10-18T12:25:14.207572+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:25:14.207572+0000","last_undegraded":"2023-10-18T12:25:14.207572+0000","last_fullsized":"2023-10-18T12:25:14.207572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217523+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:25:15.217523+0000","last_peered":"2023-10-18T12:25:15.217523+0000","last_clean":"2023-10-18T12:25:15.217523+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:25:15.217523+0000","last_undegraded":"2023-10-18T12:25:15.217523+0000","last_fullsized":"2023-10-18T12:25:15.217523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214480+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:25:14.214480+0000","last_peered":"2023-10-18T12:25:14.214480+0000","last_clean":"2023-10-18T12:25:14.214480+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:25:14.214480+0000","last_undegraded":"2023-10-18T12:25:14.214480+0000","last_fullsized":"2023-10-18T12:25:14.214480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215870+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:25:14.215870+0000","last_peered":"2023-10-18T12:25:14.215870+0000","last_clean":"2023-10-18T12:25:14.215870+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:25:14.215870+0000","last_undegraded":"2023-10-18T12:25:14.215870+0000","last_fullsized":"2023-10-18T12:25:14.215870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208601+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:25:14.208601+0000","last_peered":"2023-10-18T12:25:14.208601+0000","last_clean":"2023-10-18T12:25:14.208601+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:25:14.208601+0000","last_undegraded":"2023-10-18T12:25:14.208601+0000","last_fullsized":"2023-10-18T12:25:14.208601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210763+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:25:14.210763+0000","last_peered":"2023-10-18T12:25:14.210763+0000","last_clean":"2023-10-18T12:25:14.210763+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:25:14.210763+0000","last_undegraded":"2023-10-18T12:25:14.210763+0000","last_fullsized":"2023-10-18T12:25:14.210763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214833+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:25:14.214833+0000","last_peered":"2023-10-18T12:25:14.214833+0000","last_clean":"2023-10-18T12:25:14.214833+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:25:14.214833+0000","last_undegraded":"2023-10-18T12:25:14.214833+0000","last_fullsized":"2023-10-18T12:25:14.214833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211642+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:25:14.211642+0000","last_peered":"2023-10-18T12:25:14.211642+0000","last_clean":"2023-10-18T12:25:14.211642+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:25:14.211642+0000","last_undegraded":"2023-10-18T12:25:14.211642+0000","last_fullsized":"2023-10-18T12:25:14.211642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212653+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:25:14.212653+0000","last_peered":"2023-10-18T12:25:14.212653+0000","last_clean":"2023-10-18T12:25:14.212653+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:25:14.212653+0000","last_undegraded":"2023-10-18T12:25:14.212653+0000","last_fullsized":"2023-10-18T12:25:14.212653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213075+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:14.213075+0000","last_peered":"2023-10-18T12:25:14.213075+0000","last_clean":"2023-10-18T12:25:14.213075+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:14.213075+0000","last_undegraded":"2023-10-18T12:25:14.213075+0000","last_fullsized":"2023-10-18T12:25:14.213075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218907+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:25:14.218907+0000","last_peered":"2023-10-18T12:25:14.218907+0000","last_clean":"2023-10-18T12:25:14.218907+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:25:14.218907+0000","last_undegraded":"2023-10-18T12:25:14.218907+0000","last_fullsized":"2023-10-18T12:25:14.218907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223250+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223250+0000","last_peered":"2023-10-18T12:25:15.223250+0000","last_clean":"2023-10-18T12:25:15.223250+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223250+0000","last_undegraded":"2023-10-18T12:25:15.223250+0000","last_fullsized":"2023-10-18T12:25:15.223250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214524+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:25:14.214524+0000","last_peered":"2023-10-18T12:25:14.214524+0000","last_clean":"2023-10-18T12:25:14.214524+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:25:14.214524+0000","last_undegraded":"2023-10-18T12:25:14.214524+0000","last_fullsized":"2023-10-18T12:25:14.214524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216149+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:25:14.216149+0000","last_peered":"2023-10-18T12:25:14.216149+0000","last_clean":"2023-10-18T12:25:14.216149+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:25:14.216149+0000","last_undegraded":"2023-10-18T12:25:14.216149+0000","last_fullsized":"2023-10-18T12:25:14.216149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.220890+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:25:15.220890+0000","last_peered":"2023-10-18T12:25:15.220890+0000","last_clean":"2023-10-18T12:25:15.220890+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:25:15.220890+0000","last_undegraded":"2023-10-18T12:25:15.220890+0000","last_fullsized":"2023-10-18T12:25:15.220890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210834+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:25:14.210834+0000","last_peered":"2023-10-18T12:25:14.210834+0000","last_clean":"2023-10-18T12:25:14.210834+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:25:14.210834+0000","last_undegraded":"2023-10-18T12:25:14.210834+0000","last_fullsized":"2023-10-18T12:25:14.210834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214117+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:25:14.214117+0000","last_peered":"2023-10-18T12:25:14.214117+0000","last_clean":"2023-10-18T12:25:14.214117+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:25:14.214117+0000","last_undegraded":"2023-10-18T12:25:14.214117+0000","last_fullsized":"2023-10-18T12:25:14.214117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211350+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:14.211350+0000","last_peered":"2023-10-18T12:25:14.211350+0000","last_clean":"2023-10-18T12:25:14.211350+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:14.211350+0000","last_undegraded":"2023-10-18T12:25:14.211350+0000","last_fullsized":"2023-10-18T12:25:14.211350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210884+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:25:14.210884+0000","last_peered":"2023-10-18T12:25:14.210884+0000","last_clean":"2023-10-18T12:25:14.210884+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:25:14.210884+0000","last_undegraded":"2023-10-18T12:25:14.210884+0000","last_fullsized":"2023-10-18T12:25:14.210884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211266+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:25:14.211266+0000","last_peered":"2023-10-18T12:25:14.211266+0000","last_clean":"2023-10-18T12:25:14.211266+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:25:14.211266+0000","last_undegraded":"2023-10-18T12:25:14.211266+0000","last_fullsized":"2023-10-18T12:25:14.211266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212507+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:14.212507+0000","last_peered":"2023-10-18T12:25:14.212507+0000","last_clean":"2023-10-18T12:25:14.212507+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:14.212507+0000","last_undegraded":"2023-10-18T12:25:14.212507+0000","last_fullsized":"2023-10-18T12:25:14.212507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216327+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:25:14.216327+0000","last_peered":"2023-10-18T12:25:14.216327+0000","last_clean":"2023-10-18T12:25:14.216327+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:25:14.216327+0000","last_undegraded":"2023-10-18T12:25:14.216327+0000","last_fullsized":"2023-10-18T12:25:14.216327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210067+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:25:14.210067+0000","last_peered":"2023-10-18T12:25:14.210067+0000","last_clean":"2023-10-18T12:25:14.210067+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:25:14.210067+0000","last_undegraded":"2023-10-18T12:25:14.210067+0000","last_fullsized":"2023-10-18T12:25:14.210067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214357+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:25:14.214357+0000","last_peered":"2023-10-18T12:25:14.214357+0000","last_clean":"2023-10-18T12:25:14.214357+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:25:14.214357+0000","last_undegraded":"2023-10-18T12:25:14.214357+0000","last_fullsized":"2023-10-18T12:25:14.214357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211315+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:25:14.211315+0000","last_peered":"2023-10-18T12:25:14.211315+0000","last_clean":"2023-10-18T12:25:14.211315+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:25:14.211315+0000","last_undegraded":"2023-10-18T12:25:14.211315+0000","last_fullsized":"2023-10-18T12:25:14.211315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210851+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:14.210851+0000","last_peered":"2023-10-18T12:25:14.210851+0000","last_clean":"2023-10-18T12:25:14.210851+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:14.210851+0000","last_undegraded":"2023-10-18T12:25:14.210851+0000","last_fullsized":"2023-10-18T12:25:14.210851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218308+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:25:14.218308+0000","last_peered":"2023-10-18T12:25:14.218308+0000","last_clean":"2023-10-18T12:25:14.218308+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:25:14.218308+0000","last_undegraded":"2023-10-18T12:25:14.218308+0000","last_fullsized":"2023-10-18T12:25:14.218308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211578+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:25:14.211578+0000","last_peered":"2023-10-18T12:25:14.211578+0000","last_clean":"2023-10-18T12:25:14.211578+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:25:14.211578+0000","last_undegraded":"2023-10-18T12:25:14.211578+0000","last_fullsized":"2023-10-18T12:25:14.211578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210274+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:14.210274+0000","last_peered":"2023-10-18T12:25:14.210274+0000","last_clean":"2023-10-18T12:25:14.210274+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:14.210274+0000","last_undegraded":"2023-10-18T12:25:14.210274+0000","last_fullsized":"2023-10-18T12:25:14.210274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210493+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:25:14.210493+0000","last_peered":"2023-10-18T12:25:14.210493+0000","last_clean":"2023-10-18T12:25:14.210493+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:25:14.210493+0000","last_undegraded":"2023-10-18T12:25:14.210493+0000","last_fullsized":"2023-10-18T12:25:14.210493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225441+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:25:15.225441+0000","last_peered":"2023-10-18T12:25:15.225441+0000","last_clean":"2023-10-18T12:25:15.225441+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:25:15.225441+0000","last_undegraded":"2023-10-18T12:25:15.225441+0000","last_fullsized":"2023-10-18T12:25:15.225441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211245+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:14.211245+0000","last_peered":"2023-10-18T12:25:14.211245+0000","last_clean":"2023-10-18T12:25:14.211245+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:14.211245+0000","last_undegraded":"2023-10-18T12:25:14.211245+0000","last_fullsized":"2023-10-18T12:25:14.211245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214225+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:25:14.214225+0000","last_peered":"2023-10-18T12:25:14.214225+0000","last_clean":"2023-10-18T12:25:14.214225+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:25:14.214225+0000","last_undegraded":"2023-10-18T12:25:14.214225+0000","last_fullsized":"2023-10-18T12:25:14.214225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212847+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:25:14.212847+0000","last_peered":"2023-10-18T12:25:14.212847+0000","last_clean":"2023-10-18T12:25:14.212847+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:25:14.212847+0000","last_undegraded":"2023-10-18T12:25:14.212847+0000","last_fullsized":"2023-10-18T12:25:14.212847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210805+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:25:14.210805+0000","last_peered":"2023-10-18T12:25:14.210805+0000","last_clean":"2023-10-18T12:25:14.210805+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:25:14.210805+0000","last_undegraded":"2023-10-18T12:25:14.210805+0000","last_fullsized":"2023-10-18T12:25:14.210805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218547+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:25:14.218547+0000","last_peered":"2023-10-18T12:25:14.218547+0000","last_clean":"2023-10-18T12:25:14.218547+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:25:14.218547+0000","last_undegraded":"2023-10-18T12:25:14.218547+0000","last_fullsized":"2023-10-18T12:25:14.218547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208516+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:25:14.208516+0000","last_peered":"2023-10-18T12:25:14.208516+0000","last_clean":"2023-10-18T12:25:14.208516+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:25:14.208516+0000","last_undegraded":"2023-10-18T12:25:14.208516+0000","last_fullsized":"2023-10-18T12:25:14.208516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214769+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:25:14.214769+0000","last_peered":"2023-10-18T12:25:14.214769+0000","last_clean":"2023-10-18T12:25:14.214769+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:25:14.214769+0000","last_undegraded":"2023-10-18T12:25:14.214769+0000","last_fullsized":"2023-10-18T12:25:14.214769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211339+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:25:14.211339+0000","last_peered":"2023-10-18T12:25:14.211339+0000","last_clean":"2023-10-18T12:25:14.211339+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:25:14.211339+0000","last_undegraded":"2023-10-18T12:25:14.211339+0000","last_fullsized":"2023-10-18T12:25:14.211339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211786+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:25:14.211786+0000","last_peered":"2023-10-18T12:25:14.211786+0000","last_clean":"2023-10-18T12:25:14.211786+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:25:14.211786+0000","last_undegraded":"2023-10-18T12:25:14.211786+0000","last_fullsized":"2023-10-18T12:25:14.211786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210728+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:25:14.210728+0000","last_peered":"2023-10-18T12:25:14.210728+0000","last_clean":"2023-10-18T12:25:14.210728+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:25:14.210728+0000","last_undegraded":"2023-10-18T12:25:14.210728+0000","last_fullsized":"2023-10-18T12:25:14.210728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211556+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:25:14.211556+0000","last_peered":"2023-10-18T12:25:14.211556+0000","last_clean":"2023-10-18T12:25:14.211556+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:25:14.211556+0000","last_undegraded":"2023-10-18T12:25:14.211556+0000","last_fullsized":"2023-10-18T12:25:14.211556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214616+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:25:14.214616+0000","last_peered":"2023-10-18T12:25:14.214616+0000","last_clean":"2023-10-18T12:25:14.214616+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:25:14.214616+0000","last_undegraded":"2023-10-18T12:25:14.214616+0000","last_fullsized":"2023-10-18T12:25:14.214616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208435+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:25:14.208435+0000","last_peered":"2023-10-18T12:25:14.208435+0000","last_clean":"2023-10-18T12:25:14.208435+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:25:14.208435+0000","last_undegraded":"2023-10-18T12:25:14.208435+0000","last_fullsized":"2023-10-18T12:25:14.208435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214670+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:25:14.214670+0000","last_peered":"2023-10-18T12:25:14.214670+0000","last_clean":"2023-10-18T12:25:14.214670+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:25:14.214670+0000","last_undegraded":"2023-10-18T12:25:14.214670+0000","last_fullsized":"2023-10-18T12:25:14.214670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61683,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":119,"num_read_kb":121,"num_write":72,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383707,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1,"5min":1,"15min":1},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85899999999999999}]},{"osd":9,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.754},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.90300000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.80000000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612639,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1160475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.63400000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.44}]},{"osd":2,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80500000000000005,"5min":0.999,"15min":0.999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.03,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78000000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.52700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.59699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.99399999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.752,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99299999999999999,"5min":1.099,"15min":1.099},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76900000000000002,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.623},{"interface":"front","average":{"1min":0.747,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.71499999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0269999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.82999999999999996,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.96299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66200000000000003}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.78400000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":517802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50600000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.56899999999999995,"15min":0.59099999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.60699999999999998,"5min":1.167,"15min":1.167},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0109999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89200000000000002,"5min":1.155,"15min":1.155},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84199999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91600000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.44},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.69799999999999995,"15min":0.72299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97799999999999998,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":0.871},"min":{"1min":0.628,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.98099999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.80200000000000005,"15min":0.82599999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.85199999999999998,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.76100000000000001,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88100000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.89200000000000002,"15min":0.91400000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.90900000000000003,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.90000000000000002,"15min":0.92700000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.89400000000000002,"5min":1.556,"15min":1.556},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.85299999999999998,"15min":0.87},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.995,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.65700000000000003}]},{"osd":8,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.95599999999999996,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.503},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.996,"5min":1.357,"15min":1.357},"last":0.55900000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.89600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.92300000000000004,"15min":0.93999999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1459999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65100000000000002}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.51000000000000001}]}]},{"osd":8,"up_from":60,"seq":257698037800,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":513706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59999999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.747},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.497}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.42999999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73199999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96799999999999997,"5min":1.044,"15min":1.044},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0409999999999999,"5min":1.26,"15min":1.26},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.74199999999999999,"15min":0.71999999999999997},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.77000000000000002}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68200000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1108170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.40699999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58399999999999996,"15min":0.56100000000000005},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.497,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89500000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.42899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.749,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.34100000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64400000000000002,"15min":0.61599999999999999},"min":{"1min":0.51400000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83599999999999997,"5min":0.999,"15min":0.999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97099999999999997,"5min":1.137,"15min":1.137},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.49299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99399999999999999,"5min":1.071,"15min":1.071},"last":0.54100000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.504}]},{"osd":9,"last update":"Wed Oct 18 12:24:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.61199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97199999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.70699999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79600000000000004,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98799999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.54300000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.622}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72499999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462960,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":509028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.59799999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79300000000000004,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.755},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59199999999999997,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98299999999999998,"5min":1.028,"15min":1.028},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0009999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86799999999999999,"5min":1.234,"15min":1.234},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.97599999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.03}]},{"osd":8,"last update":"Wed Oct 18 12:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.80300000000000005,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.004}]},{"osd":9,"last update":"Wed Oct 18 12:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.64200000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80800000000000005,"15min":0.81399999999999995},"min":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.89800000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":0.94999999999999996,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":7.9290000000000003,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.94699999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.077},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":1.046}]}]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":569913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.56000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":1.161,"15min":1.161},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.79100000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89200000000000002,"5min":1.454,"15min":1.454},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.61099999999999999,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96399999999999997,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71399999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.64800000000000002,"15min":0.59099999999999997},"min":{"1min":0.63200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":1.298,"15min":1.298},"last":0.749}]},{"osd":6,"last update":"Wed Oct 18 12:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.55400000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.496},{"interface":"front","average":{"1min":0.625,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.623}]},{"osd":7,"last update":"Wed Oct 18 12:25:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.31,"15min":0.31},"max":{"1min":0.91500000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67900000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.103,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71699999999999997,"15min":0.69299999999999995},"min":{"1min":0.58299999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.53300000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72699999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.871}]}]},{"osd":0,"up_from":9,"seq":38654705740,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":509118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.53400000000000003},"min":{"1min":0.40500000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.67900000000000005,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.379,"15min":0.379},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.67200000000000004,"15min":0.59599999999999997},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.67400000000000004}]},{"osd":3,"last update":"Wed Oct 18 12:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.48799999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60999999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99099999999999999,"5min":1.272,"15min":1.272},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.59799999999999998},"min":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.66200000000000003},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.65700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.80300000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70499999999999996,"15min":0.66900000000000004},"min":{"1min":0.56699999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.71199999999999997}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.877},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":0.83499999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1639999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.78800000000000003}]},{"osd":10,"last update":"Wed Oct 18 12:25:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.92400000000000004,"15min":0.89500000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.91000000000000003,"15min":0.89600000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.5710000000000002,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.82099999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.84799999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378693,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":569674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.29999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55500000000000005,"15min":0.51700000000000002},"min":{"1min":0.47299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.499},{"interface":"front","average":{"1min":0.629,"5min":0.56000000000000005,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.749}]},{"osd":4,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.34000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73799999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.44},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.42299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.623}]},{"osd":7,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58999999999999997}]},{"osd":9,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.747,"15min":0.746},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98199999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.083,"15min":1.083},"last":0.875}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79000000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":509132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69299999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.60499999999999998},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54000000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93200000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.66200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78600000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.876,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.67800000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.877,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.78400000000000003,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.60099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":0.84299999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.86099999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.80100000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68600000000000005,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.77600000000000002,"15min":0.73399999999999999},"min":{"1min":0.61299999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.63300000000000001}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97599999999999998,"5min":1.155,"15min":1.155},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.008,"5min":1.26,"15min":1.26},"last":0.85799999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.90600000000000003,"15min":0.89400000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":0.82199999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":4.8879999999999999,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.89900000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.378,"15min":0.36299999999999999},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.441}]},{"osd":2,"last update":"Wed Oct 18 12:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57599999999999996,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.04,"15min":1.04},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.54600000000000004},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82099999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":1.22,"15min":1.22},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.81399999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.93899999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.879,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.5},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58699999999999997,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56100000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80300000000000005,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.05,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.871,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54100000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86799999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1103343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.503},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84699999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.629}]},{"osd":7,"last update":"Wed Oct 18 12:25:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.48199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.76800000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed Oct 18 12:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86499999999999999,"15min":0.89500000000000002},"min":{"1min":0.55100000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":4.548,"15min":4.548},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0269999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.871},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:25:28.895 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-18T12:25:28.896 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T12:25:28.896 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-18T12:25:28.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph health --format=json 2023-10-18T12:25:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:28 smithi070 ceph-mon[98639]: pgmap v296: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 1 B/s, 0 objects/s recovering 2023-10-18T12:25:29.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:28 smithi152 ceph-mon[134767]: pgmap v296: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 1 B/s, 0 objects/s recovering 2023-10-18T12:25:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:28 smithi186 ceph-mon[135253]: pgmap v296: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 1 B/s, 0 objects/s recovering 2023-10-18T12:25:29.637 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:30.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:30 smithi070 ceph-mon[98639]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:25:30.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:30 smithi152 ceph-mon[134767]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:25:30.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:30 smithi186 ceph-mon[135253]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:25:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 -- 172.21.15.70:0/1964871364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac10101260 msgr2=0x7fac1010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 --2- 172.21.15.70:0/1964871364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac10101260 0x7fac1010d650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b0a0 tx=0x7fac0800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 -- 172.21.15.70:0/1964871364 shutdown_connections 2023-10-18T12:25:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 --2- 172.21.15.70:0/1964871364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac10101260 0x7fac1010d650 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 --2- 172.21.15.70:0/1964871364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 --2- 172.21.15.70:0/1964871364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac101008a0 0x7fac10100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.148+0000 7fac17930700 1 -- 172.21.15.70:0/1964871364 >> 172.21.15.70:0/1964871364 conn(0x7fac100779e0 msgr2=0x7fac10077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.149+0000 7fac17930700 1 -- 172.21.15.70:0/1964871364 shutdown_connections 2023-10-18T12:25:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.149+0000 7fac17930700 1 -- 172.21.15.70:0/1964871364 wait complete. 2023-10-18T12:25:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 Processor -- start 2023-10-18T12:25:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 -- start start 2023-10-18T12:25:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac101008a0 0x7fac10072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac10101260 0x7fac10073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac101a5090 con 0x7fac101008a0 2023-10-18T12:25:31.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac101a5200 con 0x7fac10101260 2023-10-18T12:25:31.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.150+0000 7fac17930700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac101a53c0 con 0x7fac101057c0 2023-10-18T12:25:31.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:31.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:55964/0 (socket says 172.21.15.70:55964) 2023-10-18T12:25:31.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 -- 172.21.15.70:0/3734882312 learned_addr learned my addr 172.21.15.70:0/3734882312 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac14ecb700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac10101260 0x7fac10073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac156cc700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac101008a0 0x7fac10072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 -- 172.21.15.70:0/3734882312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac10101260 msgr2=0x7fac10073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac10101260 0x7fac10073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 -- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac101008a0 msgr2=0x7fac10072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac101008a0 0x7fac10072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 -- 172.21.15.70:0/3734882312 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08009d00 con 0x7fac101057c0 2023-10-18T12:25:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.151+0000 7fac15ecd700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10073570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b070 tx=0x7fac080078c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.153+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800e040 con 0x7fac101057c0 2023-10-18T12:25:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.153+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0801d470 con 0x7fac101057c0 2023-10-18T12:25:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.153+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08005280 con 0x7fac101057c0 2023-10-18T12:25:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.153+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac101a5640 con 0x7fac101057c0 2023-10-18T12:25:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.154+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac101a5a60 con 0x7fac101057c0 2023-10-18T12:25:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.155+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac10108da0 con 0x7fac101057c0 2023-10-18T12:25:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.157+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fac0801d5e0 con 0x7fac101057c0 2023-10-18T12:25:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.158+0000 7fac067fc700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fabfc0a15e0 0x7fabfc0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.158+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fac080ce940 con 0x7fac101057c0 2023-10-18T12:25:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.158+0000 7fac156cc700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fabfc0a15e0 0x7fabfc0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.159+0000 7fac156cc700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fabfc0a15e0 0x7fabfc0a3aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fac10101930 tx=0x7fac0c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:31.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.160+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fac08097a80 con 0x7fac101057c0 2023-10-18T12:25:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.408+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fac10101f30 con 0x7fac101057c0 2023-10-18T12:25:31.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.409+0000 7fac067fc700 1 -- 172.21.15.70:0/3734882312 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fac0809b0a0 con 0x7fac101057c0 2023-10-18T12:25:31.411 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:31.412 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-18T12:25:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fabfc0a15e0 msgr2=0x7fabfc0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:31.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fabfc0a15e0 0x7fabfc0a3aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fac10101930 tx=0x7fac0c009260 comp rx=0 tx=0).stop 2023-10-18T12:25:31.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 msgr2=0x7fac10073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:31.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10073570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b070 tx=0x7fac080078c0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 shutdown_connections 2023-10-18T12:25:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac101057c0 0x7fac10073570 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac101008a0 0x7fac10072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fabfc0a15e0 0x7fabfc0a3aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 --2- 172.21.15.70:0/3734882312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac10101260 0x7fac10073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.411+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 >> 172.21.15.70:0/3734882312 conn(0x7fac100779e0 msgr2=0x7fac100fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.412+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 shutdown_connections 2023-10-18T12:25:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:31.412+0000 7fac17930700 1 -- 172.21.15.70:0/3734882312 wait complete. 2023-10-18T12:25:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:31 smithi152 ceph-mon[134767]: pgmap v297: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 1 B/s, 0 objects/s recovering 2023-10-18T12:25:31.435 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:31 smithi070 ceph-mon[98639]: pgmap v297: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 1 B/s, 0 objects/s recovering 2023-10-18T12:25:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:31 smithi186 ceph-mon[135253]: pgmap v297: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 1 B/s, 0 objects/s recovering 2023-10-18T12:25:31.832 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-18T12:25:31.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:25:32.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 -- 172.21.15.70:0/212989780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 msgr2=0x7ff27810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/212989780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff27810d570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b0a0 tx=0x7ff27000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 -- 172.21.15.70:0/212989780 shutdown_connections 2023-10-18T12:25:32.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/212989780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff27810d570 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/212989780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2781066b0 0x7ff278106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/212989780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781006a0 0x7ff278100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.033+0000 7ff27f9ff700 1 -- 172.21.15.70:0/212989780 >> 172.21.15.70:0/212989780 conn(0x7ff2780fc170 msgr2=0x7ff2780fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:32.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.034+0000 7ff27f9ff700 1 -- 172.21.15.70:0/212989780 shutdown_connections 2023-10-18T12:25:32.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.034+0000 7ff27f9ff700 1 -- 172.21.15.70:0/212989780 wait complete. 2023-10-18T12:25:32.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.034+0000 7ff27f9ff700 1 Processor -- start 2023-10-18T12:25:32.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 -- start start 2023-10-18T12:25:32.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2781006a0 0x7ff278072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff278073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781066b0 0x7ff278073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278074ac0 con 0x7ff278101060 2023-10-18T12:25:32.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2781a9450 con 0x7ff2781066b0 2023-10-18T12:25:32.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2781a9610 con 0x7ff2781006a0 2023-10-18T12:25:32.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27cf9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff278073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27cf9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff278073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58844/0 (socket says 172.21.15.70:58844) 2023-10-18T12:25:32.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27cf9a700 1 -- 172.21.15.70:0/1637831529 learned_addr learned my addr 172.21.15.70:0/1637831529 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:32.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.035+0000 7ff27d79b700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2781006a0 0x7ff278072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27df9c700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781066b0 0x7ff278073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27cf9a700 1 -- 172.21.15.70:0/1637831529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2781006a0 msgr2=0x7ff278072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27cf9a700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2781006a0 0x7ff278072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27cf9a700 1 -- 172.21.15.70:0/1637831529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781066b0 msgr2=0x7ff278073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27cf9a700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781066b0 0x7ff278073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27cf9a700 1 -- 172.21.15.70:0/1637831529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff270009d00 con 0x7ff278101060 2023-10-18T12:25:32.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27df9c700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781066b0 0x7ff278073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:32.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27cf9a700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff278073030 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff26800e9a0 tx=0x7ff26800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:32.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268009790 con 0x7ff278101060 2023-10-18T12:25:32.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26800cbd0 con 0x7ff278101060 2023-10-18T12:25:32.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2781a98f0 con 0x7ff278101060 2023-10-18T12:25:32.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268018680 con 0x7ff278101060 2023-10-18T12:25:32.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.036+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781a9e50 con 0x7ff278101060 2023-10-18T12:25:32.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.037+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278108ca0 con 0x7ff278101060 2023-10-18T12:25:32.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.039+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff26800cd40 con 0x7ff278101060 2023-10-18T12:25:32.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.041+0000 7ff26e7fc700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2640a15e0 0x7ff2640a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.041+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7ff268014070 con 0x7ff278101060 2023-10-18T12:25:32.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.041+0000 7ff27d79b700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2640a15e0 0x7ff2640a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.041+0000 7ff27d79b700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2640a15e0 0x7ff2640a3aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff274009be0 tx=0x7ff274009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:32.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.045+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff268052100 con 0x7ff278101060 2023-10-18T12:25:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.252+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff278101de0 con 0x7ff278101060 2023-10-18T12:25:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.254+0000 7ff26e7fc700 1 -- 172.21.15.70:0/1637831529 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7ff2680965c0 con 0x7ff278101060 2023-10-18T12:25:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:32.257 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:25:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2640a15e0 msgr2=0x7ff2640a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2640a15e0 0x7ff2640a3aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff274009be0 tx=0x7ff274009260 comp rx=0 tx=0).stop 2023-10-18T12:25:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 msgr2=0x7ff278073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff278073030 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff26800e9a0 tx=0x7ff26800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 shutdown_connections 2023-10-18T12:25:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2640a15e0 0x7ff2640a3aa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2781066b0 0x7ff278073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2781006a0 0x7ff278072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 --2- 172.21.15.70:0/1637831529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff278101060 0x7ff278073030 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 >> 172.21.15.70:0/1637831529 conn(0x7ff2780fc170 msgr2=0x7ff2780fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 shutdown_connections 2023-10-18T12:25:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.257+0000 7ff27f9ff700 1 -- 172.21.15.70:0/1637831529 wait complete. 2023-10-18T12:25:32.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:25:32.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:25:32.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:32 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3734882312' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-18T12:25:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:32 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3734882312' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-18T12:25:32.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 -- 172.21.15.70:0/2960574013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec072c20 msgr2=0x7f52ec073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 --2- 172.21.15.70:0/2960574013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec072c20 0x7f52ec073000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098f0 tx=0x7f52dc009c00 comp rx=0 tx=0).stop 2023-10-18T12:25:32.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 -- 172.21.15.70:0/2960574013 shutdown_connections 2023-10-18T12:25:32.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 --2- 172.21.15.70:0/2960574013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec10cf60 0x7f52ec10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 --2- 172.21.15.70:0/2960574013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec072c20 0x7f52ec073000 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 --2- 172.21.15.70:0/2960574013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec073540 0x7f52ec10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 -- 172.21.15.70:0/2960574013 >> 172.21.15.70:0/2960574013 conn(0x7f52ec0fc1b0 msgr2=0x7f52ec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:32.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 -- 172.21.15.70:0/2960574013 shutdown_connections 2023-10-18T12:25:32.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.477+0000 7f52f15ff700 1 -- 172.21.15.70:0/2960574013 wait complete. 2023-10-18T12:25:32.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.478+0000 7f52f15ff700 1 Processor -- start 2023-10-18T12:25:32.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.478+0000 7f52f15ff700 1 -- start start 2023-10-18T12:25:32.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52f15ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec072c20 0x7f52ec198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52f15ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec073540 0x7f52ec199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52f15ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 0x7f52ec19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52f15ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19dde0 con 0x7f52ec10cf60 2023-10-18T12:25:32.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52f15ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19df50 con 0x7f52ec073540 2023-10-18T12:25:32.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52f15ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19e110 con 0x7f52ec072c20 2023-10-18T12:25:32.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 0x7f52ec19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 0x7f52ec19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58868/0 (socket says 172.21.15.70:58868) 2023-10-18T12:25:32.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52eb7fe700 1 -- 172.21.15.70:0/992068417 learned_addr learned my addr 172.21.15.70:0/992068417 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:32.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52eaffd700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec072c20 0x7f52ec198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.479+0000 7f52ea7fc700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec073540 0x7f52ec199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52eb7fe700 1 -- 172.21.15.70:0/992068417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec072c20 msgr2=0x7f52ec198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52eb7fe700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec072c20 0x7f52ec198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52eb7fe700 1 -- 172.21.15.70:0/992068417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec073540 msgr2=0x7f52ec199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52eb7fe700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec073540 0x7f52ec199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52eb7fe700 1 -- 172.21.15.70:0/992068417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009d00 con 0x7f52ec10cf60 2023-10-18T12:25:32.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52eb7fe700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 0x7f52ec19d7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e3c0 tx=0x7f52e400e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:32.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e40090d0 con 0x7f52ec10cf60 2023-10-18T12:25:32.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc009520 con 0x7f52ec10cf60 2023-10-18T12:25:32.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e400f040 con 0x7f52ec10cf60 2023-10-18T12:25:32.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4007c20 con 0x7f52ec10cf60 2023-10-18T12:25:32.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.480+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1a5a50 con 0x7f52ec10cf60 2023-10-18T12:25:32.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.481+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec108170 con 0x7f52ec10cf60 2023-10-18T12:25:32.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.483+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f52e4004ad0 con 0x7f52ec10cf60 2023-10-18T12:25:32.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.485+0000 7f52d3fff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d40a1560 0x7f52d40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.485+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f52e40cd2f0 con 0x7f52ec10cf60 2023-10-18T12:25:32.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.485+0000 7f52eaffd700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d40a1560 0x7f52d40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.485+0000 7f52eaffd700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d40a1560 0x7f52d40a3a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098c0 tx=0x7f52dc019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:32.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.486+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f52e4096430 con 0x7f52ec10cf60 2023-10-18T12:25:32.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:32 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3734882312' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-18T12:25:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.687+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52ec19a1c0 con 0x7f52ec10cf60 2023-10-18T12:25:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.688+0000 7f52d3fff700 1 -- 172.21.15.70:0/992068417 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52e4099a50 con 0x7f52ec10cf60 2023-10-18T12:25:32.690 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:25:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d40a1560 msgr2=0x7f52d40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d40a1560 0x7f52d40a3a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098c0 tx=0x7f52dc019040 comp rx=0 tx=0).stop 2023-10-18T12:25:32.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 msgr2=0x7f52ec19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 0x7f52ec19d7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e3c0 tx=0x7f52e400e6d0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 shutdown_connections 2023-10-18T12:25:32.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ec10cf60 0x7f52ec19d7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec072c20 0x7f52ec198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d40a1560 0x7f52d40a3a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 --2- 172.21.15.70:0/992068417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec073540 0x7f52ec199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 >> 172.21.15.70:0/992068417 conn(0x7f52ec0fc1b0 msgr2=0x7f52ec0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 shutdown_connections 2023-10-18T12:25:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.690+0000 7f52f15ff700 1 -- 172.21.15.70:0/992068417 wait complete. 2023-10-18T12:25:32.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:25:32.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 -- 172.21.15.70:0/2430923412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318100710 msgr2=0x7f9318100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 --2- 172.21.15.70:0/2430923412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318100710 0x7f9318100b90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9308009990 tx=0x7f9308009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 -- 172.21.15.70:0/2430923412 shutdown_connections 2023-10-18T12:25:32.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 --2- 172.21.15.70:0/2430923412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181010d0 0x7f931810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 --2- 172.21.15.70:0/2430923412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318106720 0x7f9318106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 --2- 172.21.15.70:0/2430923412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318100710 0x7f9318100b90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 -- 172.21.15.70:0/2430923412 >> 172.21.15.70:0/2430923412 conn(0x7f93180fc200 msgr2=0x7f93180fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 -- 172.21.15.70:0/2430923412 shutdown_connections 2023-10-18T12:25:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.912+0000 7f931f9ff700 1 -- 172.21.15.70:0/2430923412 wait complete. 2023-10-18T12:25:32.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.913+0000 7f931f9ff700 1 Processor -- start 2023-10-18T12:25:32.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.913+0000 7f931f9ff700 1 -- start start 2023-10-18T12:25:32.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931f9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 0x7f9318198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931f9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181010d0 0x7f93181992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318106720 0x7f931819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931f9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931819dcd0 con 0x7f93181010d0 2023-10-18T12:25:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931f9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931819de40 con 0x7f9318100710 2023-10-18T12:25:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931819e000 con 0x7f9318106720 2023-10-18T12:25:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 0x7f9318198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931cf9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181010d0 0x7f93181992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 0x7f9318198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46256/0 (socket says 172.21.15.70:46256) 2023-10-18T12:25:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 -- 172.21.15.70:0/3005738976 learned_addr learned my addr 172.21.15.70:0/3005738976 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931df9c700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318106720 0x7f931819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 -- 172.21.15.70:0/3005738976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318106720 msgr2=0x7f931819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318106720 0x7f931819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 -- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181010d0 msgr2=0x7f93181992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:32.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181010d0 0x7f93181992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:32.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931d79b700 1 -- 172.21.15.70:0/3005738976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009520 con 0x7f9318100710 2023-10-18T12:25:32.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.914+0000 7f931df9c700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318106720 0x7f931819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:32.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.915+0000 7f931d79b700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 0x7f9318198db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f931400eb30 tx=0x7f931400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:32.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.915+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314009880 con 0x7f9318100710 2023-10-18T12:25:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.915+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93080095f0 con 0x7f9318100710 2023-10-18T12:25:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.915+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93140044e0 con 0x7f9318100710 2023-10-18T12:25:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.915+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314003e90 con 0x7f9318100710 2023-10-18T12:25:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.915+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931819e4d0 con 0x7f9318100710 2023-10-18T12:25:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.916+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9318101e50 con 0x7f9318100710 2023-10-18T12:25:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.918+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9314004000 con 0x7f9318100710 2023-10-18T12:25:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.919+0000 7f930e7fc700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93040a1560 0x7f93040a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.919+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9314014070 con 0x7f9318100710 2023-10-18T12:25:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.920+0000 7f931cf9a700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93040a1560 0x7f93040a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.920+0000 7f931cf9a700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93040a1560 0x7f93040a3a20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f931819a300 tx=0x7f930800b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:32.922+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9314096900 con 0x7f9318100710 2023-10-18T12:25:33.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.122+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f931819a0b0 con 0x7f9318100710 2023-10-18T12:25:33.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.123+0000 7f930e7fc700 1 -- 172.21.15.70:0/3005738976 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9314099f20 con 0x7f9318100710 2023-10-18T12:25:33.125 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:25:33.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.125+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93040a1560 msgr2=0x7f93040a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.125+0000 7f931f9ff700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93040a1560 0x7f93040a3a20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f931819a300 tx=0x7f930800b260 comp rx=0 tx=0).stop 2023-10-18T12:25:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.125+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 msgr2=0x7f9318198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.125+0000 7f931f9ff700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 0x7f9318198db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f931400eb30 tx=0x7f931400ee40 comp rx=0 tx=0).stop 2023-10-18T12:25:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 shutdown_connections 2023-10-18T12:25:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9318106720 0x7f931819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93040a1560 0x7f93040a3a20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9318100710 0x7f9318198db0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 --2- 172.21.15.70:0/3005738976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181010d0 0x7f93181992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 >> 172.21.15.70:0/3005738976 conn(0x7f93180fc200 msgr2=0x7f931810bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:33.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 shutdown_connections 2023-10-18T12:25:33.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.126+0000 7f931f9ff700 1 -- 172.21.15.70:0/3005738976 wait complete. 2023-10-18T12:25:33.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:25:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 -- 172.21.15.70:0/610599174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 msgr2=0x7f8890106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 --2- 172.21.15.70:0/610599174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 0x7f8890106b10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f888c009990 tx=0x7f888c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 -- 172.21.15.70:0/610599174 shutdown_connections 2023-10-18T12:25:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 --2- 172.21.15.70:0/610599174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88901010e0 0x7f889010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 --2- 172.21.15.70:0/610599174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 0x7f8890106b10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 --2- 172.21.15.70:0/610599174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f8890100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 -- 172.21.15.70:0/610599174 >> 172.21.15.70:0/610599174 conn(0x7f88900fc1b0 msgr2=0x7f88900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:33.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.346+0000 7f8897dff700 1 -- 172.21.15.70:0/610599174 shutdown_connections 2023-10-18T12:25:33.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.347+0000 7f8897dff700 1 -- 172.21.15.70:0/610599174 wait complete. 2023-10-18T12:25:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.347+0000 7f8897dff700 1 Processor -- start 2023-10-18T12:25:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 -- start start 2023-10-18T12:25:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f889019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88901010e0 0x7f889019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 0x7f8890198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890199200 con 0x7f8890106730 2023-10-18T12:25:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890199370 con 0x7f88901010e0 2023-10-18T12:25:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8897dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890199530 con 0x7f8890100720 2023-10-18T12:25:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8895b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f889019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8895b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f889019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:56056/0 (socket says 172.21.15.70:56056) 2023-10-18T12:25:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f8895b9b700 1 -- 172.21.15.70:0/3869428937 learned_addr learned my addr 172.21.15.70:0/3869428937 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f889639c700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 0x7f8890198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.348+0000 7f889539a700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88901010e0 0x7f889019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8895b9b700 1 -- 172.21.15.70:0/3869428937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88901010e0 msgr2=0x7f889019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8895b9b700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88901010e0 0x7f889019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8895b9b700 1 -- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 msgr2=0x7f8890198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8895b9b700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 0x7f8890198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8895b9b700 1 -- 172.21.15.70:0/3869428937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0095f0 con 0x7f8890100720 2023-10-18T12:25:33.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8895b9b700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f889019ea70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f888c005910 tx=0x7f888c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c01d070 con 0x7f8890100720 2023-10-18T12:25:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f888c022470 con 0x7f8890100720 2023-10-18T12:25:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88901997b0 con 0x7f8890100720 2023-10-18T12:25:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c017650 con 0x7f8890100720 2023-10-18T12:25:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.349+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88901a5a60 con 0x7f8890100720 2023-10-18T12:25:33.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.350+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8890101e60 con 0x7f8890100720 2023-10-18T12:25:33.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.352+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f888c0225e0 con 0x7f8890100720 2023-10-18T12:25:33.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.353+0000 7f8886ffd700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f887c0a1560 0x7f887c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.354+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f888c0ce550 con 0x7f8890100720 2023-10-18T12:25:33.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.354+0000 7f889539a700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f887c0a1560 0x7f887c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.354+0000 7f889539a700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f887c0a1560 0x7f887c0a3a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f889019a240 tx=0x7f8880009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:33.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.358+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f888c0977b0 con 0x7f8890100720 2023-10-18T12:25:33.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:33 smithi152 ceph-mon[134767]: pgmap v298: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:33 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1637831529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:33 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/992068417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:25:33.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:33 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3005738976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:25:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:33 smithi186 ceph-mon[135253]: pgmap v298: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:33 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1637831529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:33 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/992068417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:25:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:33 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3005738976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:25:33.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.557+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8890110690 con 0x7f8890100720 2023-10-18T12:25:33.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.558+0000 7f8886ffd700 1 -- 172.21.15.70:0/3869428937 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f888c0270a0 con 0x7f8890100720 2023-10-18T12:25:33.560 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:25:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f887c0a1560 msgr2=0x7f887c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f887c0a1560 0x7f887c0a3a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f889019a240 tx=0x7f8880009260 comp rx=0 tx=0).stop 2023-10-18T12:25:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 msgr2=0x7f889019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f889019ea70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f888c005910 tx=0x7f888c004730 comp rx=0 tx=0).stop 2023-10-18T12:25:33.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 shutdown_connections 2023-10-18T12:25:33.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890106730 0x7f8890198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8890100720 0x7f889019ea70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f887c0a1560 0x7f887c0a3a20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 --2- 172.21.15.70:0/3869428937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88901010e0 0x7f889019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 >> 172.21.15.70:0/3869428937 conn(0x7f88900fc1b0 msgr2=0x7f889010be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 shutdown_connections 2023-10-18T12:25:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.561+0000 7f8897dff700 1 -- 172.21.15.70:0/3869428937 wait complete. 2023-10-18T12:25:33.579 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:33 smithi070 ceph-mon[98639]: pgmap v298: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:33.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:33 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1637831529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:33.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:33 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/992068417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:25:33.580 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:33 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3005738976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:25:33.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:25:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 -- 172.21.15.70:0/2398124633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c100720 msgr2=0x7f161c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 --2- 172.21.15.70:0/2398124633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c100720 0x7f161c100ba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1610009990 tx=0x7f1610009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 -- 172.21.15.70:0/2398124633 shutdown_connections 2023-10-18T12:25:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 --2- 172.21.15.70:0/2398124633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f161c1010e0 0x7f161c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 --2- 172.21.15.70:0/2398124633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f161c106730 0x7f161c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 --2- 172.21.15.70:0/2398124633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c100720 0x7f161c100ba0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 -- 172.21.15.70:0/2398124633 >> 172.21.15.70:0/2398124633 conn(0x7f161c0fc1b0 msgr2=0x7f161c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 -- 172.21.15.70:0/2398124633 shutdown_connections 2023-10-18T12:25:33.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.783+0000 7f16227ff700 1 -- 172.21.15.70:0/2398124633 wait complete. 2023-10-18T12:25:33.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.784+0000 7f16227ff700 1 Processor -- start 2023-10-18T12:25:33.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.784+0000 7f16227ff700 1 -- start start 2023-10-18T12:25:33.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f16227ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f161c100720 0x7f161c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f16227ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 0x7f161c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f16227ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f161c106730 0x7f161c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f16227ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c19dcb0 con 0x7f161c1010e0 2023-10-18T12:25:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f16227ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c19de20 con 0x7f161c106730 2023-10-18T12:25:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f16227ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c19dfe0 con 0x7f161c100720 2023-10-18T12:25:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 0x7f161c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 0x7f161c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58920/0 (socket says 172.21.15.70:58920) 2023-10-18T12:25:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 -- 172.21.15.70:0/914307473 learned_addr learned my addr 172.21.15.70:0/914307473 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f1620d9c700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f161c106730 0x7f161c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161bfff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f161c100720 0x7f161c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 -- 172.21.15.70:0/914307473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f161c100720 msgr2=0x7f161c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f161c100720 0x7f161c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 -- 172.21.15.70:0/914307473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f161c106730 msgr2=0x7f161c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f161c106730 0x7f161c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.785+0000 7f161b7fe700 1 -- 172.21.15.70:0/914307473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100095f0 con 0x7f161c1010e0 2023-10-18T12:25:33.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.786+0000 7f161b7fe700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 0x7f161c1992d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f16100095c0 tx=0x7f161000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:33.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.786+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161001d070 con 0x7f161c1010e0 2023-10-18T12:25:33.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.786+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1610005030 con 0x7f161c1010e0 2023-10-18T12:25:33.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.786+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c19e260 con 0x7f161c1010e0 2023-10-18T12:25:33.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.786+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610004a90 con 0x7f161c1010e0 2023-10-18T12:25:33.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.786+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c1a5b00 con 0x7f161c1010e0 2023-10-18T12:25:33.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.787+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c101e60 con 0x7f161c1010e0 2023-10-18T12:25:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.790+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1610005410 con 0x7f161c1010e0 2023-10-18T12:25:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.791+0000 7f16197fa700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16040a1560 0x7f16040a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.792+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f16100ce420 con 0x7f161c1010e0 2023-10-18T12:25:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.792+0000 7f161bfff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16040a1560 0x7f16040a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.792+0000 7f161bfff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16040a1560 0x7f16040a3a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f160c005950 tx=0x7f160c00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.793+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1610097600 con 0x7f161c1010e0 2023-10-18T12:25:33.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.995+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f161c19a090 con 0x7f161c1010e0 2023-10-18T12:25:33.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.996+0000 7f16197fa700 1 -- 172.21.15.70:0/914307473 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f161009ac20 con 0x7f161c1010e0 2023-10-18T12:25:33.998 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:25:34.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16040a1560 msgr2=0x7f16040a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16040a1560 0x7f16040a3a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f160c005950 tx=0x7f160c00b410 comp rx=0 tx=0).stop 2023-10-18T12:25:34.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 msgr2=0x7f161c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 0x7f161c1992d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f16100095c0 tx=0x7f161000f690 comp rx=0 tx=0).stop 2023-10-18T12:25:34.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 shutdown_connections 2023-10-18T12:25:34.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16040a1560 0x7f16040a3a20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f161c106730 0x7f161c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f161c100720 0x7f161c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 --2- 172.21.15.70:0/914307473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161c1010e0 0x7f161c1992d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.998+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 >> 172.21.15.70:0/914307473 conn(0x7f161c0fc1b0 msgr2=0x7f161c10bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:34.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.999+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 shutdown_connections 2023-10-18T12:25:34.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:33.999+0000 7f16227ff700 1 -- 172.21.15.70:0/914307473 wait complete. 2023-10-18T12:25:34.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:25:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 -- 172.21.15.70:0/3086218208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 msgr2=0x7fddc810d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 --2- 172.21.15.70:0/3086218208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc810d320 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fddc000b0a0 tx=0x7fddc000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 -- 172.21.15.70:0/3086218208 shutdown_connections 2023-10-18T12:25:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 --2- 172.21.15.70:0/3086218208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc810d320 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 --2- 172.21.15.70:0/3086218208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8106510 0x7fddc81068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 --2- 172.21.15.70:0/3086218208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc8067830 0x7fddc8067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 -- 172.21.15.70:0/3086218208 >> 172.21.15.70:0/3086218208 conn(0x7fddc8074ee0 msgr2=0x7fddc80752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.220+0000 7fddce19d700 1 -- 172.21.15.70:0/3086218208 shutdown_connections 2023-10-18T12:25:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.221+0000 7fddce19d700 1 -- 172.21.15.70:0/3086218208 wait complete. 2023-10-18T12:25:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.221+0000 7fddce19d700 1 Processor -- start 2023-10-18T12:25:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 -- start start 2023-10-18T12:25:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8067830 0x7fddc819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc8106510 0x7fddc81967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8196e40 con 0x7fddc80681f0 2023-10-18T12:25:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8196fb0 con 0x7fddc8067830 2023-10-18T12:25:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddce19d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8197170 con 0x7fddc8106510 2023-10-18T12:25:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddcc99a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddcc99a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58936/0 (socket says 172.21.15.70:58936) 2023-10-18T12:25:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddcc99a700 1 -- 172.21.15.70:0/328999032 learned_addr learned my addr 172.21.15.70:0/328999032 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddcd19b700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8067830 0x7fddc819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.222+0000 7fddcd99c700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc8106510 0x7fddc81967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcc99a700 1 -- 172.21.15.70:0/328999032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc8106510 msgr2=0x7fddc81967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcc99a700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc8106510 0x7fddc81967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcc99a700 1 -- 172.21.15.70:0/328999032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8067830 msgr2=0x7fddc819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcc99a700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8067830 0x7fddc819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcc99a700 1 -- 172.21.15.70:0/328999032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009d00 con 0x7fddc80681f0 2023-10-18T12:25:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcd19b700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8067830 0x7fddc819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddcc99a700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc819cbe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e9a0 tx=0x7fddb800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8009790 con 0x7fddc80681f0 2023-10-18T12:25:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddb8010430 con 0x7fddc80681f0 2023-10-18T12:25:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8018680 con 0x7fddc80681f0 2023-10-18T12:25:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc8197450 con 0x7fddc80681f0 2023-10-18T12:25:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.223+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc81a3690 con 0x7fddc80681f0 2023-10-18T12:25:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.224+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc8108a70 con 0x7fddc80681f0 2023-10-18T12:25:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.226+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fddb80105a0 con 0x7fddc80681f0 2023-10-18T12:25:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.227+0000 7fddbe7fc700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb40a1560 0x7fddb40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.228+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fddb8014070 con 0x7fddc80681f0 2023-10-18T12:25:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.228+0000 7fddcd19b700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb40a1560 0x7fddb40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.228+0000 7fddcd19b700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb40a1560 0x7fddb40a3a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fddc4011490 tx=0x7fddc4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.231+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fddb8052100 con 0x7fddc80681f0 2023-10-18T12:25:34.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:34 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3869428937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:25:34.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:34 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/914307473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:25:34.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:34 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3869428937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:25:34.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:34 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/914307473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:25:34.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:34 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3869428937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:25:34.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:34 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/914307473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:25:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.475+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddc8101bc0 con 0x7fddc80681f0 2023-10-18T12:25:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.475+0000 7fddbe7fc700 1 -- 172.21.15.70:0/328999032 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fddb80965a0 con 0x7fddc80681f0 2023-10-18T12:25:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:25:34.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb40a1560 msgr2=0x7fddb40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb40a1560 0x7fddb40a3a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fddc4011490 tx=0x7fddc4009260 comp rx=0 tx=0).stop 2023-10-18T12:25:34.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 msgr2=0x7fddc819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc819cbe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e9a0 tx=0x7fddb800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 shutdown_connections 2023-10-18T12:25:34.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc8106510 0x7fddc81967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fddb40a1560 0x7fddb40a3a20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddc8067830 0x7fddc819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 --2- 172.21.15.70:0/328999032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddc80681f0 0x7fddc819cbe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 >> 172.21.15.70:0/328999032 conn(0x7fddc8074ee0 msgr2=0x7fddc80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:34.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 shutdown_connections 2023-10-18T12:25:34.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.478+0000 7fddce19d700 1 -- 172.21.15.70:0/328999032 wait complete. 2023-10-18T12:25:34.485 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:25:34.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:25:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 -- 172.21.15.70:0/2822582609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 msgr2=0x7fd35410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/2822582609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd35410d570 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00b0a0 tx=0x7fd34c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 -- 172.21.15.70:0/2822582609 shutdown_connections 2023-10-18T12:25:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/2822582609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd35410d570 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/2822582609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3541066b0 0x7fd354106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/2822582609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 0x7fd354100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 -- 172.21.15.70:0/2822582609 >> 172.21.15.70:0/2822582609 conn(0x7fd3540fc170 msgr2=0x7fd3540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 -- 172.21.15.70:0/2822582609 shutdown_connections 2023-10-18T12:25:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.697+0000 7fd35b3ff700 1 -- 172.21.15.70:0/2822582609 wait complete. 2023-10-18T12:25:34.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 Processor -- start 2023-10-18T12:25:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 -- start start 2023-10-18T12:25:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 0x7fd354072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd354073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3541066b0 0x7fd354073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541a4e70 con 0x7fd354101060 2023-10-18T12:25:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541a4fe0 con 0x7fd3541006a0 2023-10-18T12:25:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.698+0000 7fd35b3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541a51a0 con 0x7fd3541066b0 2023-10-18T12:25:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd354073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd354073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58962/0 (socket says 172.21.15.70:58962) 2023-10-18T12:25:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35919b700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 0x7fd354072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 -- 172.21.15.70:0/872533869 learned_addr learned my addr 172.21.15.70:0/872533869 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:34.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35999c700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3541066b0 0x7fd354073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 -- 172.21.15.70:0/872533869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3541066b0 msgr2=0x7fd354073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3541066b0 0x7fd354073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 -- 172.21.15.70:0/872533869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 msgr2=0x7fd354072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 0x7fd354072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 -- 172.21.15.70:0/872533869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009d00 con 0x7fd354101060 2023-10-18T12:25:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35919b700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 0x7fd354072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.699+0000 7fd35899a700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd354073030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd34400d980 tx=0x7fd34400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.700+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344004220 con 0x7fd354101060 2023-10-18T12:25:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.700+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3541a5480 con 0x7fd354101060 2023-10-18T12:25:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.700+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd344007640 con 0x7fd354101060 2023-10-18T12:25:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.700+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344005610 con 0x7fd354101060 2023-10-18T12:25:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.700+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3541a59e0 con 0x7fd354101060 2023-10-18T12:25:34.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.701+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd354108ca0 con 0x7fd354101060 2023-10-18T12:25:34.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.703+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd344004750 con 0x7fd354101060 2023-10-18T12:25:34.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.704+0000 7fd34a7fc700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3400a1560 0x7fd3400a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:34.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.704+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fd3440cd670 con 0x7fd354101060 2023-10-18T12:25:34.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.704+0000 7fd35919b700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3400a1560 0x7fd3400a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:34.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.705+0000 7fd35919b700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3400a1560 0x7fd3400a3a20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd350009fd0 tx=0x7fd350009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:34.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.706+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd344096720 con 0x7fd354101060 2023-10-18T12:25:34.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.912+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd354101de0 con 0x7fd354101060 2023-10-18T12:25:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.914+0000 7fd34a7fc700 1 -- 172.21.15.70:0/872533869 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7fd344016090 con 0x7fd354101060 2023-10-18T12:25:34.917 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:34.917 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:25:34.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3400a1560 msgr2=0x7fd3400a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3400a1560 0x7fd3400a3a20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd350009fd0 tx=0x7fd350009260 comp rx=0 tx=0).stop 2023-10-18T12:25:34.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 msgr2=0x7fd354073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:34.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd354073030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd34400d980 tx=0x7fd34400dc90 comp rx=0 tx=0).stop 2023-10-18T12:25:34.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 shutdown_connections 2023-10-18T12:25:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3541066b0 0x7fd354073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd3400a1560 0x7fd3400a3a20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3541006a0 0x7fd354072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 --2- 172.21.15.70:0/872533869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd354101060 0x7fd354073030 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 >> 172.21.15.70:0/872533869 conn(0x7fd3540fc170 msgr2=0x7fd3540fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 shutdown_connections 2023-10-18T12:25:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:34.917+0000 7fd35b3ff700 1 -- 172.21.15.70:0/872533869 wait complete. 2023-10-18T12:25:34.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:25:34.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:25:35.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 -- 172.21.15.70:0/2966804122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d41010e0 msgr2=0x7f53d410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2966804122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d41010e0 0x7f53d410d5f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00b0a0 tx=0x7f53cc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 -- 172.21.15.70:0/2966804122 shutdown_connections 2023-10-18T12:25:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2966804122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d41010e0 0x7f53d410d5f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2966804122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d4106730 0x7f53d4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2966804122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53d4100720 0x7f53d4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 -- 172.21.15.70:0/2966804122 >> 172.21.15.70:0/2966804122 conn(0x7f53d40fc1b0 msgr2=0x7f53d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 -- 172.21.15.70:0/2966804122 shutdown_connections 2023-10-18T12:25:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.137+0000 7f53d8fff700 1 -- 172.21.15.70:0/2966804122 wait complete. 2023-10-18T12:25:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.138+0000 7f53d8fff700 1 Processor -- start 2023-10-18T12:25:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.138+0000 7f53d8fff700 1 -- start start 2023-10-18T12:25:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.138+0000 7f53d8fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 0x7f53d419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.138+0000 7f53d8fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d41010e0 0x7f53d419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.138+0000 7f53d8fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53d4106730 0x7f53d4196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.138+0000 7f53d8fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4196f90 con 0x7f53d4100720 2023-10-18T12:25:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d8fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4197100 con 0x7f53d41010e0 2023-10-18T12:25:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d8fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41972c0 con 0x7f53d4106730 2023-10-18T12:25:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 0x7f53d419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d41010e0 0x7f53d419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 0x7f53d419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58972/0 (socket says 172.21.15.70:58972) 2023-10-18T12:25:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 -- 172.21.15.70:0/2537690284 learned_addr learned my addr 172.21.15.70:0/2537690284 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d2d9d700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53d4106730 0x7f53d4196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 -- 172.21.15.70:0/2537690284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53d4106730 msgr2=0x7f53d4196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53d4106730 0x7f53d4196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 -- 172.21.15.70:0/2537690284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d41010e0 msgr2=0x7f53d419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d41010e0 0x7f53d419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 -- 172.21.15.70:0/2537690284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc009d00 con 0x7f53d4100720 2023-10-18T12:25:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.139+0000 7f53d259c700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 0x7f53d419c800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e9a0 tx=0x7f53c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.140+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4009790 con 0x7f53d4100720 2023-10-18T12:25:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.140+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53c400cbd0 con 0x7f53d4100720 2023-10-18T12:25:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.140+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d41975a0 con 0x7f53d4100720 2023-10-18T12:25:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.140+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4018680 con 0x7f53d4100720 2023-10-18T12:25:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.140+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d41a37f0 con 0x7f53d4100720 2023-10-18T12:25:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.141+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d4108d20 con 0x7f53d4100720 2023-10-18T12:25:35.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.143+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f53c400cd40 con 0x7f53d4100720 2023-10-18T12:25:35.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.144+0000 7f53cb7fe700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53c00a15e0 0x7f53c00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.144+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f53c4014070 con 0x7f53d4100720 2023-10-18T12:25:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.144+0000 7f53cbfff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53c00a15e0 0x7f53c00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.145+0000 7f53cbfff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53c00a15e0 0x7f53c00a3aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f53d4197af0 tx=0x7f53bc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.148+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f53c4052100 con 0x7f53d4100720 2023-10-18T12:25:35.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.351+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53d4197df0 con 0x7f53d4100720 2023-10-18T12:25:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:35 smithi070 ceph-mon[98639]: pgmap v299: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/328999032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/872533869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:35.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.352+0000 7f53cb7fe700 1 -- 172.21.15.70:0/2537690284 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53c4096560 con 0x7f53d4100720 2023-10-18T12:25:35.356 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:25:35.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53c00a15e0 msgr2=0x7f53c00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53c00a15e0 0x7f53c00a3aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f53d4197af0 tx=0x7f53bc009260 comp rx=0 tx=0).stop 2023-10-18T12:25:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 msgr2=0x7f53d419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 0x7f53d419c800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e9a0 tx=0x7f53c400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 shutdown_connections 2023-10-18T12:25:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53d4106730 0x7f53d4196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53d4100720 0x7f53d419c800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f53c00a15e0 0x7f53c00a3aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 --2- 172.21.15.70:0/2537690284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d41010e0 0x7f53d419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.354+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 >> 172.21.15.70:0/2537690284 conn(0x7f53d40fc1b0 msgr2=0x7f53d40fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:35.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.355+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 shutdown_connections 2023-10-18T12:25:35.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.355+0000 7f53d8fff700 1 -- 172.21.15.70:0/2537690284 wait complete. 2023-10-18T12:25:35.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:25:35.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:35 smithi152 ceph-mon[134767]: pgmap v299: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:35.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/328999032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:35.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/872533869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:35 smithi186 ceph-mon[135253]: pgmap v299: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/328999032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/872533869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:35.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 -- 172.21.15.70:0/2667598860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac106730 msgr2=0x7f58ac106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2667598860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac106730 0x7f58ac106b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f589c0098f0 tx=0x7f589c009c00 comp rx=0 tx=0).stop 2023-10-18T12:25:35.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 -- 172.21.15.70:0/2667598860 shutdown_connections 2023-10-18T12:25:35.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2667598860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac1010e0 0x7f58ac10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2667598860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac106730 0x7f58ac106b10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2667598860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58ac100720 0x7f58ac100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 -- 172.21.15.70:0/2667598860 >> 172.21.15.70:0/2667598860 conn(0x7f58ac0fc1b0 msgr2=0x7f58ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:35.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.604+0000 7f58b09ff700 1 -- 172.21.15.70:0/2667598860 shutdown_connections 2023-10-18T12:25:35.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.605+0000 7f58b09ff700 1 -- 172.21.15.70:0/2667598860 wait complete. 2023-10-18T12:25:35.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.605+0000 7f58b09ff700 1 Processor -- start 2023-10-18T12:25:35.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 -- start start 2023-10-18T12:25:35.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58ac100720 0x7f58ac198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1010e0 0x7f58ac199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 0x7f58ac19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac19dcf0 con 0x7f58ac106730 2023-10-18T12:25:35.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac19de60 con 0x7f58ac1010e0 2023-10-18T12:25:35.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58b09ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac19e020 con 0x7f58ac100720 2023-10-18T12:25:35.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58aad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 0x7f58ac19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58aad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 0x7f58ac19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58992/0 (socket says 172.21.15.70:58992) 2023-10-18T12:25:35.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58aad9d700 1 -- 172.21.15.70:0/2212207664 learned_addr learned my addr 172.21.15.70:0/2212207664 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:35.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58aa59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58ac100720 0x7f58ac198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.606+0000 7f58a9d9b700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1010e0 0x7f58ac199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58aad9d700 1 -- 172.21.15.70:0/2212207664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58ac100720 msgr2=0x7f58ac198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58aad9d700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58ac100720 0x7f58ac198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58aad9d700 1 -- 172.21.15.70:0/2212207664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1010e0 msgr2=0x7f58ac199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58aad9d700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1010e0 0x7f58ac199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58aad9d700 1 -- 172.21.15.70:0/2212207664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00095f0 con 0x7f58ac106730 2023-10-18T12:25:35.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58aad9d700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 0x7f58ac19d6b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c370 tx=0x7f58a400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:35.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4017070 con 0x7f58ac106730 2023-10-18T12:25:35.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a400f040 con 0x7f58ac106730 2023-10-18T12:25:35.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a4009c40 con 0x7f58ac106730 2023-10-18T12:25:35.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a400e050 con 0x7f58ac106730 2023-10-18T12:25:35.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.607+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589c009520 con 0x7f58ac106730 2023-10-18T12:25:35.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.608+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac101e60 con 0x7f58ac106730 2023-10-18T12:25:35.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.610+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f58a400cb60 con 0x7f58ac106730 2023-10-18T12:25:35.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.612+0000 7f589b7fe700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58940a1560 0x7f58940a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:35.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.613+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f58a40ccc20 con 0x7f58ac106730 2023-10-18T12:25:35.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.613+0000 7f58aa59c700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58940a1560 0x7f58940a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:35.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.613+0000 7f58aa59c700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58940a1560 0x7f58940a3a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f589c009fd0 tx=0x7f589c019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:35.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.617+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58a4095d30 con 0x7f58ac106730 2023-10-18T12:25:35.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.819+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58ac0675a0 con 0x7f58ac106730 2023-10-18T12:25:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.819+0000 7f589b7fe700 1 -- 172.21.15.70:0/2212207664 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58a4099350 con 0x7f58ac106730 2023-10-18T12:25:35.823 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:25:35.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58940a1560 msgr2=0x7f58940a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58940a1560 0x7f58940a3a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f589c009fd0 tx=0x7f589c019040 comp rx=0 tx=0).stop 2023-10-18T12:25:35.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 msgr2=0x7f58ac19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:35.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 0x7f58ac19d6b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c370 tx=0x7f58a400c680 comp rx=0 tx=0).stop 2023-10-18T12:25:35.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 shutdown_connections 2023-10-18T12:25:35.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58ac106730 0x7f58ac19d6b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58ac100720 0x7f58ac198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58940a1560 0x7f58940a3a20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 --2- 172.21.15.70:0/2212207664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1010e0 0x7f58ac199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 >> 172.21.15.70:0/2212207664 conn(0x7f58ac0fc1b0 msgr2=0x7f58ac10d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 shutdown_connections 2023-10-18T12:25:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:35.822+0000 7f58b09ff700 1 -- 172.21.15.70:0/2212207664 wait complete. 2023-10-18T12:25:35.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:25:36.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 -- 172.21.15.70:0/2510990046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8106730 msgr2=0x7fd4f8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 --2- 172.21.15.70:0/2510990046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8106730 0x7fd4f8106b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009990 tx=0x7fd4ec009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 -- 172.21.15.70:0/2510990046 shutdown_connections 2023-10-18T12:25:36.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 --2- 172.21.15.70:0/2510990046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f81010e0 0x7fd4f810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 --2- 172.21.15.70:0/2510990046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8106730 0x7fd4f8106b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 --2- 172.21.15.70:0/2510990046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f8100720 0x7fd4f8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 -- 172.21.15.70:0/2510990046 >> 172.21.15.70:0/2510990046 conn(0x7fd4f80fc1b0 msgr2=0x7fd4f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:36.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 -- 172.21.15.70:0/2510990046 shutdown_connections 2023-10-18T12:25:36.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.043+0000 7fd4fedff700 1 -- 172.21.15.70:0/2510990046 wait complete. 2023-10-18T12:25:36.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 Processor -- start 2023-10-18T12:25:36.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 -- start start 2023-10-18T12:25:36.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 0x7fd4f819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f81010e0 0x7fd4f819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8106730 0x7fd4f8198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8199200 con 0x7fd4f8100720 2023-10-18T12:25:36.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8199370 con 0x7fd4f81010e0 2023-10-18T12:25:36.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.044+0000 7fd4fedff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8199530 con 0x7fd4f8106730 2023-10-18T12:25:36.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 0x7fd4f819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 0x7fd4f819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59018/0 (socket says 172.21.15.70:59018) 2023-10-18T12:25:36.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 -- 172.21.15.70:0/1533153035 learned_addr learned my addr 172.21.15.70:0/1533153035 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:36.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4f7fff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f81010e0 0x7fd4f819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fd39c700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8106730 0x7fd4f8198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 -- 172.21.15.70:0/1533153035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8106730 msgr2=0x7fd4f8198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8106730 0x7fd4f8198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 -- 172.21.15.70:0/1533153035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f81010e0 msgr2=0x7fd4f819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f81010e0 0x7fd4f819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 -- 172.21.15.70:0/1533153035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0095f0 con 0x7fd4f8100720 2023-10-18T12:25:36.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fd39c700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8106730 0x7fd4f8198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:36.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4fcb9b700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 0x7fd4f819ea70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:36.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.045+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec01d070 con 0x7fd4f8100720 2023-10-18T12:25:36.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.046+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4ec022470 con 0x7fd4f8100720 2023-10-18T12:25:36.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.046+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f81997b0 con 0x7fd4f8100720 2023-10-18T12:25:36.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.046+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec017650 con 0x7fd4f8100720 2023-10-18T12:25:36.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.046+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f81a5a60 con 0x7fd4f8100720 2023-10-18T12:25:36.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.047+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f8101e60 con 0x7fd4f8100720 2023-10-18T12:25:36.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.049+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd4ec0225e0 con 0x7fd4f8100720 2023-10-18T12:25:36.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.050+0000 7fd4f5ffb700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd4e80a15e0 0x7fd4e80a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.050+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fd4ec0ce3e0 con 0x7fd4f8100720 2023-10-18T12:25:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.050+0000 7fd4f7fff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd4e80a15e0 0x7fd4e80a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.051+0000 7fd4f7fff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd4e80a15e0 0x7fd4e80a3aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4f819a240 tx=0x7fd4e4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.054+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd4ec05d100 con 0x7fd4f8100720 2023-10-18T12:25:36.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.257+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4f8110690 con 0x7fd4f8100720 2023-10-18T12:25:36.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.258+0000 7fd4f5ffb700 1 -- 172.21.15.70:0/1533153035 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4ec0a1070 con 0x7fd4f8100720 2023-10-18T12:25:36.261 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:25:36.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd4e80a15e0 msgr2=0x7fd4e80a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd4e80a15e0 0x7fd4e80a3aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4f819a240 tx=0x7fd4e4009260 comp rx=0 tx=0).stop 2023-10-18T12:25:36.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 msgr2=0x7fd4f819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 0x7fd4f819ea70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec005740 comp rx=0 tx=0).stop 2023-10-18T12:25:36.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 shutdown_connections 2023-10-18T12:25:36.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8106730 0x7fd4f8198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f8100720 0x7fd4f819ea70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd4e80a15e0 0x7fd4e80a3aa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 --2- 172.21.15.70:0/1533153035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f81010e0 0x7fd4f819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.260+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 >> 172.21.15.70:0/1533153035 conn(0x7fd4f80fc1b0 msgr2=0x7fd4f810be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:36.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.261+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 shutdown_connections 2023-10-18T12:25:36.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.261+0000 7fd4fedff700 1 -- 172.21.15.70:0/1533153035 wait complete. 2023-10-18T12:25:36.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:25:36.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:36 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2537690284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:25:36.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:36 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2212207664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:25:36.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 -- 172.21.15.70:0/3071872158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 msgr2=0x7f4b3010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3071872158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3010d5f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800b0a0 tx=0x7f4b2800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 -- 172.21.15.70:0/3071872158 shutdown_connections 2023-10-18T12:25:36.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3071872158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3010d5f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3071872158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30106730 0x7f4b30106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3071872158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100720 0x7f4b30100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 -- 172.21.15.70:0/3071872158 >> 172.21.15.70:0/3071872158 conn(0x7f4b300fc1b0 msgr2=0x7f4b300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 -- 172.21.15.70:0/3071872158 shutdown_connections 2023-10-18T12:25:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.482+0000 7f4b367ff700 1 -- 172.21.15.70:0/3071872158 wait complete. 2023-10-18T12:25:36.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.483+0000 7f4b367ff700 1 Processor -- start 2023-10-18T12:25:36.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 -- start start 2023-10-18T12:25:36.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30100720 0x7f4b3019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30106730 0x7f4b30196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30197140 con 0x7f4b301010e0 2023-10-18T12:25:36.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301972b0 con 0x7f4b30106730 2023-10-18T12:25:36.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b367ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30197470 con 0x7f4b30100720 2023-10-18T12:25:36.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b2f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b2ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30100720 0x7f4b3019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b2f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59044/0 (socket says 172.21.15.70:59044) 2023-10-18T12:25:36.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b2ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30100720 0x7f4b3019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:56172/0 (socket says 172.21.15.70:56172) 2023-10-18T12:25:36.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.484+0000 7f4b2f7fe700 1 -- 172.21.15.70:0/3002836611 learned_addr learned my addr 172.21.15.70:0/3002836611 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:36.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b34d9c700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30106730 0x7f4b30196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2f7fe700 1 -- 172.21.15.70:0/3002836611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30100720 msgr2=0x7f4b3019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2f7fe700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30100720 0x7f4b3019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2f7fe700 1 -- 172.21.15.70:0/3002836611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30106730 msgr2=0x7f4b30196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2f7fe700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30106730 0x7f4b30196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2f7fe700 1 -- 172.21.15.70:0/3002836611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b28009d00 con 0x7f4b301010e0 2023-10-18T12:25:36.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b34d9c700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30106730 0x7f4b30196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:36.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2f7fe700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3019cdc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400e9a0 tx=0x7f4b2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:36.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24009790 con 0x7f4b301010e0 2023-10-18T12:25:36.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b2400cbd0 con 0x7f4b301010e0 2023-10-18T12:25:36.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b30197750 con 0x7f4b301010e0 2023-10-18T12:25:36.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24018680 con 0x7f4b301010e0 2023-10-18T12:25:36.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.485+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b301a3800 con 0x7f4b301010e0 2023-10-18T12:25:36.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.486+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b30108d40 con 0x7f4b301010e0 2023-10-18T12:25:36.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.488+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f4b2400cd40 con 0x7f4b301010e0 2023-10-18T12:25:36.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.490+0000 7f4b2d7fa700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b180a1560 0x7f4b180a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.490+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f4b24014070 con 0x7f4b301010e0 2023-10-18T12:25:36.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.490+0000 7f4b2ffff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b180a1560 0x7f4b180a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.490+0000 7f4b2ffff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b180a1560 0x7f4b180a3a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009be0 tx=0x7f4b20009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:36.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.493+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4b24052100 con 0x7f4b301010e0 2023-10-18T12:25:36.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:36 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2537690284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:25:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:36 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2212207664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:25:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:36 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2537690284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:25:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:36 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2212207664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:25:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.698+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4b30197e00 con 0x7f4b301010e0 2023-10-18T12:25:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.698+0000 7f4b2d7fa700 1 -- 172.21.15.70:0/3002836611 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4b24096690 con 0x7f4b301010e0 2023-10-18T12:25:36.702 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:25:36.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b180a1560 msgr2=0x7f4b180a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b180a1560 0x7f4b180a3a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009be0 tx=0x7f4b20009260 comp rx=0 tx=0).stop 2023-10-18T12:25:36.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 msgr2=0x7f4b3019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3019cdc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400e9a0 tx=0x7f4b2400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 shutdown_connections 2023-10-18T12:25:36.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b180a1560 0x7f4b180a3a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30106730 0x7f4b30196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b30100720 0x7f4b3019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 --2- 172.21.15.70:0/3002836611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b301010e0 0x7f4b3019cdc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 >> 172.21.15.70:0/3002836611 conn(0x7f4b300fc1b0 msgr2=0x7f4b300fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:36.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 shutdown_connections 2023-10-18T12:25:36.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.701+0000 7f4b367ff700 1 -- 172.21.15.70:0/3002836611 wait complete. 2023-10-18T12:25:36.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:25:36.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 -- 172.21.15.70:0/421616558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 msgr2=0x7fbcc4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 --2- 172.21.15.70:0/421616558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4106a90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009990 tx=0x7fbcb8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 -- 172.21.15.70:0/421616558 shutdown_connections 2023-10-18T12:25:36.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 --2- 172.21.15.70:0/421616558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 0x7fbcc410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 --2- 172.21.15.70:0/421616558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4106a90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 --2- 172.21.15.70:0/421616558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 -- 172.21.15.70:0/421616558 >> 172.21.15.70:0/421616558 conn(0x7fbcc40fc170 msgr2=0x7fbcc40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:36.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 -- 172.21.15.70:0/421616558 shutdown_connections 2023-10-18T12:25:36.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.917+0000 7fbccabff700 1 -- 172.21.15.70:0/421616558 wait complete. 2023-10-18T12:25:36.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.918+0000 7fbccabff700 1 Processor -- start 2023-10-18T12:25:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 -- start start 2023-10-18T12:25:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 0x7fbcc4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41a4e80 con 0x7fbcc41066b0 2023-10-18T12:25:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41a4ff0 con 0x7fbcc41006a0 2023-10-18T12:25:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbccabff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41a51b0 con 0x7fbcc4101060 2023-10-18T12:25:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbcc899b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.919+0000 7fbcc919c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc899b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46458/0 (socket says 172.21.15.70:46458) 2023-10-18T12:25:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc899b700 1 -- 172.21.15.70:0/2850745304 learned_addr learned my addr 172.21.15.70:0/2850745304 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc3fff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 0x7fbcc4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc919c700 1 -- 172.21.15.70:0/2850745304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 msgr2=0x7fbcc4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc919c700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 0x7fbcc4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc919c700 1 -- 172.21.15.70:0/2850745304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 msgr2=0x7fbcc4072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc919c700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc919c700 1 -- 172.21.15.70:0/2850745304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80095f0 con 0x7fbcc41066b0 2023-10-18T12:25:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc3fff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 0x7fbcc4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc899b700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc919c700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4073570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00efc0 tx=0x7fbcbc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc018070 con 0x7fbcc41066b0 2023-10-18T12:25:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc41a5430 con 0x7fbcc41066b0 2023-10-18T12:25:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcbc00f040 con 0x7fbcc41066b0 2023-10-18T12:25:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.920+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00cdf0 con 0x7fbcc41066b0 2023-10-18T12:25:36.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.921+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc41a5940 con 0x7fbcc41066b0 2023-10-18T12:25:36.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.922+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc4108ca0 con 0x7fbcc41066b0 2023-10-18T12:25:36.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.924+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fbcbc003a00 con 0x7fbcc41066b0 2023-10-18T12:25:36.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.925+0000 7fbcc1ffb700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcb40a1560 0x7fbcb40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:36.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.925+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fbcbc0ccf30 con 0x7fbcc41066b0 2023-10-18T12:25:36.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.925+0000 7fbcc899b700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcb40a1560 0x7fbcb40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:36.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.926+0000 7fbcc899b700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcb40a1560 0x7fbcb40a3a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb800b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:36.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:36.929+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbcbc0960d0 con 0x7fbcc41066b0 2023-10-18T12:25:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.177+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcc4073fd0 con 0x7fbcc41066b0 2023-10-18T12:25:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.178+0000 7fbcc1ffb700 1 -- 172.21.15.70:0/2850745304 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fbcbc0996f0 con 0x7fbcc41066b0 2023-10-18T12:25:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:37.181 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:25:37.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcb40a1560 msgr2=0x7fbcb40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcb40a1560 0x7fbcb40a3a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb800b260 comp rx=0 tx=0).stop 2023-10-18T12:25:37.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 msgr2=0x7fbcc4073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4073570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00efc0 tx=0x7fbcbc00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 shutdown_connections 2023-10-18T12:25:37.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc41066b0 0x7fbcc4073570 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcb40a1560 0x7fbcb40a3a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcc41006a0 0x7fbcc4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 --2- 172.21.15.70:0/2850745304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcc4101060 0x7fbcc4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.180+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 >> 172.21.15.70:0/2850745304 conn(0x7fbcc40fc170 msgr2=0x7fbcc40fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:37.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.181+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 shutdown_connections 2023-10-18T12:25:37.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.181+0000 7fbccabff700 1 -- 172.21.15.70:0/2850745304 wait complete. 2023-10-18T12:25:37.186 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:25:37.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:25:37.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 -- 172.21.15.70:0/1905841751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 msgr2=0x7f8800072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 --2- 172.21.15.70:0/1905841751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 0x7f8800072fa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009990 tx=0x7f87fc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 -- 172.21.15.70:0/1905841751 shutdown_connections 2023-10-18T12:25:37.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 --2- 172.21.15.70:0/1905841751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 --2- 172.21.15.70:0/1905841751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 0x7f8800072fa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 --2- 172.21.15.70:0/1905841751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000734e0 0x7f880010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 -- 172.21.15.70:0/1905841751 >> 172.21.15.70:0/1905841751 conn(0x7f88000fc1d0 msgr2=0x7f88000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:37.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 -- 172.21.15.70:0/1905841751 shutdown_connections 2023-10-18T12:25:37.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.399+0000 7f88075ff700 1 -- 172.21.15.70:0/1905841751 wait complete. 2023-10-18T12:25:37.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 Processor -- start 2023-10-18T12:25:37.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 -- start start 2023-10-18T12:25:37.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 0x7f8800198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000734e0 0x7f8800199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019dcf0 con 0x7f880010cf40 2023-10-18T12:25:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019de60 con 0x7f8800072bc0 2023-10-18T12:25:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f88075ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019e020 con 0x7f88000734e0 2023-10-18T12:25:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.400+0000 7f8805b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59084/0 (socket says 172.21.15.70:59084) 2023-10-18T12:25:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 -- 172.21.15.70:0/1923438050 learned_addr learned my addr 172.21.15.70:0/1923438050 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:37.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f880539b700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 0x7f8800198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8804b9a700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000734e0 0x7f8800199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 -- 172.21.15.70:0/1923438050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000734e0 msgr2=0x7f8800199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000734e0 0x7f8800199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 -- 172.21.15.70:0/1923438050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 msgr2=0x7f8800198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 0x7f8800198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 -- 172.21.15.70:0/1923438050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0095f0 con 0x7f880010cf40 2023-10-18T12:25:37.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f8805b9c700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880019d6b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f87f800ef90 tx=0x7f87f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:37.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8018070 con 0x7f880010cf40 2023-10-18T12:25:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880019e2a0 con 0x7f880010cf40 2023-10-18T12:25:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f800f040 con 0x7f880010cf40 2023-10-18T12:25:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.401+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f800cda0 con 0x7f880010cf40 2023-10-18T12:25:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.402+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88001a5b40 con 0x7f880010cf40 2023-10-18T12:25:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.403+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800108150 con 0x7f880010cf40 2023-10-18T12:25:37.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.404+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f87f8003a00 con 0x7f880010cf40 2023-10-18T12:25:37.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.406+0000 7f87f67fc700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87ec0a1560 0x7f87ec0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.406+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f87f80cce50 con 0x7f880010cf40 2023-10-18T12:25:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.406+0000 7f880539b700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87ec0a1560 0x7f87ec0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.407+0000 7f880539b700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87ec0a1560 0x7f87ec0a3a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87fc006010 tx=0x7f87fc00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.410+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87f8095f90 con 0x7f880010cf40 2023-10-18T12:25:37.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:37 smithi186 ceph-mon[135253]: pgmap v300: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:37 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1533153035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:25:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:37 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3002836611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:25:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:37 smithi070 ceph-mon[98639]: pgmap v300: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:37 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1533153035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:25:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:37 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3002836611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:25:37.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.613+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8800074270 con 0x7f880010cf40 2023-10-18T12:25:37.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.615+0000 7f87f67fc700 1 -- 172.21.15.70:0/1923438050 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f87f80995b0 con 0x7f880010cf40 2023-10-18T12:25:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:37.618 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:25:37.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87ec0a1560 msgr2=0x7f87ec0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87ec0a1560 0x7f87ec0a3a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87fc006010 tx=0x7f87fc00b260 comp rx=0 tx=0).stop 2023-10-18T12:25:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 msgr2=0x7f880019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880019d6b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f87f800ef90 tx=0x7f87f800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 shutdown_connections 2023-10-18T12:25:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f880010cf40 0x7f880019d6b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87ec0a1560 0x7f87ec0a3a20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8800072bc0 0x7f8800198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 --2- 172.21.15.70:0/1923438050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000734e0 0x7f8800199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.617+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 >> 172.21.15.70:0/1923438050 conn(0x7f88000fc1d0 msgr2=0x7f88000fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:37.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.618+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 shutdown_connections 2023-10-18T12:25:37.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.618+0000 7f88075ff700 1 -- 172.21.15.70:0/1923438050 wait complete. 2023-10-18T12:25:37.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:25:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:37 smithi152 ceph-mon[134767]: pgmap v300: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:37 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1533153035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:25:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:37 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3002836611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:25:37.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 -- 172.21.15.70:0/1408826099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc600101060 msgr2=0x7fc60010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 --2- 172.21.15.70:0/1408826099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc600101060 0x7fc60010d570 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800b0a0 tx=0x7fc5f800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 -- 172.21.15.70:0/1408826099 shutdown_connections 2023-10-18T12:25:37.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 --2- 172.21.15.70:0/1408826099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc600101060 0x7fc60010d570 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 --2- 172.21.15.70:0/1408826099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6001066b0 0x7fc600106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 --2- 172.21.15.70:0/1408826099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6001006a0 0x7fc600100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 -- 172.21.15.70:0/1408826099 >> 172.21.15.70:0/1408826099 conn(0x7fc6000fc170 msgr2=0x7fc6000fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 -- 172.21.15.70:0/1408826099 shutdown_connections 2023-10-18T12:25:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.836+0000 7fc6051ff700 1 -- 172.21.15.70:0/1408826099 wait complete. 2023-10-18T12:25:37.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.837+0000 7fc6051ff700 1 Processor -- start 2023-10-18T12:25:37.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 -- start start 2023-10-18T12:25:37.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 0x7fc60019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc600101060 0x7fc60019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6001066b0 0x7fc600198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600199170 con 0x7fc6001006a0 2023-10-18T12:25:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001992e0 con 0x7fc6001066b0 2023-10-18T12:25:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc6051ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001994a0 con 0x7fc600101060 2023-10-18T12:25:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc5fed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 0x7fc60019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc5fed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 0x7fc60019e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47678/0 (socket says 172.21.15.70:47678) 2023-10-18T12:25:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc5fed9d700 1 -- 172.21.15.70:0/3545965556 learned_addr learned my addr 172.21.15.70:0/3545965556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc5fe59c700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc600101060 0x7fc60019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.838+0000 7fc5ff59e700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6001066b0 0x7fc600198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5fed9d700 1 -- 172.21.15.70:0/3545965556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc600101060 msgr2=0x7fc60019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5fed9d700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc600101060 0x7fc60019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5fed9d700 1 -- 172.21.15.70:0/3545965556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6001066b0 msgr2=0x7fc600198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:37.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5fed9d700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6001066b0 0x7fc600198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:37.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5fed9d700 1 -- 172.21.15.70:0/3545965556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8009d00 con 0x7fc6001006a0 2023-10-18T12:25:37.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5fed9d700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 0x7fc60019e9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e9a0 tx=0x7fc5f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:37.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0009790 con 0x7fc6001006a0 2023-10-18T12:25:37.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5f0010430 con 0x7fc6001006a0 2023-10-18T12:25:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc600199780 con 0x7fc6001006a0 2023-10-18T12:25:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0018680 con 0x7fc6001006a0 2023-10-18T12:25:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.839+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6001a59d0 con 0x7fc6001006a0 2023-10-18T12:25:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.840+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600101de0 con 0x7fc6001006a0 2023-10-18T12:25:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.842+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fc5f00105a0 con 0x7fc6001006a0 2023-10-18T12:25:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.843+0000 7fc5e7fff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc5e80a1560 0x7fc5e80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.843+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fc5f0014070 con 0x7fc6001006a0 2023-10-18T12:25:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.844+0000 7fc5fe59c700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc5e80a1560 0x7fc5e80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.844+0000 7fc5fe59c700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc5e80a1560 0x7fc5e80a3a20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc600199cd0 tx=0x7fc5f4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:37.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:37.847+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5f0052100 con 0x7fc6001006a0 2023-10-18T12:25:38.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.101+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc600067530 con 0x7fc6001006a0 2023-10-18T12:25:38.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.102+0000 7fc5e7fff700 1 -- 172.21.15.70:0/3545965556 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fc5f0096540 con 0x7fc6001006a0 2023-10-18T12:25:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:38.105 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:25:38.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc5e80a1560 msgr2=0x7fc5e80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc5e80a1560 0x7fc5e80a3a20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc600199cd0 tx=0x7fc5f4009260 comp rx=0 tx=0).stop 2023-10-18T12:25:38.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 msgr2=0x7fc60019e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 0x7fc60019e9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e9a0 tx=0x7fc5f000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 shutdown_connections 2023-10-18T12:25:38.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc5e80a1560 0x7fc5e80a3a20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6001066b0 0x7fc600198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6001006a0 0x7fc60019e9e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 --2- 172.21.15.70:0/3545965556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc600101060 0x7fc60019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 >> 172.21.15.70:0/3545965556 conn(0x7fc6000fc170 msgr2=0x7fc60010b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 shutdown_connections 2023-10-18T12:25:38.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.104+0000 7fc6051ff700 1 -- 172.21.15.70:0/3545965556 wait complete. 2023-10-18T12:25:38.110 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:25:38.126 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-18T12:24:09.485042+0000', 'modified': '2023-10-18T12:24:34.631959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24719': {'gid': 24719, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.70:6839/3607185387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6838', 'nonce': 3607185387}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 3607185387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6839/1810263953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1810263953}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1810263953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:25:38.127 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:25:38.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T12:25:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 -- 172.21.15.70:0/280881709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01066b0 msgr2=0x7f0dc0106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/280881709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01066b0 0x7f0dc0106a90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009990 tx=0x7f0db0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 -- 172.21.15.70:0/280881709 shutdown_connections 2023-10-18T12:25:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/280881709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc0101060 0x7f0dc010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/280881709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01066b0 0x7f0dc0106a90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/280881709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01006a0 0x7f0dc0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 -- 172.21.15.70:0/280881709 >> 172.21.15.70:0/280881709 conn(0x7f0dc00fc170 msgr2=0x7f0dc00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 -- 172.21.15.70:0/280881709 shutdown_connections 2023-10-18T12:25:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.324+0000 7f0dc63ff700 1 -- 172.21.15.70:0/280881709 wait complete. 2023-10-18T12:25:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.325+0000 7f0dc63ff700 1 Processor -- start 2023-10-18T12:25:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 -- start start 2023-10-18T12:25:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 0x7f0dc0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc0101060 0x7f0dc0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01066b0 0x7f0dc0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc01a4e90 con 0x7f0dc01006a0 2023-10-18T12:25:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc01a5000 con 0x7f0dc01066b0 2023-10-18T12:25:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dc63ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc01a51c0 con 0x7f0dc0101060 2023-10-18T12:25:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 0x7f0dc0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 0x7f0dc0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47692/0 (socket says 172.21.15.70:47692) 2023-10-18T12:25:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0dbffff700 1 -- 172.21.15.70:0/1470591457 learned_addr learned my addr 172.21.15.70:0/1470591457 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.326+0000 7f0db7fff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc0101060 0x7f0dc0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dc499c700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01066b0 0x7f0dc0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbffff700 1 -- 172.21.15.70:0/1470591457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc0101060 msgr2=0x7f0dc0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbffff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc0101060 0x7f0dc0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbffff700 1 -- 172.21.15.70:0/1470591457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01066b0 msgr2=0x7f0dc0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbffff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01066b0 0x7f0dc0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbffff700 1 -- 172.21.15.70:0/1470591457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00095f0 con 0x7f0dc01006a0 2023-10-18T12:25:38.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dc499c700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01066b0 0x7f0dc0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:25:38.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbffff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 0x7f0dc0072af0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000c00 tx=0x7f0db0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:38.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbdffb700 1 -- 172.21.15.70:0/1470591457 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db001d070 con 0x7f0dc01006a0 2023-10-18T12:25:38.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbdffb700 1 -- 172.21.15.70:0/1470591457 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db0022470 con 0x7f0dc01006a0 2023-10-18T12:25:38.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc01a5440 con 0x7f0dc01006a0 2023-10-18T12:25:38.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dbdffb700 1 -- 172.21.15.70:0/1470591457 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0017650 con 0x7f0dc01006a0 2023-10-18T12:25:38.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.327+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc01a5920 con 0x7f0dc01006a0 2023-10-18T12:25:38.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.332+0000 7f0dbdffb700 1 -- 172.21.15.70:0/1470591457 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f0db00225e0 con 0x7f0dc01006a0 2023-10-18T12:25:38.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 Processor -- start 2023-10-18T12:25:38.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 -- start start 2023-10-18T12:25:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735680f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x56273567b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273567ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627355ee690 con 0x5627356814b0 2023-10-18T12:25:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627355d4c10 con 0x56273567b580 2023-10-18T12:25:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627355d2530 con 0x562735680b90 2023-10-18T12:25:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dbdffb700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0dac0a1560 0x7f0dac0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dbdffb700 1 -- 172.21.15.70:0/1470591457 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f0db00ce5d0 con 0x7f0dc01006a0 2023-10-18T12:25:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x56273567b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.333+0000 7f0dc499c700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0dac0a1560 0x7f0dac0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x56273567b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47702/0 (socket says 172.21.15.70:47702) 2023-10-18T12:25:38.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 -- 172.21.15.70:0/1012335089 learned_addr learned my addr 172.21.15.70:0/1012335089 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:38.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0db7fff700 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273567ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dc499c700 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735680f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 -- 172.21.15.70:0/1012335089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 msgr2=0x562735680f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735680f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 -- 172.21.15.70:0/1012335089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 msgr2=0x56273567ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273567ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 -- 172.21.15.70:0/1012335089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56273558b3a0 con 0x5627356814b0 2023-10-18T12:25:38.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dc499c700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0dac0a1560 0x7f0dac0a3a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0db8015e20 tx=0x7f0db8015420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:38.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0dbffff700 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x56273567b040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0db00098e0 tx=0x7f0db0005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0b384b0310097e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:38.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0db67fc700 1 -- 172.21.15.70:0/1012335089 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db009b200 con 0x5627356814b0 2023-10-18T12:25:38.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0db67fc700 1 -- 172.21.15.70:0/1012335089 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db009af70 con 0x5627356814b0 2023-10-18T12:25:38.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.334+0000 7f0db67fc700 1 -- 172.21.15.70:0/1012335089 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00a3730 con 0x5627356814b0 2023-10-18T12:25:38.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/1012335089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 msgr2=0x56273567b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x56273567b040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0db00098e0 tx=0x7f0db0005860 comp rx=0 tx=0).stop 2023-10-18T12:25:38.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/1012335089 shutdown_connections 2023-10-18T12:25:38.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273567ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735680f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/1012335089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x56273567b040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/1012335089 >> 172.21.15.70:0/1012335089 conn(0x5627355ef6d0 msgr2=0x562735675b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/1012335089 shutdown_connections 2023-10-18T12:25:38.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.335+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/1012335089 wait complete. 2023-10-18T12:25:38.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 -- start start 2023-10-18T12:25:38.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 Processor -- start 2023-10-18T12:25:38.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 0x562735687320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x5627356846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735684c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627355cde20 con 0x56273567b580 2023-10-18T12:25:38.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273556a6c0 con 0x5627356814b0 2023-10-18T12:25:38.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.336+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273556a360 con 0x562735680b90 2023-10-18T12:25:38.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dd9c53b80 10 client.0 _ll_register_callbacks cb 0x5627354b4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:25:38.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 0x562735687320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 0x562735687320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47714/0 (socket says 172.21.15.70:47714) 2023-10-18T12:25:38.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 -- 172.21.15.70:0/2872425419 learned_addr learned my addr 172.21.15.70:0/2872425419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:38.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0db7fff700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x5627356846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dc499c700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735684c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 -- 172.21.15.70:0/2872425419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 msgr2=0x5627356846b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x5627356846b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 -- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 msgr2=0x562735684c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735684c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 -- 172.21.15.70:0/2872425419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc01a5920 con 0x56273567b580 2023-10-18T12:25:38.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0dbffff700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 0x562735687320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0db0097930 tx=0x7f0db00cc010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7d8dcc85dc6b94 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:38.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0d9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:25:38.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.337+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/2872425419 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00b7070 con 0x56273567b580 2023-10-18T12:25:38.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/2872425419 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f0db00b2070 con 0x56273567b580 2023-10-18T12:25:38.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627355d2ea0 con 0x56273567b580 2023-10-18T12:25:38.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/2872425419 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0db009b200 con 0x56273567b580 2023-10-18T12:25:38.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 10 client.15279 fetch_fsmap learned FSMap version 20 2023-10-18T12:25:38.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5627353f8d90 con 0x56273567b580 2023-10-18T12:25:38.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/2872425419 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db00c9080 con 0x56273567b580 2023-10-18T12:25:38.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/2872425419 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0db009f170 con 0x56273567b580 2023-10-18T12:25:38.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:25:38.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T12:25:38.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 20 client.15279 populate_metadata read hostname 'smithi070' 2023-10-18T12:25:38.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562735694700 0x562735693360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 4 client.15279 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:25:38.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.338+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56273564a860 con 0x562735694700 2023-10-18T12:25:38.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.339+0000 7f0db7fff700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562735694700 0x562735693360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.339+0000 7f0db7fff700 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562735694700 0x562735693360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8964c0dc45d2ac11 server_cookie=a819c5dac7b4ba47 in_seq=0 out_seq=0 2023-10-18T12:25:38.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.339+0000 7f0d9f7fe700 10 client.15279 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:25:38.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/2872425419 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56273564a860 con 0x562735694700 2023-10-18T12:25:38.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0d9f7fe700 10 client.15279 handle_command_reply: tid=0 2023-10-18T12:25:38.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 client.15279 shutdown 2023-10-18T12:25:38.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 msgr2=0x562735687320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 0x562735687320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0db0097930 tx=0x7f0db00cc010 comp rx=0 tx=0).stop 2023-10-18T12:25:38.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 shutdown_connections 2023-10-18T12:25:38.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562735694700 0x562735693360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735684c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.340+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56273567b580 0x562735687320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.341+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2872425419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x5627356846b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.341+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 >> 172.21.15.70:0/2872425419 conn(0x5627355ef6d0 msgr2=0x56273567a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.341+0000 7f0d9f7fe700 0 client.15279 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:25:38.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.341+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 shutdown_connections 2023-10-18T12:25:38.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.341+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2872425419 wait complete. 2023-10-18T12:25:38.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.341+0000 7f0dd9c53b80 20 client.15279 trim_cache size 0 max 16384 2023-10-18T12:25:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 Processor -- start 2023-10-18T12:25:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 -- start start 2023-10-18T12:25:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56273567b580 0x562735690a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 0x56273569d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735686cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562735670aa0 con 0x562735680b90 2023-10-18T12:25:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273544e8d0 con 0x5627356814b0 2023-10-18T12:25:38.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273558c590 con 0x56273567b580 2023-10-18T12:25:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0db7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 0x56273569d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0db7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 0x56273569d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47724/0 (socket says 172.21.15.70:47724) 2023-10-18T12:25:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0db7fff700 1 -- 172.21.15.70:0/2490771249 learned_addr learned my addr 172.21.15.70:0/2490771249 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dbffff700 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56273567b580 0x562735690a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0dc499c700 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735686cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0db7fff700 1 -- 172.21.15.70:0/2490771249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56273567b580 msgr2=0x562735690a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.359+0000 7f0db7fff700 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56273567b580 0x562735690a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db7fff700 1 -- 172.21.15.70:0/2490771249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 msgr2=0x562735686cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db7fff700 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735686cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db7fff700 1 -- 172.21.15.70:0/2490771249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627353e1c10 con 0x562735680b90 2023-10-18T12:25:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0dc499c700 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735686cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:25:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db7fff700 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 0x56273569d070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0da8004790 tx=0x7f0da8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8476756ce882e5d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db6ffd700 1 -- 172.21.15.70:0/2490771249 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8017070 con 0x562735680b90 2023-10-18T12:25:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db6ffd700 1 -- 172.21.15.70:0/2490771249 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0da8018040 con 0x562735680b90 2023-10-18T12:25:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.360+0000 7f0db6ffd700 1 -- 172.21.15.70:0/2490771249 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8010690 con 0x562735680b90 2023-10-18T12:25:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2490771249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 msgr2=0x56273569d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 0x56273569d070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0da8004790 tx=0x7f0da8009260 comp rx=0 tx=0).stop 2023-10-18T12:25:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2490771249 shutdown_connections 2023-10-18T12:25:38.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627356814b0 0x562735686cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56273567b580 0x562735690a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/2490771249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562735680b90 0x56273569d070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2490771249 >> 172.21.15.70:0/2490771249 conn(0x5627355ef6d0 msgr2=0x56273567a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2490771249 shutdown_connections 2023-10-18T12:25:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/2490771249 wait complete. 2023-10-18T12:25:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 -- start start 2023-10-18T12:25:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.361+0000 7f0dd9c53b80 1 Processor -- start 2023-10-18T12:25:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273569d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735682f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x5627356834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562735683a90 con 0x5627356814b0 2023-10-18T12:25:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562735683c00 con 0x56273567b580 2023-10-18T12:25:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562735683dc0 con 0x562735680b90 2023-10-18T12:25:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dd9c53b80 10 client.0 _ll_register_callbacks cb 0x5627354b4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:25:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x5627356834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x5627356834c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47726/0 (socket says 172.21.15.70:47726) 2023-10-18T12:25:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 -- 172.21.15.70:0/3824003732 learned_addr learned my addr 172.21.15.70:0/3824003732 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dbffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273569d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0db7fff700 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735682f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 -- 172.21.15.70:0/3824003732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 msgr2=0x562735682f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735682f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 -- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 msgr2=0x56273569d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:25:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273569d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.362+0000 7f0dc499c700 1 -- 172.21.15.70:0/3824003732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da8004040 con 0x5627356814b0 2023-10-18T12:25:38.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0dc499c700 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x5627356834c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009fd0 tx=0x7f0db800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212fea059c9b1b6e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:38.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0d9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:25:38.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/3824003732 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800e040 con 0x5627356814b0 2023-10-18T12:25:38.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0db009af70 con 0x5627356814b0 2023-10-18T12:25:38.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/3824003732 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f0db8020040 con 0x5627356814b0 2023-10-18T12:25:38.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/3824003732 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0db8007b70 con 0x5627356814b0 2023-10-18T12:25:38.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0dd9c53b80 10 client.15285 fetch_fsmap learned FSMap version 20 2023-10-18T12:25:38.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56273541f200 con 0x5627356814b0 2023-10-18T12:25:38.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.363+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/3824003732 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db009af70 con 0x5627356814b0 2023-10-18T12:25:38.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/3824003732 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0db801a440 con 0x5627356814b0 2023-10-18T12:25:38.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dd9c53b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:25:38.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dd9c53b80 10 client.15285 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T12:25:38.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dd9c53b80 20 client.15285 populate_metadata read hostname 'smithi070' 2023-10-18T12:25:38.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5627356ed250 0x5627356e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:38.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dd9c53b80 4 client.15285 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:25:38.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5627355c6e00 con 0x5627356ed250 2023-10-18T12:25:38.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.364+0000 7f0dbffff700 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5627356ed250 0x5627356e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:38.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0d9f7fe700 10 client.15285 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:25:38.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dbffff700 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5627356ed250 0x5627356e96b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa0e420f94143d2 server_cookie=9cae04745697536f in_seq=0 out_seq=0 2023-10-18T12:25:38.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0d9f7fe700 1 -- 172.21.15.70:0/3824003732 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5627355c6e00 con 0x5627356ed250 2023-10-18T12:25:38.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0d9f7fe700 10 client.15285 handle_command_reply: tid=0 2023-10-18T12:25:38.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 client.15285 shutdown 2023-10-18T12:25:38.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 msgr2=0x5627356834c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x5627356834c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009fd0 tx=0x7f0db800b040 comp rx=0 tx=0).stop 2023-10-18T12:25:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 shutdown_connections 2023-10-18T12:25:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5627356ed250 0x5627356e96b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5627356814b0 0x5627356834c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56273567b580 0x56273569d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0d9f7fe700 0 client.15285 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:25:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 --2- 172.21.15.70:0/3824003732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562735680b90 0x562735682f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 >> 172.21.15.70:0/3824003732 conn(0x5627355ef6d0 msgr2=0x56273567e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.365+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 shutdown_connections 2023-10-18T12:25:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dd9c53b80 1 -- 172.21.15.70:0/3824003732 wait complete. 2023-10-18T12:25:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dd9c53b80 20 client.15285 trim_cache size 0 max 16384 2023-10-18T12:25:38.385 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:25:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "f94fde2a-6daf-11ee-8db6-212e2dc638e7", 2023-10-18T12:25:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-18T12:25:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 24743, 2023-10-18T12:25:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-18T12:25:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-18T12:25:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-18T12:25:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 88.246543626999994, 2023-10-18T12:25:38.387 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 20, 2023-10-18T12:25:38.387 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 91, 2023-10-18T12:25:38.387 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 89, 2023-10-18T12:25:38.389 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 95.222848321000001 2023-10-18T12:25:38.389 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:25:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0dac0a1560 msgr2=0x7f0dac0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0dac0a1560 0x7f0dac0a3a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0db8015e20 tx=0x7f0db8015420 comp rx=0 tx=0).stop 2023-10-18T12:25:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 msgr2=0x7f0dc0072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 0x7f0dc0072af0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000c00 tx=0x7f0db0005740 comp rx=0 tx=0).stop 2023-10-18T12:25:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 shutdown_connections 2023-10-18T12:25:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0dac0a1560 0x7f0dac0a3a20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dc01066b0 0x7f0dc0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dc01006a0 0x7f0dc0072af0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 --2- 172.21.15.70:0/1470591457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc0101060 0x7f0dc0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 >> 172.21.15.70:0/1470591457 conn(0x7f0dc00fc170 msgr2=0x7f0dc00fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 shutdown_connections 2023-10-18T12:25:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:38.366+0000 7f0dc63ff700 1 -- 172.21.15.70:0/1470591457 wait complete. 2023-10-18T12:25:38.393 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2850745304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.394 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1923438050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.394 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:38 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3545965556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.395 INFO:tasks.cephadm:Setup complete, yielding 2023-10-18T12:25:38.395 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-18T12:25:38.406 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:25:38.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-18T12:25:38.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2850745304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1923438050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:38 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3545965556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2850745304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1923438050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:25:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:38 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3545965556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:25:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:39.441 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:39 smithi070 ceph-mon[98639]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:39.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:39 smithi186 ceph-mon[135253]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:39 smithi152 ceph-mon[134767]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:40.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 -- 172.21.15.70:0/72297352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1057c0 msgr2=0x7f907c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 --2- 172.21.15.70:0/72297352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1057c0 0x7f907c105ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f906c009990 tx=0x7f906c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 -- 172.21.15.70:0/72297352 shutdown_connections 2023-10-18T12:25:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 --2- 172.21.15.70:0/72297352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f907c101260 0x7f907c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 --2- 172.21.15.70:0/72297352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1057c0 0x7f907c105ba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 --2- 172.21.15.70:0/72297352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f907c1008a0 0x7f907c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 -- 172.21.15.70:0/72297352 >> 172.21.15.70:0/72297352 conn(0x7f907c0779e0 msgr2=0x7f907c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:40.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 -- 172.21.15.70:0/72297352 shutdown_connections 2023-10-18T12:25:40.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.639+0000 7f9080fc1700 1 -- 172.21.15.70:0/72297352 wait complete. 2023-10-18T12:25:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.640+0000 7f9080fc1700 1 Processor -- start 2023-10-18T12:25:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.640+0000 7f9080fc1700 1 -- start start 2023-10-18T12:25:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9080fc1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 0x7f907c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9080fc1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f907c101260 0x7f907c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9080fc1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f907c1057c0 0x7f907c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9080fc1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c19dd50 con 0x7f907c1008a0 2023-10-18T12:25:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9080fc1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c19dec0 con 0x7f907c1057c0 2023-10-18T12:25:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9080fc1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c19e080 con 0x7f907c101260 2023-10-18T12:25:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 0x7f907c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 0x7f907c198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47750/0 (socket says 172.21.15.70:47750) 2023-10-18T12:25:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 -- 172.21.15.70:0/421341467 learned_addr learned my addr 172.21.15.70:0/421341467 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f9079d9b700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f907c101260 0x7f907c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907ad9d700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f907c1057c0 0x7f907c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 -- 172.21.15.70:0/421341467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f907c101260 msgr2=0x7f907c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f907c101260 0x7f907c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 -- 172.21.15.70:0/421341467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f907c1057c0 msgr2=0x7f907c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f907c1057c0 0x7f907c19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.641+0000 7f907a59c700 1 -- 172.21.15.70:0/421341467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0095f0 con 0x7f907c1008a0 2023-10-18T12:25:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.642+0000 7f907a59c700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 0x7f907c198e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f906c000c00 tx=0x7f906c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.642+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c01d070 con 0x7f907c1008a0 2023-10-18T12:25:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.642+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906c022470 con 0x7f907c1008a0 2023-10-18T12:25:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.642+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c19e300 con 0x7f907c1008a0 2023-10-18T12:25:40.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.642+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c017650 con 0x7f907c1008a0 2023-10-18T12:25:40.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.642+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c1a5bc0 con 0x7f907c1008a0 2023-10-18T12:25:40.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.643+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c101f30 con 0x7f907c1008a0 2023-10-18T12:25:40.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.645+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f906c00fad0 con 0x7f907c1008a0 2023-10-18T12:25:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.646+0000 7f906b7fe700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90640a1560 0x7f90640a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.646+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f906c0ce3e0 con 0x7f907c1008a0 2023-10-18T12:25:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.647+0000 7f9079d9b700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90640a1560 0x7f90640a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.647+0000 7f9079d9b700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90640a1560 0x7f90640a3a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f907c19a3b0 tx=0x7f9070009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:40.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.651+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f906c097620 con 0x7f907c1008a0 2023-10-18T12:25:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.868+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f907c19a160 con 0x7f90640a1560 2023-10-18T12:25:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.869+0000 7f906b7fe700 1 -- 172.21.15.70:0/421341467 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f907c19a160 con 0x7f90640a1560 2023-10-18T12:25:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:Backend: cephadm 2023-10-18T12:25:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:Available: Yes 2023-10-18T12:25:40.873 INFO:teuthology.orchestra.run.smithi070.stdout:Paused: No 2023-10-18T12:25:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90640a1560 msgr2=0x7f90640a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90640a1560 0x7f90640a3a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f907c19a3b0 tx=0x7f9070009260 comp rx=0 tx=0).stop 2023-10-18T12:25:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 msgr2=0x7f907c198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 0x7f907c198e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f906c000c00 tx=0x7f906c005740 comp rx=0 tx=0).stop 2023-10-18T12:25:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 shutdown_connections 2023-10-18T12:25:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f90640a1560 0x7f90640a3a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f907c1057c0 0x7f907c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c1008a0 0x7f907c198e50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 --2- 172.21.15.70:0/421341467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f907c101260 0x7f907c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 >> 172.21.15.70:0/421341467 conn(0x7f907c0779e0 msgr2=0x7f907c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.871+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 shutdown_connections 2023-10-18T12:25:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:40.872+0000 7f9080fc1700 1 -- 172.21.15.70:0/421341467 wait complete. 2023-10-18T12:25:41.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-18T12:25:41.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:41 smithi186 ceph-mon[135253]: pgmap v302: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:41 smithi070 ceph-mon[98639]: pgmap v302: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:41 smithi152 ceph-mon[134767]: pgmap v302: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:42.073 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:42.450 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:42 smithi070 ceph-mon[98639]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:42.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:42 smithi186 ceph-mon[135253]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:42.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:42 smithi152 ceph-mon[134767]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:43.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:43 smithi186 ceph-mon[135253]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:43.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 -- 172.21.15.70:0/1009959611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3498101260 msgr2=0x7f349810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:43.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 --2- 172.21.15.70:0/1009959611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3498101260 0x7f349810d650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f349000b0a0 tx=0x7f349000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 -- 172.21.15.70:0/1009959611 shutdown_connections 2023-10-18T12:25:43.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 --2- 172.21.15.70:0/1009959611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3498101260 0x7f349810d650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 --2- 172.21.15.70:0/1009959611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34981057c0 0x7f3498105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 --2- 172.21.15.70:0/1009959611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34981008a0 0x7f3498100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 -- 172.21.15.70:0/1009959611 >> 172.21.15.70:0/1009959611 conn(0x7f34980779e0 msgr2=0x7f3498077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:43.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 -- 172.21.15.70:0/1009959611 shutdown_connections 2023-10-18T12:25:43.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.596+0000 7f349f602700 1 -- 172.21.15.70:0/1009959611 wait complete. 2023-10-18T12:25:43.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.597+0000 7f349f602700 1 Processor -- start 2023-10-18T12:25:43.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 -- start start 2023-10-18T12:25:43.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34981008a0 0x7f3498196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:43.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3498101260 0x7f3498197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:43.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 0x7f349819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:43.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349819ab30 con 0x7f34981057c0 2023-10-18T12:25:43.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349819aca0 con 0x7f34981008a0 2023-10-18T12:25:43.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349f602700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349819ae60 con 0x7f3498101260 2023-10-18T12:25:43.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349db9f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 0x7f349819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349cb9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3498101260 0x7f3498197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349d39e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34981008a0 0x7f3498196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349db9f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 0x7f349819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47760/0 (socket says 172.21.15.70:47760) 2023-10-18T12:25:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.598+0000 7f349db9f700 1 -- 172.21.15.70:0/3551370269 learned_addr learned my addr 172.21.15.70:0/3551370269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349db9f700 1 -- 172.21.15.70:0/3551370269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3498101260 msgr2=0x7f3498197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349db9f700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3498101260 0x7f3498197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349db9f700 1 -- 172.21.15.70:0/3551370269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34981008a0 msgr2=0x7f3498196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349db9f700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34981008a0 0x7f3498196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349db9f700 1 -- 172.21.15.70:0/3551370269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490009d00 con 0x7f34981057c0 2023-10-18T12:25:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349db9f700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 0x7f349819a530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f349000ceb0 tx=0x7f349000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000e040 con 0x7f34981057c0 2023-10-18T12:25:43.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349819b0e0 con 0x7f34981057c0 2023-10-18T12:25:43.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f349001b450 con 0x7f34981057c0 2023-10-18T12:25:43.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.599+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34900052a0 con 0x7f34981057c0 2023-10-18T12:25:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.600+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498072bf0 con 0x7f34981057c0 2023-10-18T12:25:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.601+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3498108da0 con 0x7f34981057c0 2023-10-18T12:25:43.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:43 smithi070 ceph-mon[98639]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:43.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.610+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f349001b5c0 con 0x7f34981057c0 2023-10-18T12:25:43.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.612+0000 7f348e7fc700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34840a15e0 0x7f34840a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:43.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.613+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f349009c280 con 0x7f34981057c0 2023-10-18T12:25:43.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.614+0000 7f349d39e700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34840a15e0 0x7f34840a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:43.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.615+0000 7f349d39e700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34840a15e0 0x7f34840a3aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3498101930 tx=0x7f349400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:43.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.618+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3490098650 con 0x7f34981057c0 2023-10-18T12:25:43.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:43 smithi152 ceph-mon[134767]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:25:43.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.848+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f349819b410 con 0x7f34840a15e0 2023-10-18T12:25:43.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.862+0000 7f348e7fc700 1 -- 172.21.15.70:0/3551370269 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f349819b410 con 0x7f34840a15e0 2023-10-18T12:25:43.865 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-18T12:25:43.866 INFO:teuthology.orchestra.run.smithi070.stdout:mds.a smithi070 running (103s) 19s ago 102s 13.9M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 60e3449957ea 2023-10-18T12:25:43.866 INFO:teuthology.orchestra.run.smithi070.stdout:mds.b smithi152 running (101s) 29s ago 100s 32.1M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 c275d358b42f 2023-10-18T12:25:43.866 INFO:teuthology.orchestra.run.smithi070.stdout:mds.c smithi186 running (113s) 29s ago 113s 16.4M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 d23577b765d5 2023-10-18T12:25:43.866 INFO:teuthology.orchestra.run.smithi070.stdout:mds.d smithi070 running (2m) 19s ago 2m 13.8M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 3e6c508fefb9 2023-10-18T12:25:43.867 INFO:teuthology.orchestra.run.smithi070.stdout:mds.e smithi152 running (118s) 29s ago 117s 15.0M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 9cf2b8a6022c 2023-10-18T12:25:43.867 INFO:teuthology.orchestra.run.smithi070.stdout:mds.f smithi186 running (116s) 29s ago 115s 16.5M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 bdce724402a9 2023-10-18T12:25:43.867 INFO:teuthology.orchestra.run.smithi070.stdout:mds.g smithi070 running (106s) 19s ago 106s 17.8M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 429a5edd1f24 2023-10-18T12:25:43.867 INFO:teuthology.orchestra.run.smithi070.stdout:mds.h smithi152 running (111s) 29s ago 111s 16.5M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 f62118d4fa44 2023-10-18T12:25:43.867 INFO:teuthology.orchestra.run.smithi070.stdout:mds.i smithi186 running (109s) 29s ago 109s 16.5M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 e62d2df361d5 2023-10-18T12:25:43.868 INFO:teuthology.orchestra.run.smithi070.stdout:mds.j smithi070 running (2m) 19s ago 2m 13.9M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 dbdede8061c0 2023-10-18T12:25:43.868 INFO:teuthology.orchestra.run.smithi070.stdout:mds.k smithi152 running (2m) 29s ago 2m 17.6M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 8181e5187e61 2023-10-18T12:25:43.868 INFO:teuthology.orchestra.run.smithi070.stdout:mds.l smithi186 running (2m) 29s ago 2m 16.6M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 764ad2d4a69c 2023-10-18T12:25:43.868 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.x smithi070 *:9283,8765 running (9m) 19s ago 9m 522M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 5879f69518d5 2023-10-18T12:25:43.868 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.y smithi152 *:8443,8765 running (7m) 29s ago 7m 452M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 b3646de8663c 2023-10-18T12:25:43.869 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.z smithi186 *:8443,8765 running (7m) 29s ago 7m 450M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 f9c9a71d42d0 2023-10-18T12:25:43.869 INFO:teuthology.orchestra.run.smithi070.stdout:mon.a smithi070 running (9m) 19s ago 9m 100M 2048M 18.0.0-6786-gf7420d7f 34f3475ed6c6 7c1966836a81 2023-10-18T12:25:43.869 INFO:teuthology.orchestra.run.smithi070.stdout:mon.b smithi152 running (7m) 29s ago 7m 70.9M 2048M 18.0.0-6786-gf7420d7f 34f3475ed6c6 09965e2e0023 2023-10-18T12:25:43.869 INFO:teuthology.orchestra.run.smithi070.stdout:mon.c smithi186 running (8m) 29s ago 7m 68.9M 2048M 18.0.0-6786-gf7420d7f 34f3475ed6c6 7d107bdaa63c 2023-10-18T12:25:43.869 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (6m) 19s ago 6m 70.3M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 138941c7d602 2023-10-18T12:25:43.869 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi152 running (6m) 29s ago 6m 79.4M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 dd43f9e1e808 2023-10-18T12:25:43.870 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi186 running (5m) 29s ago 5m 75.7M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 38128dd5c015 2023-10-18T12:25:43.870 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi070 running (5m) 19s ago 5m 67.8M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 66838235fd3a 2023-10-18T12:25:43.870 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi152 running (5m) 29s ago 5m 61.8M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 1a53a1c4d115 2023-10-18T12:25:43.870 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi186 running (4m) 29s ago 4m 67.6M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 233850bf1fa8 2023-10-18T12:25:43.870 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6 smithi070 running (4m) 19s ago 4m 60.7M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 0ef1faa07155 2023-10-18T12:25:43.871 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 smithi152 running (3m) 29s ago 3m 64.8M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 726b09fbf0cb 2023-10-18T12:25:43.871 INFO:teuthology.orchestra.run.smithi070.stdout:osd.8 smithi186 running (3m) 29s ago 3m 63.4M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 5c7f06c0fcba 2023-10-18T12:25:43.871 INFO:teuthology.orchestra.run.smithi070.stdout:osd.9 smithi070 running (3m) 19s ago 3m 58.9M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 262d3157d87c 2023-10-18T12:25:43.871 INFO:teuthology.orchestra.run.smithi070.stdout:osd.10 smithi152 running (2m) 29s ago 2m 62.0M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 dae2609f1fc5 2023-10-18T12:25:43.872 INFO:teuthology.orchestra.run.smithi070.stdout:osd.11 smithi186 running (2m) 29s ago 2m 53.6M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 9f7cd5523cc3 2023-10-18T12:25:43.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34840a15e0 msgr2=0x7f34840a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34840a15e0 0x7f34840a3aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3498101930 tx=0x7f349400b410 comp rx=0 tx=0).stop 2023-10-18T12:25:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 msgr2=0x7f349819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 0x7f349819a530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f349000ceb0 tx=0x7f349000cee0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 shutdown_connections 2023-10-18T12:25:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34981057c0 0x7f349819a530 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34840a15e0 0x7f34840a3aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34981008a0 0x7f3498196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 --2- 172.21.15.70:0/3551370269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3498101260 0x7f3498197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:43.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 >> 172.21.15.70:0/3551370269 conn(0x7f34980779e0 msgr2=0x7f34980fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:43.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 shutdown_connections 2023-10-18T12:25:43.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:43.865+0000 7f349f602700 1 -- 172.21.15.70:0/3551370269 wait complete. 2023-10-18T12:25:44.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-18T12:25:45.103 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:45.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:45 smithi186 ceph-mon[135253]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:45.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:45 smithi186 ceph-mon[135253]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:45.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:45 smithi186 ceph-mon[135253]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-18T12:25:45.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:45 smithi070 ceph-mon[98639]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:45.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:45 smithi070 ceph-mon[98639]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:45.502 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:45 smithi070 ceph-mon[98639]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-18T12:25:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:45 smithi152 ceph-mon[134767]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:45.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:45 smithi152 ceph-mon[134767]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:45.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:45 smithi152 ceph-mon[134767]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-18T12:25:46.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 -- 172.21.15.70:0/3809885999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d8073340 msgr2=0x7f12d810cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:46.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/3809885999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d8073340 0x7f12d810cd10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009990 tx=0x7f12cc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 -- 172.21.15.70:0/3809885999 shutdown_connections 2023-10-18T12:25:46.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/3809885999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d810d250 0x7f12d810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/3809885999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 0x7f12d8072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/3809885999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d8073340 0x7f12d810cd10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 -- 172.21.15.70:0/3809885999 >> 172.21.15.70:0/3809885999 conn(0x7f12d80779e0 msgr2=0x7f12d8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:46.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 -- 172.21.15.70:0/3809885999 shutdown_connections 2023-10-18T12:25:46.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.495+0000 7f12dd3ff700 1 -- 172.21.15.70:0/3809885999 wait complete. 2023-10-18T12:25:46.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.496+0000 7f12dd3ff700 1 Processor -- start 2023-10-18T12:25:46.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.496+0000 7f12dd3ff700 1 -- start start 2023-10-18T12:25:46.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12dd3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 0x7f12d8198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:46.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12dd3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 0x7f12d8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:46.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12dd3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d810d250 0x7f12d819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:46.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12dd3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d819dd80 con 0x7f12d8073340 2023-10-18T12:25:46.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12dd3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d819def0 con 0x7f12d8072a20 2023-10-18T12:25:46.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12dd3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d819e0b0 con 0x7f12d810d250 2023-10-18T12:25:46.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 0x7f12d8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:46.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 0x7f12d8198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:46.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 0x7f12d8199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47780/0 (socket says 172.21.15.70:47780) 2023-10-18T12:25:46.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d67fc700 1 -- 172.21.15.70:0/1146595374 learned_addr learned my addr 172.21.15.70:0/1146595374 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:46.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d77fe700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d810d250 0x7f12d819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:46.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d6ffd700 1 -- 172.21.15.70:0/1146595374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d810d250 msgr2=0x7f12d819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:46.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d6ffd700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d810d250 0x7f12d819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d6ffd700 1 -- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 msgr2=0x7f12d8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:46.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d6ffd700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 0x7f12d8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d6ffd700 1 -- 172.21.15.70:0/1146595374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0095f0 con 0x7f12d8072a20 2023-10-18T12:25:46.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.497+0000 7f12d67fc700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 0x7f12d8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:46.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.498+0000 7f12d6ffd700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 0x7f12d8198ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9f0 tx=0x7f12c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:46.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.498+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800c9c0 con 0x7f12d8072a20 2023-10-18T12:25:46.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.498+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12c8010430 con 0x7f12d8072a20 2023-10-18T12:25:46.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.498+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8018680 con 0x7f12d8072a20 2023-10-18T12:25:46.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.498+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d819e390 con 0x7f12d8072a20 2023-10-18T12:25:46.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.498+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d81a5bf0 con 0x7f12d8072a20 2023-10-18T12:25:46.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.499+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d8108460 con 0x7f12d8072a20 2023-10-18T12:25:46.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.501+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f12c8010860 con 0x7f12d8072a20 2023-10-18T12:25:46.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.503+0000 7f12bffff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12c00a1550 0x7f12c00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:46.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.503+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f12c8014070 con 0x7f12d8072a20 2023-10-18T12:25:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.503+0000 7f12d67fc700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12c00a1550 0x7f12c00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.503+0000 7f12d67fc700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12c00a1550 0x7f12c00a3a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f12d819a3e0 tx=0x7f12cc005bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.505+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f12c8096640 con 0x7f12d8072a20 2023-10-18T12:25:46.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.712+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f12d819a190 con 0x7f12c00a1550 2023-10-18T12:25:46.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.718+0000 7f12bffff700 1 -- 172.21.15.70:0/1146595374 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f12d819a190 con 0x7f12c00a1550 2023-10-18T12:25:46.722 INFO:teuthology.orchestra.run.smithi070.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-18T12:25:46.722 INFO:teuthology.orchestra.run.smithi070.stdout:mds.all 12/12 32s ago 2m smithi070=a;smithi070=d;smithi070=g;smithi070=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-18T12:25:46.722 INFO:teuthology.orchestra.run.smithi070.stdout:mgr 3/3 32s ago 7m smithi070=x;smithi152=y;smithi186=z;count:3 2023-10-18T12:25:46.722 INFO:teuthology.orchestra.run.smithi070.stdout:mon 3/3 32s ago 8m smithi070:172.21.15.70=a;smithi152:172.21.15.152=b;smithi186:172.21.15.186=c;count:3 2023-10-18T12:25:46.723 INFO:teuthology.orchestra.run.smithi070.stdout:osd 12 32s ago - 2023-10-18T12:25:46.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12c00a1550 msgr2=0x7f12c00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:46.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12c00a1550 0x7f12c00a3a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f12d819a3e0 tx=0x7f12cc005bc0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 msgr2=0x7f12d8198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:46.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 0x7f12d8198ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9f0 tx=0x7f12c800ed00 comp rx=0 tx=0).stop 2023-10-18T12:25:46.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 shutdown_connections 2023-10-18T12:25:46.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d810d250 0x7f12d819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12c00a1550 0x7f12c00a3a10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12d8072a20 0x7f12d8198ed0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 --2- 172.21.15.70:0/1146595374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d8073340 0x7f12d8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:46.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 >> 172.21.15.70:0/1146595374 conn(0x7f12d80779e0 msgr2=0x7f12d80fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:46.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 shutdown_connections 2023-10-18T12:25:46.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:46.721+0000 7f12dd3ff700 1 -- 172.21.15.70:0/1146595374 wait complete. 2023-10-18T12:25:47.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-18T12:25:47.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:47 smithi186 ceph-mon[135253]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:47.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:47 smithi070 ceph-mon[98639]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:47 smithi152 ceph-mon[134767]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:25:48.033 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:48.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:48 smithi186 ceph-mon[135253]: from='client.25325 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:48.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:48 smithi070 ceph-mon[98639]: from='client.25325 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:48.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:48 smithi152 ceph-mon[134767]: from='client.25325 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:49.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:49 smithi186 ceph-mon[135253]: pgmap v306: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:49.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 -- 172.21.15.70:0/3269689736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 msgr2=0x7fa8f8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:49.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/3269689736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8100d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009990 tx=0x7fa8ec009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 -- 172.21.15.70:0/3269689736 shutdown_connections 2023-10-18T12:25:49.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/3269689736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f8101260 0x7fa8f810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/3269689736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8f81057c0 0x7fa8f8105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/3269689736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8100d20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 -- 172.21.15.70:0/3269689736 >> 172.21.15.70:0/3269689736 conn(0x7fa8f80779e0 msgr2=0x7fa8f8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:49.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 -- 172.21.15.70:0/3269689736 shutdown_connections 2023-10-18T12:25:49.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.512+0000 7fa8fe13f700 1 -- 172.21.15.70:0/3269689736 wait complete. 2023-10-18T12:25:49.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.513+0000 7fa8fe13f700 1 Processor -- start 2023-10-18T12:25:49.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.513+0000 7fa8fe13f700 1 -- start start 2023-10-18T12:25:49.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8fe13f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:49.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8fe13f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f8101260 0x7fa8f8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:49.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8fe13f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8f81057c0 0x7fa8f819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:49.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8fe13f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f819dd30 con 0x7fa8f81008a0 2023-10-18T12:25:49.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8fe13f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f819dea0 con 0x7fa8f81057c0 2023-10-18T12:25:49.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8fe13f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f819e060 con 0x7fa8f8101260 2023-10-18T12:25:49.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:49.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49846/0 (socket says 172.21.15.70:49846) 2023-10-18T12:25:49.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 -- 172.21.15.70:0/2674354142 learned_addr learned my addr 172.21.15.70:0/2674354142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:49.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f8101260 0x7fa8f8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:49.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f7fff700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8f81057c0 0x7fa8f819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:49.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 -- 172.21.15.70:0/2674354142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f8101260 msgr2=0x7fa8f8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:49.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f8101260 0x7fa8f8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 -- 172.21.15.70:0/2674354142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8f81057c0 msgr2=0x7fa8f819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:49.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8f81057c0 0x7fa8f819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.514+0000 7fa8f77fe700 1 -- 172.21.15.70:0/2674354142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec0095f0 con 0x7fa8f81008a0 2023-10-18T12:25:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.515+0000 7fa8f77fe700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8198e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e970 tx=0x7fa8e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.515+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8009790 con 0x7fa8f81008a0 2023-10-18T12:25:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.515+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8e800c7e0 con 0x7fa8f81008a0 2023-10-18T12:25:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.515+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f819e340 con 0x7fa8f81008a0 2023-10-18T12:25:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.515+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8003c70 con 0x7fa8f81008a0 2023-10-18T12:25:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.515+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f81a5ba0 con 0x7fa8f81008a0 2023-10-18T12:25:49.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.516+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f8101f30 con 0x7fa8f81008a0 2023-10-18T12:25:49.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.518+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa8e800cc50 con 0x7fa8f81008a0 2023-10-18T12:25:49.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.519+0000 7fa8f4ff9700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8e00a14d0 0x7fa8e00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:49.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.520+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fa8e8014070 con 0x7fa8f81008a0 2023-10-18T12:25:49.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.520+0000 7fa8f6ffd700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8e00a14d0 0x7fa8e00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:49.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.520+0000 7fa8f6ffd700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8e00a14d0 0x7fa8e00a3990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8f819a390 tx=0x7fa8ec005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:49.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.524+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa8e8096500 con 0x7fa8f81008a0 2023-10-18T12:25:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:49 smithi070 ceph-mon[98639]: pgmap v306: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:49.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:49 smithi152 ceph-mon[134767]: pgmap v306: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:49.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.732+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8f81107a0 con 0x7fa8e00a14d0 2023-10-18T12:25:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.734+0000 7fa8f4ff9700 1 -- 172.21.15.70:0/2674354142 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fa8f81107a0 con 0x7fa8e00a14d0 2023-10-18T12:25:49.737 INFO:teuthology.orchestra.run.smithi070.stdout:HOST ADDR LABELS STATUS 2023-10-18T12:25:49.737 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070 172.21.15.70 2023-10-18T12:25:49.737 INFO:teuthology.orchestra.run.smithi070.stdout:smithi152 172.21.15.152 2023-10-18T12:25:49.738 INFO:teuthology.orchestra.run.smithi070.stdout:smithi186 172.21.15.186 2023-10-18T12:25:49.738 INFO:teuthology.orchestra.run.smithi070.stdout:3 hosts in cluster 2023-10-18T12:25:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8e00a14d0 msgr2=0x7fa8e00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8e00a14d0 0x7fa8e00a3990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8f819a390 tx=0x7fa8ec005fd0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 msgr2=0x7fa8f8198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8198e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e970 tx=0x7fa8e800ec80 comp rx=0 tx=0).stop 2023-10-18T12:25:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 shutdown_connections 2023-10-18T12:25:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa8e00a14d0 0x7fa8e00a3990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8f81057c0 0x7fa8f819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f81008a0 0x7fa8f8198e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 --2- 172.21.15.70:0/2674354142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f8101260 0x7fa8f8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:49.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 >> 172.21.15.70:0/2674354142 conn(0x7fa8f80779e0 msgr2=0x7fa8f810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:49.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.736+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 shutdown_connections 2023-10-18T12:25:49.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:49.737+0000 7fa8fe13f700 1 -- 172.21.15.70:0/2674354142 wait complete. 2023-10-18T12:25:50.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-18T12:25:50.865 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:51.452 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:51 smithi070 ceph-mon[98639]: pgmap v307: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2023-10-18T12:25:51.452 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:51 smithi070 ceph-mon[98639]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:51 smithi186 ceph-mon[135253]: pgmap v307: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2023-10-18T12:25:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:51 smithi186 ceph-mon[135253]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:51.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:51 smithi152 ceph-mon[134767]: pgmap v307: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2023-10-18T12:25:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:51 smithi152 ceph-mon[134767]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 -- 172.21.15.70:0/3370111747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201008a0 msgr2=0x7fae20100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 --2- 172.21.15.70:0/3370111747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201008a0 0x7fae20100d20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae10009990 tx=0x7fae10009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 -- 172.21.15.70:0/3370111747 shutdown_connections 2023-10-18T12:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 --2- 172.21.15.70:0/3370111747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae20101260 0x7fae2010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 --2- 172.21.15.70:0/3370111747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae201057c0 0x7fae20105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 --2- 172.21.15.70:0/3370111747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201008a0 0x7fae20100d20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 -- 172.21.15.70:0/3370111747 >> 172.21.15.70:0/3370111747 conn(0x7fae200779e0 msgr2=0x7fae20077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:52.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 -- 172.21.15.70:0/3370111747 shutdown_connections 2023-10-18T12:25:52.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.346+0000 7fae273af700 1 -- 172.21.15.70:0/3370111747 wait complete. 2023-10-18T12:25:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.347+0000 7fae273af700 1 Processor -- start 2023-10-18T12:25:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.347+0000 7fae273af700 1 -- start start 2023-10-18T12:25:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae273af700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 0x7fae2019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae273af700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae20101260 0x7fae2019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae273af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201057c0 0x7fae20196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae273af700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae201970d0 con 0x7fae201057c0 2023-10-18T12:25:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae273af700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20197240 con 0x7fae201008a0 2023-10-18T12:25:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae273af700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20197400 con 0x7fae20101260 2023-10-18T12:25:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 0x7fae2019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 0x7fae2019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45372/0 (socket says 172.21.15.70:45372) 2023-10-18T12:25:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2594c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201057c0 0x7fae20196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 -- 172.21.15.70:0/3965815600 learned_addr learned my addr 172.21.15.70:0/3965815600 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2494a700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae20101260 0x7fae2019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 -- 172.21.15.70:0/3965815600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae20101260 msgr2=0x7fae2019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae20101260 0x7fae2019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 -- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201057c0 msgr2=0x7fae20196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:52.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201057c0 0x7fae20196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2514b700 1 -- 172.21.15.70:0/3965815600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100095f0 con 0x7fae201008a0 2023-10-18T12:25:52.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.348+0000 7fae2594c700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201057c0 0x7fae20196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:52.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.349+0000 7fae2514b700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 0x7fae2019c8d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e970 tx=0x7fae1c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:52.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.349+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c009790 con 0x7fae201008a0 2023-10-18T12:25:52.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.349+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae1c00c7e0 con 0x7fae201008a0 2023-10-18T12:25:52.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.349+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae201976e0 con 0x7fae201008a0 2023-10-18T12:25:52.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.349+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c003c70 con 0x7fae201008a0 2023-10-18T12:25:52.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.349+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae201a38c0 con 0x7fae201008a0 2023-10-18T12:25:52.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.350+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20108da0 con 0x7fae201008a0 2023-10-18T12:25:52.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.352+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fae1c00cc50 con 0x7fae201008a0 2023-10-18T12:25:52.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.353+0000 7fae167fc700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae0c0a14d0 0x7fae0c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:52.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.353+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fae1c014070 con 0x7fae201008a0 2023-10-18T12:25:52.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.354+0000 7fae2494a700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae0c0a14d0 0x7fae0c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:52.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.354+0000 7fae2494a700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae0c0a14d0 0x7fae0c0a3990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fae20077ed0 tx=0x7fae10005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:52.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.356+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae1c0968f0 con 0x7fae201008a0 2023-10-18T12:25:52.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.563+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fae20197ed0 con 0x7fae0c0a14d0 2023-10-18T12:25:52.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.565+0000 7fae167fc700 1 -- 172.21.15.70:0/3965815600 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fae20197ed0 con 0x7fae0c0a14d0 2023-10-18T12:25:52.568 INFO:teuthology.orchestra.run.smithi070.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-18T12:25:52.569 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002P480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-10-18T12:25:52.569 INFO:teuthology.orchestra.run.smithi070.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-18T12:25:52.569 INFO:teuthology.orchestra.run.smithi070.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-18T12:25:52.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae0c0a14d0 msgr2=0x7fae0c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:52.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae0c0a14d0 0x7fae0c0a3990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fae20077ed0 tx=0x7fae10005fd0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 msgr2=0x7fae2019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:52.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 0x7fae2019c8d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e970 tx=0x7fae1c00ec80 comp rx=0 tx=0).stop 2023-10-18T12:25:52.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 shutdown_connections 2023-10-18T12:25:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae201057c0 0x7fae20196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fae0c0a14d0 0x7fae0c0a3990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae201008a0 0x7fae2019c8d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 --2- 172.21.15.70:0/3965815600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae20101260 0x7fae2019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:52.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 >> 172.21.15.70:0/3965815600 conn(0x7fae200779e0 msgr2=0x7fae200fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:52.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 shutdown_connections 2023-10-18T12:25:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:52.567+0000 7fae273af700 1 -- 172.21.15.70:0/3965815600 wait complete. 2023-10-18T12:25:53.254 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-18T12:25:53.264 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:25:53.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-18T12:25:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:53 smithi186 ceph-mon[135253]: pgmap v308: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:53 smithi070 ceph-mon[98639]: pgmap v308: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:53 smithi152 ceph-mon[134767]: pgmap v308: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:54.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:54 smithi186 ceph-mon[135253]: from='client.25337 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:54.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:54 smithi070 ceph-mon[98639]: from='client.25337 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:54.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:54 smithi152 ceph-mon[134767]: from='client.25337 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:25:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 -- 172.21.15.70:0/1319384716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c072f60 msgr2=0x7fbb0c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 --2- 172.21.15.70:0/1319384716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c072f60 0x7fbb0c0733e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009990 tx=0x7fbb00009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 -- 172.21.15.70:0/1319384716 shutdown_connections 2023-10-18T12:25:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 --2- 172.21.15.70:0/1319384716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c111570 0x7fbb0c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 --2- 172.21.15.70:0/1319384716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 0x7fbb0c072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 --2- 172.21.15.70:0/1319384716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c072f60 0x7fbb0c0733e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 -- 172.21.15.70:0/1319384716 >> 172.21.15.70:0/1319384716 conn(0x7fbb0c0779e0 msgr2=0x7fbb0c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:55.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 -- 172.21.15.70:0/1319384716 shutdown_connections 2023-10-18T12:25:55.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.362+0000 7fbb117af700 1 -- 172.21.15.70:0/1319384716 wait complete. 2023-10-18T12:25:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 Processor -- start 2023-10-18T12:25:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 -- start start 2023-10-18T12:25:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 0x7fbb0c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 0x7fbb0c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c111570 0x7fbb0c1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c1a2180 con 0x7fbb0c111570 2023-10-18T12:25:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c1a22f0 con 0x7fbb0c074770 2023-10-18T12:25:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.363+0000 7fbb117af700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c1a24b0 con 0x7fbb0c072f60 2023-10-18T12:25:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 0x7fbb0c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 0x7fbb0c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 0x7fbb0c19d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47502/0 (socket says 172.21.15.70:47502) 2023-10-18T12:25:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 -- 172.21.15.70:0/2358586837 learned_addr learned my addr 172.21.15.70:0/2358586837 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0b7fe700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c111570 0x7fbb0c1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 -- 172.21.15.70:0/2358586837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 msgr2=0x7fbb0c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 0x7fbb0c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 -- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c111570 msgr2=0x7fbb0c1a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c111570 0x7fbb0c1a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 -- 172.21.15.70:0/2358586837 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000095f0 con 0x7fbb0c072f60 2023-10-18T12:25:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0a7fc700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 0x7fbb0c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0b7fe700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c111570 0x7fbb0c1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:25:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.364+0000 7fbb0affd700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 0x7fbb0c19d2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00e9a0 tx=0x7fbafc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:55.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.365+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc009790 con 0x7fbb0c072f60 2023-10-18T12:25:55.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.365+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbafc010430 con 0x7fbb0c072f60 2023-10-18T12:25:55.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.365+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c1a2790 con 0x7fbb0c072f60 2023-10-18T12:25:55.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.365+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc018680 con 0x7fbb0c072f60 2023-10-18T12:25:55.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.365+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c1a9ff0 con 0x7fbb0c072f60 2023-10-18T12:25:55.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.366+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c0740b0 con 0x7fbb0c072f60 2023-10-18T12:25:55.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.368+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fbafc0105a0 con 0x7fbb0c072f60 2023-10-18T12:25:55.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.369+0000 7fbaf3fff700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbaf40a1550 0x7fbaf40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:55.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.369+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fbafc014070 con 0x7fbb0c072f60 2023-10-18T12:25:55.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.369+0000 7fbb0a7fc700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbaf40a1550 0x7fbaf40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:55.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.370+0000 7fbb0a7fc700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbaf40a1550 0x7fbaf40a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b2e0 tx=0x7fbb00005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:55.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.371+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbafc096570 con 0x7fbb0c072f60 2023-10-18T12:25:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:55 smithi070 ceph-mon[98639]: pgmap v309: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:55.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.611+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb0c04e4c0 con 0x7fbb0c072f60 2023-10-18T12:25:55.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.612+0000 7fbaf3fff700 1 -- 172.21.15.70:0/2358586837 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2917 (secure 0 0 0) 0x7fbafc099b90 con 0x7fbb0c072f60 2023-10-18T12:25:55.615 INFO:teuthology.orchestra.run.smithi070.stdout:e20 2023-10-18T12:25:55.615 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T12:25:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T12:25:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2023-10-18T12:25:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2023-10-18T12:25:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2023-10-18T12:25:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 20 2023-10-18T12:25:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-18T12:25:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:created 2023-10-18T12:24:09.485042+0000 2023-10-18T12:25:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2023-10-18T12:24:34.631959+0000 2023-10-18T12:25:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2023-10-18T12:25:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2023-10-18T12:25:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 300 2023-10-18T12:25:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2023-10-18T12:25:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2023-10-18T12:25:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2023-10-18T12:25:55.619 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2023-10-18T12:25:55.619 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2023-10-18T12:25:55.619 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2023-10-18T12:25:55.619 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T12:25:55.619 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2023-10-18T12:25:55.620 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2023-10-18T12:25:55.620 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=24743} 2023-10-18T12:25:55.620 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2023-10-18T12:25:55.620 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2023-10-18T12:25:55.620 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2023-10-18T12:25:55.621 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3,4] 2023-10-18T12:25:55.621 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2023-10-18T12:25:55.621 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2023-10-18T12:25:55.621 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2023-10-18T12:25:55.621 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2023-10-18T12:25:55.622 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 0 2023-10-18T12:25:55.622 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b{0:24743} state up:active seq 3 addr [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.622 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.g{0:24719} state up:standby-replay seq 1 addr [v2:172.21.15.70:6838/3607185387,v1:172.21.15.70:6839/3607185387] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:55.623 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2023-10-18T12:25:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:55.623 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.j{-1:14631} state up:standby seq 1 addr [v2:172.21.15.70:6834/104240782,v1:172.21.15.70:6835/104240782] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.623 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d{-1:14643} state up:standby seq 1 addr [v2:172.21.15.70:6836/3255312208,v1:172.21.15.70:6837/3255312208] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.623 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a{-1:14706} state up:standby seq 1 addr [v2:172.21.15.70:6840/3718886784,v1:172.21.15.70:6841/3718886784] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.624 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.l{-1:24619} state up:standby seq 1 addr [v2:172.21.15.186:6832/339260253,v1:172.21.15.186:6833/339260253] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.624 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.k{-1:24629} state up:standby seq 1 addr [v2:172.21.15.152:6832/1678353915,v1:172.21.15.152:6833/1678353915] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.624 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.e{-1:24665} state up:standby seq 1 addr [v2:172.21.15.152:6834/2963573440,v1:172.21.15.152:6835/2963573440] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.624 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.f{-1:24673} state up:standby seq 1 addr [v2:172.21.15.186:6834/3580803773,v1:172.21.15.186:6835/3580803773] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.624 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c{-1:24685} state up:standby seq 1 addr [v2:172.21.15.186:6836/569012458,v1:172.21.15.186:6837/569012458] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.625 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.h{-1:24692} state up:standby seq 1 addr [v2:172.21.15.152:6836/1543432937,v1:172.21.15.152:6837/1543432937] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.625 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.i{-1:24697} state up:standby seq 1 addr [v2:172.21.15.186:6838/2873838836,v1:172.21.15.186:6839/2873838836] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T12:25:55.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbaf40a1550 msgr2=0x7fbaf40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:55.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbaf40a1550 0x7fbaf40a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b2e0 tx=0x7fbb00005960 comp rx=0 tx=0).stop 2023-10-18T12:25:55.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 msgr2=0x7fbb0c19d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:55.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 0x7fbb0c19d2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00e9a0 tx=0x7fbafc00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 shutdown_connections 2023-10-18T12:25:55.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb0c111570 0x7fbb0c1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb0c072f60 0x7fbb0c19d2d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbaf40a1550 0x7fbaf40a3a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 --2- 172.21.15.70:0/2358586837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c074770 0x7fbb0c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:55.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 >> 172.21.15.70:0/2358586837 conn(0x7fbb0c0779e0 msgr2=0x7fbb0c1101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:55.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.614+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 shutdown_connections 2023-10-18T12:25:55.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:55.615+0000 7fbb117af700 1 -- 172.21.15.70:0/2358586837 wait complete. 2023-10-18T12:25:55.628 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:25:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:55 smithi152 ceph-mon[134767]: pgmap v309: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:55.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:55 smithi186 ceph-mon[135253]: pgmap v309: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:56.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-18T12:25:56.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:56 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2358586837' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-18T12:25:56.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:56 smithi070 ceph-mon[98639]: pgmap v310: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:56.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:56 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2358586837' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-18T12:25:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:56 smithi152 ceph-mon[134767]: pgmap v310: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:56.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:56 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2358586837' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-18T12:25:56.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:56 smithi186 ceph-mon[135253]: pgmap v310: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:25:58.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 -- 172.21.15.70:0/2114838839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1057c0 msgr2=0x7fdb8c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:58.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 --2- 172.21.15.70:0/2114838839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1057c0 0x7fdb8c105ba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009990 tx=0x7fdb7c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 -- 172.21.15.70:0/2114838839 shutdown_connections 2023-10-18T12:25:58.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 --2- 172.21.15.70:0/2114838839 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb8c101260 0x7fdb8c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 --2- 172.21.15.70:0/2114838839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1057c0 0x7fdb8c105ba0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 --2- 172.21.15.70:0/2114838839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 -- 172.21.15.70:0/2114838839 >> 172.21.15.70:0/2114838839 conn(0x7fdb8c0779e0 msgr2=0x7fdb8c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:58.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 -- 172.21.15.70:0/2114838839 shutdown_connections 2023-10-18T12:25:58.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.248+0000 7fdb91304700 1 -- 172.21.15.70:0/2114838839 wait complete. 2023-10-18T12:25:58.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 Processor -- start 2023-10-18T12:25:58.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 -- start start 2023-10-18T12:25:58.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:58.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb8c101260 0x7fdb8c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:58.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb8c1057c0 0x7fdb8c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:58.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c197160 con 0x7fdb8c1008a0 2023-10-18T12:25:58.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1972d0 con 0x7fdb8c101260 2023-10-18T12:25:58.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.250+0000 7fdb91304700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c197490 con 0x7fdb8c1057c0 2023-10-18T12:25:58.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:58.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49276/0 (socket says 172.21.15.70:49276) 2023-10-18T12:25:58.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 -- 172.21.15.70:0/2936142713 learned_addr learned my addr 172.21.15.70:0/2936142713 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:58.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8a7fc700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb8c101260 0x7fdb8c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:58.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8b7fe700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb8c1057c0 0x7fdb8c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:58.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 -- 172.21.15.70:0/2936142713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb8c1057c0 msgr2=0x7fdb8c196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:58.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb8c1057c0 0x7fdb8c196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 -- 172.21.15.70:0/2936142713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb8c101260 msgr2=0x7fdb8c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:58.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb8c101260 0x7fdb8c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 -- 172.21.15.70:0/2936142713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0095f0 con 0x7fdb8c1008a0 2023-10-18T12:25:58.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.251+0000 7fdb8affd700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c19c970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c003730 tx=0x7fdb7c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:58.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.252+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c01d070 con 0x7fdb8c1008a0 2023-10-18T12:25:58.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.252+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7c0054f0 con 0x7fdb8c1008a0 2023-10-18T12:25:58.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.252+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c197710 con 0x7fdb8c1008a0 2023-10-18T12:25:58.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.252+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c004b30 con 0x7fdb8c1008a0 2023-10-18T12:25:58.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.252+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c072bf0 con 0x7fdb8c1008a0 2023-10-18T12:25:58.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.253+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c108d80 con 0x7fdb8c1008a0 2023-10-18T12:25:58.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.255+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fdb7c004c90 con 0x7fdb8c1008a0 2023-10-18T12:25:58.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.256+0000 7fdb73fff700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb740a14d0 0x7fdb740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:58.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.256+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fdb7c0cdfa0 con 0x7fdb8c1008a0 2023-10-18T12:25:58.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.256+0000 7fdb8a7fc700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb740a14d0 0x7fdb740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:58.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.257+0000 7fdb8a7fc700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb740a14d0 0x7fdb740a3990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c197c70 tx=0x7fdb8000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:58.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.260+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb7c097240 con 0x7fdb8c1008a0 2023-10-18T12:25:58.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.463+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fdb8c197f70 con 0x7fdb8c1008a0 2023-10-18T12:25:58.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.464+0000 7fdb73fff700 1 -- 172.21.15.70:0/2936142713 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5987 (secure 0 0 0) 0x7fdb7c09a860 con 0x7fdb8c1008a0 2023-10-18T12:25:58.467 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 91 2023-10-18T12:25:58.467 INFO:teuthology.orchestra.run.smithi070.stdout:fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:25:58.467 INFO:teuthology.orchestra.run.smithi070.stdout:created 2023-10-18T12:16:03.804671+0000 2023-10-18T12:25:58.468 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2023-10-18T12:25:15.204744+0000 2023-10-18T12:25:58.468 INFO:teuthology.orchestra.run.smithi070.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-18T12:25:58.468 INFO:teuthology.orchestra.run.smithi070.stdout:crush_version 27 2023-10-18T12:25:58.468 INFO:teuthology.orchestra.run.smithi070.stdout:full_ratio 0.95 2023-10-18T12:25:58.468 INFO:teuthology.orchestra.run.smithi070.stdout:backfillfull_ratio 0.9 2023-10-18T12:25:58.469 INFO:teuthology.orchestra.run.smithi070.stdout:nearfull_ratio 0.85 2023-10-18T12:25:58.469 INFO:teuthology.orchestra.run.smithi070.stdout:require_min_compat_client luminous 2023-10-18T12:25:58.469 INFO:teuthology.orchestra.run.smithi070.stdout:min_compat_client luminous 2023-10-18T12:25:58.469 INFO:teuthology.orchestra.run.smithi070.stdout:require_osd_release reef 2023-10-18T12:25:58.469 INFO:teuthology.orchestra.run.smithi070.stdout:stretch_mode_enabled false 2023-10-18T12:25:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-18T12:25:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-18T12:25:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-10-18T12:25:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-18T12:25:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:max_osd 12 2023-10-18T12:25:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 up in weight 1 up_from 9 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6802/2979990222,v1:172.21.15.70:6803/2979990222] [v2:172.21.15.70:6804/2979990222,v1:172.21.15.70:6805/2979990222] exists,up 86cb6d70-44d3-441f-9d82-d24f1167aac3 2023-10-18T12:25:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 up in weight 1 up_from 14 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/1167081028,v1:172.21.15.152:6801/1167081028] [v2:172.21.15.152:6802/1167081028,v1:172.21.15.152:6803/1167081028] exists,up 48353e9d-5623-4c9b-b90a-35487c80253c 2023-10-18T12:25:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 up in weight 1 up_from 19 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/1687717134,v1:172.21.15.186:6801/1687717134] [v2:172.21.15.186:6802/1687717134,v1:172.21.15.186:6803/1687717134] exists,up 972ea005-3f12-4646-896e-89622a35b19a 2023-10-18T12:25:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6810/3863911164,v1:172.21.15.70:6811/3863911164] [v2:172.21.15.70:6812/3863911164,v1:172.21.15.70:6813/3863911164] exists,up c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8 2023-10-18T12:25:58.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 up in weight 1 up_from 34 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/934272440,v1:172.21.15.152:6809/934272440] [v2:172.21.15.152:6810/934272440,v1:172.21.15.152:6811/934272440] exists,up ff61a4d2-db3e-4df1-abbc-daf21ea63965 2023-10-18T12:25:58.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 up in weight 1 up_from 39 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/443683147,v1:172.21.15.186:6809/443683147] [v2:172.21.15.186:6810/443683147,v1:172.21.15.186:6811/443683147] exists,up cd152ca6-76f9-4773-887d-9a751fd85319 2023-10-18T12:25:58.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6 up in weight 1 up_from 47 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6818/3765589692,v1:172.21.15.70:6819/3765589692] [v2:172.21.15.70:6820/3765589692,v1:172.21.15.70:6821/3765589692] exists,up 09373159-f65f-4a33-a0cd-7adaba830621 2023-10-18T12:25:58.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 up in weight 1 up_from 52 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6816/2054103643,v1:172.21.15.152:6817/2054103643] [v2:172.21.15.152:6818/2054103643,v1:172.21.15.152:6819/2054103643] exists,up 2979cb11-038c-482c-b796-7d8dc3f6da42 2023-10-18T12:25:58.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.8 up in weight 1 up_from 60 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/2454566106,v1:172.21.15.186:6817/2454566106] [v2:172.21.15.186:6818/2454566106,v1:172.21.15.186:6819/2454566106] exists,up ba13df86-7cc5-4b47-8890-c35b0f2b2ce0 2023-10-18T12:25:58.473 INFO:teuthology.orchestra.run.smithi070.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6826/4034457990,v1:172.21.15.70:6827/4034457990] [v2:172.21.15.70:6828/4034457990,v1:172.21.15.70:6829/4034457990] exists,up 6976bc71-bb5c-49ac-8b46-9b64c66fd4ed 2023-10-18T12:25:58.473 INFO:teuthology.orchestra.run.smithi070.stdout:osd.10 up in weight 1 up_from 73 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6824/4238500329,v1:172.21.15.152:6825/4238500329] [v2:172.21.15.152:6826/4238500329,v1:172.21.15.152:6827/4238500329] exists,up 310462a3-9997-49ff-ab18-0a1fbc14518d 2023-10-18T12:25:58.473 INFO:teuthology.orchestra.run.smithi070.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/1072168944,v1:172.21.15.186:6825/1072168944] [v2:172.21.15.186:6826/1072168944,v1:172.21.15.186:6827/1072168944] exists,up 3695b252-80cc-489f-a5d5-48f7ce64a1f2 2023-10-18T12:25:58.473 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 2.0 [7,11] 2023-10-18T12:25:58.474 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 2.e [2,4] 2023-10-18T12:25:58.474 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 2.f [8,11] 2023-10-18T12:25:58.474 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 2.10 [2,1] 2023-10-18T12:25:58.474 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 3.13 [4,9] 2023-10-18T12:25:58.474 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 3.14 [4,6] 2023-10-18T12:25:58.475 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 4.6 [9,11] 2023-10-18T12:25:58.475 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 4.e [9,1] 2023-10-18T12:25:58.475 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 4.f [9,8] 2023-10-18T12:25:58.475 INFO:teuthology.orchestra.run.smithi070.stdout:pg_upmap_items 4.1d [5,8] 2023-10-18T12:25:58.475 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/1564492170 expires 2023-10-19T12:18:13.515343+0000 2023-10-18T12:25:58.476 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6800/2836082256 expires 2023-10-19T12:18:13.515343+0000 2023-10-18T12:25:58.476 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/392897085 expires 2023-10-19T12:18:13.515343+0000 2023-10-18T12:25:58.476 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6801/1990317867 expires 2023-10-19T12:17:02.382050+0000 2023-10-18T12:25:58.476 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/2942646472 expires 2023-10-19T12:18:13.515343+0000 2023-10-18T12:25:58.477 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6800/1990317867 expires 2023-10-19T12:17:02.382050+0000 2023-10-18T12:25:58.477 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/771435827 expires 2023-10-19T12:17:02.382050+0000 2023-10-18T12:25:58.477 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6800/1853570886 expires 2023-10-19T12:16:35.958063+0000 2023-10-18T12:25:58.477 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/4154455091 expires 2023-10-19T12:16:35.958063+0000 2023-10-18T12:25:58.477 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6801/1853570886 expires 2023-10-19T12:16:35.958063+0000 2023-10-18T12:25:58.478 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/4120653611 expires 2023-10-19T12:17:02.382050+0000 2023-10-18T12:25:58.478 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/678282316 expires 2023-10-19T12:16:35.958063+0000 2023-10-18T12:25:58.478 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/3661620656 expires 2023-10-19T12:16:35.958063+0000 2023-10-18T12:25:58.478 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6801/2836082256 expires 2023-10-19T12:18:13.515343+0000 2023-10-18T12:25:58.479 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/477422870 expires 2023-10-19T12:17:02.382050+0000 2023-10-18T12:25:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb740a14d0 msgr2=0x7fdb740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb740a14d0 0x7fdb740a3990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c197c70 tx=0x7fdb8000b410 comp rx=0 tx=0).stop 2023-10-18T12:25:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 msgr2=0x7fdb8c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c19c970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c003730 tx=0x7fdb7c00f690 comp rx=0 tx=0).stop 2023-10-18T12:25:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 shutdown_connections 2023-10-18T12:25:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb8c1057c0 0x7fdb8c196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8c1008a0 0x7fdb8c19c970 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb740a14d0 0x7fdb740a3990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 --2- 172.21.15.70:0/2936142713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb8c101260 0x7fdb8c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 >> 172.21.15.70:0/2936142713 conn(0x7fdb8c0779e0 msgr2=0x7fdb8c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 shutdown_connections 2023-10-18T12:25:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:58.466+0000 7fdb91304700 1 -- 172.21.15.70:0/2936142713 wait complete. 2023-10-18T12:25:58.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:58 smithi070 ceph-mon[98639]: pgmap v311: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:58.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:25:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2936142713' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-18T12:25:58.918 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-18T12:25:58.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:25:58.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:58 smithi186 ceph-mon[135253]: pgmap v311: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:58.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:25:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2936142713' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-18T12:25:59.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 -- 172.21.15.70:0/791215446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 msgr2=0x7f4fc0106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/791215446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0106a90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009990 tx=0x7f4fb0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 -- 172.21.15.70:0/791215446 shutdown_connections 2023-10-18T12:25:59.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/791215446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fc0101060 0x7f4fc010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/791215446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0106a90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/791215446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc01006a0 0x7f4fc0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 -- 172.21.15.70:0/791215446 >> 172.21.15.70:0/791215446 conn(0x7f4fc00fc170 msgr2=0x7f4fc00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:59.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 -- 172.21.15.70:0/791215446 shutdown_connections 2023-10-18T12:25:59.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.129+0000 7f4fc53ff700 1 -- 172.21.15.70:0/791215446 wait complete. 2023-10-18T12:25:59.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.130+0000 7f4fc53ff700 1 Processor -- start 2023-10-18T12:25:59.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.130+0000 7f4fc53ff700 1 -- start start 2023-10-18T12:25:59.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fc53ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc01006a0 0x7f4fc019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fc53ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fc0101060 0x7f4fc019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fc53ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fc53ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0199110 con 0x7f4fc01066b0 2023-10-18T12:25:59.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fc53ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0199280 con 0x7f4fc0101060 2023-10-18T12:25:59.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fc53ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0199440 con 0x7f4fc01006a0 2023-10-18T12:25:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fbf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fbeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc01006a0 0x7f4fc019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fbf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49298/0 (socket says 172.21.15.70:49298) 2023-10-18T12:25:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fbf7fe700 1 -- 172.21.15.70:0/2981702593 learned_addr learned my addr 172.21.15.70:0/2981702593 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.131+0000 7f4fb7fff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fc0101060 0x7f4fc019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbf7fe700 1 -- 172.21.15.70:0/2981702593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc01006a0 msgr2=0x7f4fc019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbf7fe700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc01006a0 0x7f4fc019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbf7fe700 1 -- 172.21.15.70:0/2981702593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fc0101060 msgr2=0x7f4fc019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbf7fe700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fc0101060 0x7f4fc019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbf7fe700 1 -- 172.21.15.70:0/2981702593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb00095f0 con 0x7f4fc01066b0 2023-10-18T12:25:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbf7fe700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0198ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800ef30 tx=0x7f4fb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018070 con 0x7f4fc01066b0 2023-10-18T12:25:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb800f040 con 0x7f4fc01066b0 2023-10-18T12:25:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc01996c0 con 0x7f4fc01066b0 2023-10-18T12:25:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800cd00 con 0x7f4fc01066b0 2023-10-18T12:25:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.132+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc01a5970 con 0x7f4fc01066b0 2023-10-18T12:25:59.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.133+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc0101de0 con 0x7f4fc01066b0 2023-10-18T12:25:59.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.135+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4fb8003a00 con 0x7f4fc01066b0 2023-10-18T12:25:59.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.137+0000 7f4fbcff9700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4fac0a1550 0x7f4fac0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.137+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f4fb80ccc30 con 0x7f4fc01066b0 2023-10-18T12:25:59.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.137+0000 7f4fbeffd700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4fac0a1550 0x7f4fac0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.137+0000 7f4fbeffd700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4fac0a1550 0x7f4fac0a3a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000b2e0 tx=0x7f4fb0005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:59.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.140+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4fb8095ed0 con 0x7f4fc01066b0 2023-10-18T12:25:59.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:58 smithi152 ceph-mon[134767]: pgmap v311: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-18T12:25:59.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:25:58 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2936142713' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-18T12:25:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.342+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fc0067530 con 0x7f4fc01066b0 2023-10-18T12:25:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.344+0000 7f4fbcff9700 1 -- 172.21.15.70:0/2981702593 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f4fb80994f0 con 0x7f4fc01066b0 2023-10-18T12:25:59.347 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:25:59.348 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:25:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4fac0a1550 msgr2=0x7f4fac0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4fac0a1550 0x7f4fac0a3a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000b2e0 tx=0x7f4fb0005960 comp rx=0 tx=0).stop 2023-10-18T12:25:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 msgr2=0x7f4fc0198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0198ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800ef30 tx=0x7f4fb800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 shutdown_connections 2023-10-18T12:25:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc01066b0 0x7f4fc0198ab0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc01006a0 0x7f4fc019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4fac0a1550 0x7f4fac0a3a10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 --2- 172.21.15.70:0/2981702593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fc0101060 0x7f4fc019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 >> 172.21.15.70:0/2981702593 conn(0x7f4fc00fc170 msgr2=0x7f4fc010be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 shutdown_connections 2023-10-18T12:25:59.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.347+0000 7f4fc53ff700 1 -- 172.21.15.70:0/2981702593 wait complete. 2023-10-18T12:25:59.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:25:59.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:25:59.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 -- 172.21.15.70:0/894291982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 msgr2=0x7f2138106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 --2- 172.21.15.70:0/894291982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 0x7f2138106b10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f21280098f0 tx=0x7f2128009c00 comp rx=0 tx=0).stop 2023-10-18T12:25:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 -- 172.21.15.70:0/894291982 shutdown_connections 2023-10-18T12:25:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 --2- 172.21.15.70:0/894291982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f213810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 --2- 172.21.15.70:0/894291982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 0x7f2138106b10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 --2- 172.21.15.70:0/894291982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2138100720 0x7f2138100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 -- 172.21.15.70:0/894291982 >> 172.21.15.70:0/894291982 conn(0x7f21380fc1b0 msgr2=0x7f21380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 -- 172.21.15.70:0/894291982 shutdown_connections 2023-10-18T12:25:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.570+0000 7f213cdff700 1 -- 172.21.15.70:0/894291982 wait complete. 2023-10-18T12:25:59.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.571+0000 7f213cdff700 1 Processor -- start 2023-10-18T12:25:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 -- start start 2023-10-18T12:25:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2138100720 0x7f2138198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f2138199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 0x7f213819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213819dcf0 con 0x7f21381010e0 2023-10-18T12:25:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213819de60 con 0x7f2138100720 2023-10-18T12:25:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213cdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213819e020 con 0x7f2138106730 2023-10-18T12:25:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f2135d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f2138199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f2135d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f2138199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49310/0 (socket says 172.21.15.70:49310) 2023-10-18T12:25:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f2135d9b700 1 -- 172.21.15.70:0/3520781487 learned_addr learned my addr 172.21.15.70:0/3520781487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:25:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f2136d9d700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 0x7f213819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.572+0000 7f213659c700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2138100720 0x7f2138198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f2135d9b700 1 -- 172.21.15.70:0/3520781487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 msgr2=0x7f213819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f2135d9b700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 0x7f213819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f2135d9b700 1 -- 172.21.15.70:0/3520781487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2138100720 msgr2=0x7f2138198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f2135d9b700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2138100720 0x7f2138198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f2135d9b700 1 -- 172.21.15.70:0/3520781487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130009d00 con 0x7f21381010e0 2023-10-18T12:25:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f2135d9b700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f2138199360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f212c00e9a0 tx=0x7f212c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c009790 con 0x7f21381010e0 2023-10-18T12:25:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212c010430 con 0x7f21381010e0 2023-10-18T12:25:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2128009520 con 0x7f21381010e0 2023-10-18T12:25:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c018680 con 0x7f21381010e0 2023-10-18T12:25:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.573+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21381a5960 con 0x7f21381010e0 2023-10-18T12:25:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.574+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2138101e60 con 0x7f21381010e0 2023-10-18T12:25:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.576+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f212c0105a0 con 0x7f21381010e0 2023-10-18T12:25:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.577+0000 7f21277fe700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f21200a14d0 0x7f21200a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:25:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.578+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f212c014070 con 0x7f21381010e0 2023-10-18T12:25:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.578+0000 7f213659c700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f21200a14d0 0x7f21200a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:25:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.578+0000 7f213659c700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f21200a14d0 0x7f21200a3990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f21280098c0 tx=0x7f2128019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:25:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.580+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f212c0967a0 con 0x7f21381010e0 2023-10-18T12:25:59.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.781+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21380675a0 con 0x7f21381010e0 2023-10-18T12:25:59.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.781+0000 7f21277fe700 1 -- 172.21.15.70:0/3520781487 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f212c0260a0 con 0x7f21381010e0 2023-10-18T12:25:59.784 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:25:59.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f21200a14d0 msgr2=0x7f21200a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f21200a14d0 0x7f21200a3990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f21280098c0 tx=0x7f2128019040 comp rx=0 tx=0).stop 2023-10-18T12:25:59.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 msgr2=0x7f2138199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:25:59.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f2138199360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f212c00e9a0 tx=0x7f212c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 shutdown_connections 2023-10-18T12:25:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2138106730 0x7f213819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f21200a14d0 0x7f21200a3990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2138100720 0x7f2138198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 --2- 172.21.15.70:0/3520781487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21381010e0 0x7f2138199360 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:25:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 >> 172.21.15.70:0/3520781487 conn(0x7f21380fc1b0 msgr2=0x7f21380fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:25:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 shutdown_connections 2023-10-18T12:25:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:25:59.784+0000 7f213cdff700 1 -- 172.21.15.70:0/3520781487 wait complete. 2023-10-18T12:25:59.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:26:00.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 -- 172.21.15.70:0/1387783115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 msgr2=0x7f494c106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 --2- 172.21.15.70:0/1387783115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c106b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f49480098f0 tx=0x7f4948009c00 comp rx=0 tx=0).stop 2023-10-18T12:26:00.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 -- 172.21.15.70:0/1387783115 shutdown_connections 2023-10-18T12:26:00.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 --2- 172.21.15.70:0/1387783115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f494c1010f0 0x7f494c10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 --2- 172.21.15.70:0/1387783115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c106b20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 --2- 172.21.15.70:0/1387783115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c100730 0x7f494c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 -- 172.21.15.70:0/1387783115 >> 172.21.15.70:0/1387783115 conn(0x7f494c0fc200 msgr2=0x7f494c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 -- 172.21.15.70:0/1387783115 shutdown_connections 2023-10-18T12:26:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.004+0000 7f49537ff700 1 -- 172.21.15.70:0/1387783115 wait complete. 2023-10-18T12:26:00.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.005+0000 7f49537ff700 1 Processor -- start 2023-10-18T12:26:00.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.005+0000 7f49537ff700 1 -- start start 2023-10-18T12:26:00.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f49537ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f494c100730 0x7f494c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f49537ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 0x7f494c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f49537ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f49537ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19dcb0 con 0x7f494c1010f0 2023-10-18T12:26:00.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f49537ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19de20 con 0x7f494c106740 2023-10-18T12:26:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f49537ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c19dfe0 con 0x7f494c100730 2023-10-18T12:26:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 0x7f494c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 0x7f494c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49318/0 (socket says 172.21.15.70:49318) 2023-10-18T12:26:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 -- 172.21.15.70:0/4275202990 learned_addr learned my addr 172.21.15.70:0/4275202990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f495159b700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f494c100730 0x7f494c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4951d9c700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 -- 172.21.15.70:0/4275202990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f494c100730 msgr2=0x7f494c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f494c100730 0x7f494c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 -- 172.21.15.70:0/4275202990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 msgr2=0x7f494c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4950d9a700 1 -- 172.21.15.70:0/4275202990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4948009520 con 0x7f494c1010f0 2023-10-18T12:26:00.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.006+0000 7f4951d9c700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:00.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.007+0000 7f4950d9a700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 0x7f494c1992d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e9c0 tx=0x7f493c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:00.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.007+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00cbd0 con 0x7f494c1010f0 2023-10-18T12:26:00.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.007+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493c00eeb0 con 0x7f494c1010f0 2023-10-18T12:26:00.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.007+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c19e2c0 con 0x7f494c1010f0 2023-10-18T12:26:00.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.007+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c005580 con 0x7f494c1010f0 2023-10-18T12:26:00.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.007+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1a5b00 con 0x7f494c1010f0 2023-10-18T12:26:00.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.008+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c101e70 con 0x7f494c1010f0 2023-10-18T12:26:00.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.010+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f493c004750 con 0x7f494c1010f0 2023-10-18T12:26:00.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.011+0000 7f49427fc700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f49380a14d0 0x7f49380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.011+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f493c014070 con 0x7f494c1010f0 2023-10-18T12:26:00.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.011+0000 7f495159b700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f49380a14d0 0x7f49380a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.012+0000 7f495159b700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f49380a14d0 0x7f49380a3990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f49480098c0 tx=0x7f4948019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:00.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.015+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f493c096740 con 0x7f494c1010f0 2023-10-18T12:26:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.217+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f494c1106a0 con 0x7f494c1010f0 2023-10-18T12:26:00.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.218+0000 7f49427fc700 1 -- 172.21.15.70:0/4275202990 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f493c099d60 con 0x7f494c1010f0 2023-10-18T12:26:00.221 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:00.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f49380a14d0 msgr2=0x7f49380a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f49380a14d0 0x7f49380a3990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f49480098c0 tx=0x7f4948019040 comp rx=0 tx=0).stop 2023-10-18T12:26:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 msgr2=0x7f494c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 0x7f494c1992d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e9c0 tx=0x7f493c00ecd0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 shutdown_connections 2023-10-18T12:26:00.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f49380a14d0 0x7f49380a3990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f494c106740 0x7f494c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f494c100730 0x7f494c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 --2- 172.21.15.70:0/4275202990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f494c1010f0 0x7f494c1992d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 >> 172.21.15.70:0/4275202990 conn(0x7f494c0fc200 msgr2=0x7f494c10bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:00.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 shutdown_connections 2023-10-18T12:26:00.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.221+0000 7f49537ff700 1 -- 172.21.15.70:0/4275202990 wait complete. 2023-10-18T12:26:00.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:26:00.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:00 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2981702593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:00.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:00 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3520781487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 -- 172.21.15.70:0/2780804941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa538100720 msgr2=0x7fa538100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 --2- 172.21.15.70:0/2780804941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa538100720 0x7fa538100ba0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa524009990 tx=0x7fa524009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 -- 172.21.15.70:0/2780804941 shutdown_connections 2023-10-18T12:26:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 --2- 172.21.15.70:0/2780804941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5381010e0 0x7fa53810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 --2- 172.21.15.70:0/2780804941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa538106730 0x7fa538106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 --2- 172.21.15.70:0/2780804941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa538100720 0x7fa538100ba0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 -- 172.21.15.70:0/2780804941 >> 172.21.15.70:0/2780804941 conn(0x7fa5380fc1b0 msgr2=0x7fa5380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 -- 172.21.15.70:0/2780804941 shutdown_connections 2023-10-18T12:26:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.440+0000 7fa53ebff700 1 -- 172.21.15.70:0/2780804941 wait complete. 2023-10-18T12:26:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.441+0000 7fa53ebff700 1 Processor -- start 2023-10-18T12:26:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.441+0000 7fa53ebff700 1 -- start start 2023-10-18T12:26:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53ebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa538100720 0x7fa53819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53ebff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 0x7fa53819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53ebff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa538106730 0x7fa5381969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53ebff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538197050 con 0x7fa5381010e0 2023-10-18T12:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53ebff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5381971c0 con 0x7fa538106730 2023-10-18T12:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53ebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538197380 con 0x7fa538100720 2023-10-18T12:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 0x7fa53819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 0x7fa53819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49332/0 (socket says 172.21.15.70:49332) 2023-10-18T12:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53d19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa538106730 0x7fa5381969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 -- 172.21.15.70:0/1873064082 learned_addr learned my addr 172.21.15.70:0/1873064082 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53c99b700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa538100720 0x7fa53819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 -- 172.21.15.70:0/1873064082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa538100720 msgr2=0x7fa53819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa538100720 0x7fa53819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 -- 172.21.15.70:0/1873064082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa538106730 msgr2=0x7fa5381969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa538106730 0x7fa5381969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 -- 172.21.15.70:0/1873064082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240095f0 con 0x7fa5381010e0 2023-10-18T12:26:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa53d19c700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa538106730 0x7fa5381969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.442+0000 7fa537fff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 0x7fa53819cdb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa524005910 tx=0x7fa524004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.443+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52401d070 con 0x7fa5381010e0 2023-10-18T12:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.443+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa538197600 con 0x7fa5381010e0 2023-10-18T12:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.443+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa524022470 con 0x7fa5381010e0 2023-10-18T12:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.443+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524017650 con 0x7fa5381010e0 2023-10-18T12:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.443+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5381a3860 con 0x7fa5381010e0 2023-10-18T12:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.444+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa538108d40 con 0x7fa5381010e0 2023-10-18T12:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.446+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa5240225e0 con 0x7fa5381010e0 2023-10-18T12:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.447+0000 7fa535ffb700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa5280a14d0 0x7fa5280a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.447+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fa5240ce470 con 0x7fa5381010e0 2023-10-18T12:26:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.447+0000 7fa53c99b700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa5280a14d0 0x7fa5280a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.448+0000 7fa53c99b700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa5280a14d0 0x7fa5280a3990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009bd0 tx=0x7fa52c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.451+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa524097770 con 0x7fa5381010e0 2023-10-18T12:26:00.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:00 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2981702593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:00.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:00 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3520781487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:00 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2981702593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:00.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:00 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3520781487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:00.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.656+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa538197e60 con 0x7fa5381010e0 2023-10-18T12:26:00.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.656+0000 7fa535ffb700 1 -- 172.21.15.70:0/1873064082 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa52409ad90 con 0x7fa5381010e0 2023-10-18T12:26:00.660 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:26:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa5280a14d0 msgr2=0x7fa5280a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa5280a14d0 0x7fa5280a3990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009bd0 tx=0x7fa52c009260 comp rx=0 tx=0).stop 2023-10-18T12:26:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 msgr2=0x7fa53819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 0x7fa53819cdb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa524005910 tx=0x7fa524004730 comp rx=0 tx=0).stop 2023-10-18T12:26:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 shutdown_connections 2023-10-18T12:26:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa5280a14d0 0x7fa5280a3990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa538106730 0x7fa5381969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa538100720 0x7fa53819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 --2- 172.21.15.70:0/1873064082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5381010e0 0x7fa53819cdb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 >> 172.21.15.70:0/1873064082 conn(0x7fa5380fc1b0 msgr2=0x7fa5380fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:00.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 shutdown_connections 2023-10-18T12:26:00.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.659+0000 7fa53ebff700 1 -- 172.21.15.70:0/1873064082 wait complete. 2023-10-18T12:26:00.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:26:00.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 -- 172.21.15.70:0/2838217619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 msgr2=0x7fc21c106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2838217619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 0x7fc21c106b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009990 tx=0x7fc20c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 -- 172.21.15.70:0/2838217619 shutdown_connections 2023-10-18T12:26:00.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2838217619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c1010d0 0x7fc21c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2838217619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 0x7fc21c106b00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2838217619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21c100710 0x7fc21c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 -- 172.21.15.70:0/2838217619 >> 172.21.15.70:0/2838217619 conn(0x7fc21c0fc200 msgr2=0x7fc21c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:00.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 -- 172.21.15.70:0/2838217619 shutdown_connections 2023-10-18T12:26:00.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.879+0000 7fc2209ff700 1 -- 172.21.15.70:0/2838217619 wait complete. 2023-10-18T12:26:00.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 Processor -- start 2023-10-18T12:26:00.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 -- start start 2023-10-18T12:26:00.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 0x7fc21c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21c1010d0 0x7fc21c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 0x7fc21c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1a4ed0 con 0x7fc21c106720 2023-10-18T12:26:00.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1a5040 con 0x7fc21c100710 2023-10-18T12:26:00.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.881+0000 7fc2209ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1a5200 con 0x7fc21c1010d0 2023-10-18T12:26:00.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 0x7fc21c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 0x7fc21c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 0x7fc21c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:33478/0 (socket says 172.21.15.70:33478) 2023-10-18T12:26:00.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 -- 172.21.15.70:0/2571180226 learned_addr learned my addr 172.21.15.70:0/2571180226 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:00.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc219d9b700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21c1010d0 0x7fc21c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 -- 172.21.15.70:0/2571180226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21c1010d0 msgr2=0x7fc21c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21c1010d0 0x7fc21c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 -- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 msgr2=0x7fc21c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:00.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 0x7fc21c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:00.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 -- 172.21.15.70:0/2571180226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0095f0 con 0x7fc21c100710 2023-10-18T12:26:00.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc21a59c700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 0x7fc21c072af0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc20c005770 tx=0x7fc20c0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:00.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c01d070 con 0x7fc21c100710 2023-10-18T12:26:00.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c1a54e0 con 0x7fc21c100710 2023-10-18T12:26:00.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc20c021410 con 0x7fc21c100710 2023-10-18T12:26:00.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.882+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c017650 con 0x7fc21c100710 2023-10-18T12:26:00.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.883+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c1a59f0 con 0x7fc21c100710 2023-10-18T12:26:00.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.883+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c108d10 con 0x7fc21c100710 2023-10-18T12:26:00.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.886+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc20c0217f0 con 0x7fc21c100710 2023-10-18T12:26:00.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.887+0000 7fc20b7fe700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc2040a14d0 0x7fc2040a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:00.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.887+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fc20c0cec10 con 0x7fc21c100710 2023-10-18T12:26:00.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.887+0000 7fc219d9b700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc2040a14d0 0x7fc2040a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:00.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.888+0000 7fc219d9b700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc2040a14d0 0x7fc2040a3990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc210005950 tx=0x7fc2100058e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:00.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:00.889+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc20c097eb0 con 0x7fc21c100710 2023-10-18T12:26:01.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.092+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc21c073f80 con 0x7fc21c100710 2023-10-18T12:26:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.092+0000 7fc20b7fe700 1 -- 172.21.15.70:0/2571180226 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc20c09b4d0 con 0x7fc21c100710 2023-10-18T12:26:01.094 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc2040a14d0 msgr2=0x7fc2040a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc2040a14d0 0x7fc2040a3990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc210005950 tx=0x7fc2100058e0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 msgr2=0x7fc21c072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 0x7fc21c072af0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc20c005770 tx=0x7fc20c0057a0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 shutdown_connections 2023-10-18T12:26:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c106720 0x7fc21c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc2040a14d0 0x7fc2040a3990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c100710 0x7fc21c072af0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 --2- 172.21.15.70:0/2571180226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc21c1010d0 0x7fc21c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 >> 172.21.15.70:0/2571180226 conn(0x7fc21c0fc200 msgr2=0x7fc21c0fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 shutdown_connections 2023-10-18T12:26:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.095+0000 7fc2209ff700 1 -- 172.21.15.70:0/2571180226 wait complete. 2023-10-18T12:26:01.116 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:26:01.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 -- 172.21.15.70:0/2296508165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c072c20 msgr2=0x7fd12c073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 --2- 172.21.15.70:0/2296508165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c072c20 0x7fd12c073000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd120009990 tx=0x7fd120009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 -- 172.21.15.70:0/2296508165 shutdown_connections 2023-10-18T12:26:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 --2- 172.21.15.70:0/2296508165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c10cee0 0x7fd12c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 --2- 172.21.15.70:0/2296508165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c072c20 0x7fd12c073000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 --2- 172.21.15.70:0/2296508165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 0x7fd12c10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 -- 172.21.15.70:0/2296508165 >> 172.21.15.70:0/2296508165 conn(0x7fd12c0fc170 msgr2=0x7fd12c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 -- 172.21.15.70:0/2296508165 shutdown_connections 2023-10-18T12:26:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.313+0000 7fd132bff700 1 -- 172.21.15.70:0/2296508165 wait complete. 2023-10-18T12:26:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 Processor -- start 2023-10-18T12:26:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 -- start start 2023-10-18T12:26:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 0x7fd12c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 0x7fd12c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c10cee0 0x7fd12c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19dc40 con 0x7fd12c10cee0 2023-10-18T12:26:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19ddb0 con 0x7fd12c072c20 2023-10-18T12:26:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.314+0000 7fd132bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c19df70 con 0x7fd12c073540 2023-10-18T12:26:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13099b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 0x7fd12c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c10cee0 0x7fd12c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13099b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 0x7fd12c198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:33500/0 (socket says 172.21.15.70:33500) 2023-10-18T12:26:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13099b700 1 -- 172.21.15.70:0/3064757419 learned_addr learned my addr 172.21.15.70:0/3064757419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd12bfff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 0x7fd12c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 -- 172.21.15.70:0/3064757419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 msgr2=0x7fd12c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 0x7fd12c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 -- 172.21.15.70:0/3064757419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 msgr2=0x7fd12c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 0x7fd12c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 -- 172.21.15.70:0/3064757419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200095f0 con 0x7fd12c10cee0 2023-10-18T12:26:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13099b700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 0x7fd12c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd12bfff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 0x7fd12c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.315+0000 7fd13119c700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c10cee0 0x7fd12c19d600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd12400efc0 tx=0x7fd12400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.316+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124018070 con 0x7fd12c10cee0 2023-10-18T12:26:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.316+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd12400f040 con 0x7fd12c10cee0 2023-10-18T12:26:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.316+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c19e1f0 con 0x7fd12c10cee0 2023-10-18T12:26:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.316+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400cdf0 con 0x7fd12c10cee0 2023-10-18T12:26:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.316+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c1a5a90 con 0x7fd12c10cee0 2023-10-18T12:26:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.317+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c1080f0 con 0x7fd12c10cee0 2023-10-18T12:26:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.318+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd124003a00 con 0x7fd12c10cee0 2023-10-18T12:26:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.320+0000 7fd129ffb700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd11c0a1550 0x7fd11c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.320+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fd1240ccd60 con 0x7fd12c10cee0 2023-10-18T12:26:01.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.320+0000 7fd13099b700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd11c0a1550 0x7fd11c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.321+0000 7fd13099b700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd11c0a1550 0x7fd11c0a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd120006010 tx=0x7fd12000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:01.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.324+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd124096000 con 0x7fd12c10cee0 2023-10-18T12:26:01.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:01 smithi152 ceph-mon[134767]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2023-10-18T12:26:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:01 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4275202990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:01 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1873064082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:01 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2571180226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:01 smithi186 ceph-mon[135253]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2023-10-18T12:26:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:01 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4275202990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:01 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1873064082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:01 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2571180226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.567+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd12c19a090 con 0x7fd12c10cee0 2023-10-18T12:26:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.568+0000 7fd129ffb700 1 -- 172.21.15.70:0/3064757419 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fd124099620 con 0x7fd12c10cee0 2023-10-18T12:26:01.570 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:01.570 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:26:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd11c0a1550 msgr2=0x7fd11c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd11c0a1550 0x7fd11c0a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd120006010 tx=0x7fd12000b260 comp rx=0 tx=0).stop 2023-10-18T12:26:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c10cee0 msgr2=0x7fd12c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c10cee0 0x7fd12c19d600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd12400efc0 tx=0x7fd12400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 shutdown_connections 2023-10-18T12:26:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c10cee0 0x7fd12c19d600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd11c0a1550 0x7fd11c0a3a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd12c072c20 0x7fd12c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 --2- 172.21.15.70:0/3064757419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c073540 0x7fd12c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 >> 172.21.15.70:0/3064757419 conn(0x7fd12c0fc170 msgr2=0x7fd12c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 shutdown_connections 2023-10-18T12:26:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.570+0000 7fd132bff700 1 -- 172.21.15.70:0/3064757419 wait complete. 2023-10-18T12:26:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:26:01.588 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:01 smithi070 ceph-mon[98639]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2023-10-18T12:26:01.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:01 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4275202990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:01 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1873064082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:01 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2571180226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:01.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:26:01.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 -- 172.21.15.70:0/2665539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 msgr2=0x7f87c410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 --2- 172.21.15.70:0/2665539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c410d570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00b0a0 tx=0x7f87bc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 -- 172.21.15.70:0/2665539591 shutdown_connections 2023-10-18T12:26:01.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 --2- 172.21.15.70:0/2665539591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c410d570 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 --2- 172.21.15.70:0/2665539591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c41066b0 0x7f87c4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 --2- 172.21.15.70:0/2665539591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 -- 172.21.15.70:0/2665539591 >> 172.21.15.70:0/2665539591 conn(0x7f87c40fc170 msgr2=0x7f87c40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 -- 172.21.15.70:0/2665539591 shutdown_connections 2023-10-18T12:26:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.788+0000 7f87c8dff700 1 -- 172.21.15.70:0/2665539591 wait complete. 2023-10-18T12:26:01.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 Processor -- start 2023-10-18T12:26:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 -- start start 2023-10-18T12:26:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c41066b0 0x7f87c4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c4199270 con 0x7f87c4101060 2023-10-18T12:26:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41993e0 con 0x7f87c41066b0 2023-10-18T12:26:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.790+0000 7f87c8dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41995a0 con 0x7f87c41006a0 2023-10-18T12:26:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87bbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41990/0 (socket says 172.21.15.70:41990) 2023-10-18T12:26:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 -- 172.21.15.70:0/1986677648 learned_addr learned my addr 172.21.15.70:0/1986677648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c2d9d700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c41066b0 0x7f87c4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 -- 172.21.15.70:0/1986677648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c41066b0 msgr2=0x7f87c4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c41066b0 0x7f87c4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 -- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 msgr2=0x7f87c419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 -- 172.21.15.70:0/1986677648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009520 con 0x7f87c41006a0 2023-10-18T12:26:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87bbfff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.791+0000 7f87c259c700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c419ea00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f87b400eb30 tx=0x7f87b400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.792+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4009880 con 0x7f87c41006a0 2023-10-18T12:26:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.792+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc009d00 con 0x7f87c41006a0 2023-10-18T12:26:01.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.792+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b40044e0 con 0x7f87c41006a0 2023-10-18T12:26:01.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.792+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4003ce0 con 0x7f87c41006a0 2023-10-18T12:26:01.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.792+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c41a5980 con 0x7f87c41006a0 2023-10-18T12:26:01.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.793+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c4101de0 con 0x7f87c41006a0 2023-10-18T12:26:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.795+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f87b4004020 con 0x7f87c41006a0 2023-10-18T12:26:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.796+0000 7f87bb7fe700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87b00a1550 0x7f87b00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.796+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f87b4014070 con 0x7f87c41006a0 2023-10-18T12:26:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.797+0000 7f87bbfff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87b00a1550 0x7f87b00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:01.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.797+0000 7f87bbfff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87b00a1550 0x7f87b00a3a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87c419a210 tx=0x7f87ac009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:01.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:01.799+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87b40968a0 con 0x7f87c41006a0 2023-10-18T12:26:02.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.005+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87c4199f80 con 0x7f87c41006a0 2023-10-18T12:26:02.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.007+0000 7f87bb7fe700 1 -- 172.21.15.70:0/1986677648 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f87b4096020 con 0x7f87c41006a0 2023-10-18T12:26:02.009 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:02.009 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:26:02.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.009+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87b00a1550 msgr2=0x7f87b00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.009+0000 7f87c8dff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87b00a1550 0x7f87b00a3a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87c419a210 tx=0x7f87ac009260 comp rx=0 tx=0).stop 2023-10-18T12:26:02.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.009+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 msgr2=0x7f87c419ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.009+0000 7f87c8dff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c419ea00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f87b400eb30 tx=0x7f87b400ee40 comp rx=0 tx=0).stop 2023-10-18T12:26:02.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 shutdown_connections 2023-10-18T12:26:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f87b00a1550 0x7f87b00a3a10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87c41066b0 0x7f87c4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87c41006a0 0x7f87c419ea00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 --2- 172.21.15.70:0/1986677648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87c4101060 0x7f87c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 >> 172.21.15.70:0/1986677648 conn(0x7f87c40fc170 msgr2=0x7f87c410bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:02.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 shutdown_connections 2023-10-18T12:26:02.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.010+0000 7f87c8dff700 1 -- 172.21.15.70:0/1986677648 wait complete. 2023-10-18T12:26:02.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:26:02.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:26:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 -- 172.21.15.70:0/2586284964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240106730 msgr2=0x7f4240106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 --2- 172.21.15.70:0/2586284964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240106730 0x7f4240106b10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f423c009990 tx=0x7f423c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 -- 172.21.15.70:0/2586284964 shutdown_connections 2023-10-18T12:26:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 --2- 172.21.15.70:0/2586284964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42401010e0 0x7f424010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 --2- 172.21.15.70:0/2586284964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240106730 0x7f4240106b10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 --2- 172.21.15.70:0/2586284964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4240100720 0x7f4240100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 -- 172.21.15.70:0/2586284964 >> 172.21.15.70:0/2586284964 conn(0x7f42400fc1b0 msgr2=0x7f42400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 -- 172.21.15.70:0/2586284964 shutdown_connections 2023-10-18T12:26:02.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.233+0000 7f42483ff700 1 -- 172.21.15.70:0/2586284964 wait complete. 2023-10-18T12:26:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 Processor -- start 2023-10-18T12:26:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 -- start start 2023-10-18T12:26:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 0x7f4240072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42401010e0 0x7f4240073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4240106730 0x7f4240073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a4f30 con 0x7f4240100720 2023-10-18T12:26:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a50a0 con 0x7f42401010e0 2023-10-18T12:26:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.234+0000 7f42483ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a5260 con 0x7f4240106730 2023-10-18T12:26:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 0x7f4240072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 0x7f4240072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49412/0 (socket says 172.21.15.70:49412) 2023-10-18T12:26:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 -- 172.21.15.70:0/384973 learned_addr learned my addr 172.21.15.70:0/384973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424699c700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4240106730 0x7f4240073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424599a700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42401010e0 0x7f4240073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 -- 172.21.15.70:0/384973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4240106730 msgr2=0x7f4240073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4240106730 0x7f4240073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 -- 172.21.15.70:0/384973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42401010e0 msgr2=0x7f4240073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42401010e0 0x7f4240073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 -- 172.21.15.70:0/384973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0095f0 con 0x7f4240100720 2023-10-18T12:26:02.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.235+0000 7f424619b700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 0x7f4240072af0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f423c0095c0 tx=0x7f423c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:02.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.236+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c01d070 con 0x7f4240100720 2023-10-18T12:26:02.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.236+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42401a54e0 con 0x7f4240100720 2023-10-18T12:26:02.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.236+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f423c005030 con 0x7f4240100720 2023-10-18T12:26:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.236+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c004a90 con 0x7f4240100720 2023-10-18T12:26:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.236+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42401a59c0 con 0x7f4240100720 2023-10-18T12:26:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.237+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4240108d20 con 0x7f4240100720 2023-10-18T12:26:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.239+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f423c005410 con 0x7f4240100720 2023-10-18T12:26:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.241+0000 7f42377fe700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f422c0a14d0 0x7f422c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.241+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f423c0cdee0 con 0x7f4240100720 2023-10-18T12:26:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.241+0000 7f424599a700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f422c0a14d0 0x7f422c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.242+0000 7f424599a700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f422c0a14d0 0x7f422c0a3990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4240074220 tx=0x7f423000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:02.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.245+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f423c097180 con 0x7f4240100720 2023-10-18T12:26:02.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:02 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3064757419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:02.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:02 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1986677648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:02.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:02 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3064757419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:02.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:02 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1986677648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:02.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.443+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4240073fd0 con 0x7f4240100720 2023-10-18T12:26:02.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.444+0000 7f42377fe700 1 -- 172.21.15.70:0/384973 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f423c09a7a0 con 0x7f4240100720 2023-10-18T12:26:02.446 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:26:02.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f422c0a14d0 msgr2=0x7f422c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f422c0a14d0 0x7f422c0a3990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4240074220 tx=0x7f423000b410 comp rx=0 tx=0).stop 2023-10-18T12:26:02.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 msgr2=0x7f4240072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 0x7f4240072af0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f423c0095c0 tx=0x7f423c00f690 comp rx=0 tx=0).stop 2023-10-18T12:26:02.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 shutdown_connections 2023-10-18T12:26:02.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4240106730 0x7f4240073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4240100720 0x7f4240072af0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f422c0a14d0 0x7f422c0a3990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 --2- 172.21.15.70:0/384973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42401010e0 0x7f4240073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 >> 172.21.15.70:0/384973 conn(0x7f42400fc1b0 msgr2=0x7f42400fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:02.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 shutdown_connections 2023-10-18T12:26:02.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.446+0000 7f42483ff700 1 -- 172.21.15.70:0/384973 wait complete. 2023-10-18T12:26:02.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:26:02.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:02 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3064757419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:02.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:02 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1986677648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 -- 172.21.15.70:0/1743658406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c106730 msgr2=0x7f9e8c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 --2- 172.21.15.70:0/1743658406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c106730 0x7f9e8c106b10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009990 tx=0x7f9e88009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 -- 172.21.15.70:0/1743658406 shutdown_connections 2023-10-18T12:26:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 --2- 172.21.15.70:0/1743658406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c1010e0 0x7f9e8c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 --2- 172.21.15.70:0/1743658406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c106730 0x7f9e8c106b10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 --2- 172.21.15.70:0/1743658406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8c100720 0x7f9e8c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 -- 172.21.15.70:0/1743658406 >> 172.21.15.70:0/1743658406 conn(0x7f9e8c0fc1b0 msgr2=0x7f9e8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 -- 172.21.15.70:0/1743658406 shutdown_connections 2023-10-18T12:26:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.666+0000 7f9e945ff700 1 -- 172.21.15.70:0/1743658406 wait complete. 2023-10-18T12:26:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 Processor -- start 2023-10-18T12:26:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 -- start start 2023-10-18T12:26:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 0x7f9e8c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8c1010e0 0x7f9e8c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c106730 0x7f9e8c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19dcc0 con 0x7f9e8c100720 2023-10-18T12:26:02.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19de30 con 0x7f9e8c106730 2023-10-18T12:26:02.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.667+0000 7f9e945ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19dff0 con 0x7f9e8c1010e0 2023-10-18T12:26:02.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 0x7f9e8c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 0x7f9e8c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49436/0 (socket says 172.21.15.70:49436) 2023-10-18T12:26:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 -- 172.21.15.70:0/756020584 learned_addr learned my addr 172.21.15.70:0/756020584 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e92b9c700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c106730 0x7f9e8c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e91b9a700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8c1010e0 0x7f9e8c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 -- 172.21.15.70:0/756020584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8c1010e0 msgr2=0x7f9e8c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8c1010e0 0x7f9e8c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 -- 172.21.15.70:0/756020584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c106730 msgr2=0x7f9e8c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c106730 0x7f9e8c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 -- 172.21.15.70:0/756020584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880095f0 con 0x7f9e8c100720 2023-10-18T12:26:02.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e92b9c700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c106730 0x7f9e8c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:02.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e9239b700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 0x7f9e8c198da0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e88004730 tx=0x7f9e88004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:02.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8801d070 con 0x7f9e8c100720 2023-10-18T12:26:02.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e88022470 con 0x7f9e8c100720 2023-10-18T12:26:02.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c19e270 con 0x7f9e8c100720 2023-10-18T12:26:02.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.668+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88017650 con 0x7f9e8c100720 2023-10-18T12:26:02.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.669+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c1a5b10 con 0x7f9e8c100720 2023-10-18T12:26:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.670+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c101e60 con 0x7f9e8c100720 2023-10-18T12:26:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.671+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9e880225e0 con 0x7f9e8c100720 2023-10-18T12:26:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.673+0000 7f9e837fe700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e780a14d0 0x7f9e780a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.673+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9e880ceba0 con 0x7f9e8c100720 2023-10-18T12:26:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.673+0000 7f9e91b9a700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e780a14d0 0x7f9e780a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.674+0000 7f9e91b9a700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e780a14d0 0x7f9e780a3990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c19a2f0 tx=0x7f9e7c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.675+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e88097e40 con 0x7f9e8c100720 2023-10-18T12:26:02.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.878+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e8c19a140 con 0x7f9e8c100720 2023-10-18T12:26:02.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.879+0000 7f9e837fe700 1 -- 172.21.15.70:0/756020584 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e8809b460 con 0x7f9e8c100720 2023-10-18T12:26:02.881 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:02.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e780a14d0 msgr2=0x7f9e780a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e780a14d0 0x7f9e780a3990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c19a2f0 tx=0x7f9e7c009260 comp rx=0 tx=0).stop 2023-10-18T12:26:02.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 msgr2=0x7f9e8c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:02.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 0x7f9e8c198da0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e88004730 tx=0x7f9e88004760 comp rx=0 tx=0).stop 2023-10-18T12:26:02.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 shutdown_connections 2023-10-18T12:26:02.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e780a14d0 0x7f9e780a3990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e8c106730 0x7f9e8c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e8c100720 0x7f9e8c198da0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 --2- 172.21.15.70:0/756020584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8c1010e0 0x7f9e8c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:02.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 >> 172.21.15.70:0/756020584 conn(0x7f9e8c0fc1b0 msgr2=0x7f9e8c10be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:02.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 shutdown_connections 2023-10-18T12:26:02.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:02.881+0000 7f9e945ff700 1 -- 172.21.15.70:0/756020584 wait complete. 2023-10-18T12:26:02.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:26:03.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 -- 172.21.15.70:0/4261811724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 msgr2=0x7f3fa4100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/4261811724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 0x7f3fa4100ba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009990 tx=0x7f3f94009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 -- 172.21.15.70:0/4261811724 shutdown_connections 2023-10-18T12:26:03.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/4261811724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/4261811724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4106730 0x7f3fa4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/4261811724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 0x7f3fa4100ba0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 -- 172.21.15.70:0/4261811724 >> 172.21.15.70:0/4261811724 conn(0x7f3fa40fc1b0 msgr2=0x7f3fa40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:03.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 -- 172.21.15.70:0/4261811724 shutdown_connections 2023-10-18T12:26:03.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.101+0000 7f3fac1ff700 1 -- 172.21.15.70:0/4261811724 wait complete. 2023-10-18T12:26:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.102+0000 7f3fac1ff700 1 Processor -- start 2023-10-18T12:26:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.102+0000 7f3fac1ff700 1 -- start start 2023-10-18T12:26:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fac1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 0x7f3fa419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fac1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fac1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4106730 0x7f3fa41969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fac1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4197180 con 0x7f3fa41010e0 2023-10-18T12:26:03.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fac1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa41972f0 con 0x7f3fa4100720 2023-10-18T12:26:03.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fac1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa41974b0 con 0x7f3fa4106730 2023-10-18T12:26:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49448/0 (socket says 172.21.15.70:49448) 2023-10-18T12:26:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 -- 172.21.15.70:0/3082079818 learned_addr learned my addr 172.21.15.70:0/3082079818 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa9f9b700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 0x7f3fa419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3faa79c700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4106730 0x7f3fa41969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 -- 172.21.15.70:0/3082079818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4106730 msgr2=0x7f3fa41969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4106730 0x7f3fa41969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 -- 172.21.15.70:0/3082079818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 msgr2=0x7f3fa419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 0x7f3fa419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.103+0000 7f3fa979a700 1 -- 172.21.15.70:0/3082079818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940095f0 con 0x7f3fa41010e0 2023-10-18T12:26:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.104+0000 7f3fa979a700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa419ce50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3f94000c00 tx=0x7f3f94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.104+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9401d070 con 0x7f3fa41010e0 2023-10-18T12:26:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.104+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f94022470 con 0x7f3fa41010e0 2023-10-18T12:26:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.104+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa4197730 con 0x7f3fa41010e0 2023-10-18T12:26:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.104+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94017650 con 0x7f3fa41010e0 2023-10-18T12:26:03.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.104+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa41a3890 con 0x7f3fa41010e0 2023-10-18T12:26:03.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.105+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa4108d40 con 0x7f3fa41010e0 2023-10-18T12:26:03.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.107+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3f940225e0 con 0x7f3fa41010e0 2023-10-18T12:26:03.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.109+0000 7f3f9affd700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3f900a14d0 0x7f3f900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.109+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f3f940ce850 con 0x7f3fa41010e0 2023-10-18T12:26:03.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.110+0000 7f3fa9f9b700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3f900a14d0 0x7f3f900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.110+0000 7f3fa9f9b700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3f900a14d0 0x7f3f900a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009be0 tx=0x7f3fa0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.112+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f94097b50 con 0x7f3fa41010e0 2023-10-18T12:26:03.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.313+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3fa4197e90 con 0x7f3fa41010e0 2023-10-18T12:26:03.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.313+0000 7f3f9affd700 1 -- 172.21.15.70:0/3082079818 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f9409b170 con 0x7f3fa41010e0 2023-10-18T12:26:03.316 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:26:03.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3f900a14d0 msgr2=0x7f3f900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3f900a14d0 0x7f3f900a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009be0 tx=0x7f3fa0009260 comp rx=0 tx=0).stop 2023-10-18T12:26:03.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 msgr2=0x7f3fa419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa419ce50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3f94000c00 tx=0x7f3f94004730 comp rx=0 tx=0).stop 2023-10-18T12:26:03.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 shutdown_connections 2023-10-18T12:26:03.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fa4106730 0x7f3fa41969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3f900a14d0 0x7f3f900a3990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fa4100720 0x7f3fa419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 --2- 172.21.15.70:0/3082079818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa41010e0 0x7f3fa419ce50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 >> 172.21.15.70:0/3082079818 conn(0x7f3fa40fc1b0 msgr2=0x7f3fa40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:03.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 shutdown_connections 2023-10-18T12:26:03.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.316+0000 7f3fac1ff700 1 -- 172.21.15.70:0/3082079818 wait complete. 2023-10-18T12:26:03.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:26:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:03 smithi152 ceph-mon[134767]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2023-10-18T12:26:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:03 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/384973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:03.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:03 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/756020584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:03 smithi186 ceph-mon[135253]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2023-10-18T12:26:03.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:03 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/384973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:03.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:03 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/756020584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1402669697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e00734c0 msgr2=0x7fb3e010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1402669697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e00734c0 0x7fb3e010cab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009990 tx=0x7fb3cc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1402669697 shutdown_connections 2023-10-18T12:26:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1402669697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e010cff0 0x7fb3e010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1402669697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3e0072ba0 0x7fb3e0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1402669697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e00734c0 0x7fb3e010cab0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1402669697 >> 172.21.15.70:0/1402669697 conn(0x7fb3e00fc200 msgr2=0x7fb3e00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:03.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1402669697 shutdown_connections 2023-10-18T12:26:03.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.535+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1402669697 wait complete. 2023-10-18T12:26:03.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.536+0000 7fb3e6dff700 1 Processor -- start 2023-10-18T12:26:03.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.536+0000 7fb3e6dff700 1 -- start start 2023-10-18T12:26:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e6dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3e0072ba0 0x7fb3e0198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e6dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 0x7fb3e01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e6dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e010cff0 0x7fb3e019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e6dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e019dd40 con 0x7fb3e00734c0 2023-10-18T12:26:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e6dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e019deb0 con 0x7fb3e010cff0 2023-10-18T12:26:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e6dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e019e070 con 0x7fb3e0072ba0 2023-10-18T12:26:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3dffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 0x7fb3e01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3e0072ba0 0x7fb3e0198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e539c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e010cff0 0x7fb3e019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3dffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 0x7fb3e01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49468/0 (socket says 172.21.15.70:49468) 2023-10-18T12:26:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3dffff700 1 -- 172.21.15.70:0/1672549205 learned_addr learned my addr 172.21.15.70:0/1672549205 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e539c700 1 -- 172.21.15.70:0/1672549205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3e0072ba0 msgr2=0x7fb3e0198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e539c700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3e0072ba0 0x7fb3e0198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e539c700 1 -- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 msgr2=0x7fb3e01993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e539c700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 0x7fb3e01993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3e539c700 1 -- 172.21.15.70:0/1672549205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0095f0 con 0x7fb3e010cff0 2023-10-18T12:26:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.537+0000 7fb3dffff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 0x7fb3e01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.538+0000 7fb3e539c700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e010cff0 0x7fb3e019d700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800efc0 tx=0x7fb3d800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.538+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8018070 con 0x7fb3e010cff0 2023-10-18T12:26:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.538+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e019e2f0 con 0x7fb3e010cff0 2023-10-18T12:26:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.538+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3d800f040 con 0x7fb3e010cff0 2023-10-18T12:26:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.538+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d800cd40 con 0x7fb3e010cff0 2023-10-18T12:26:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.538+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e01a5b90 con 0x7fb3e010cff0 2023-10-18T12:26:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.539+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e0108200 con 0x7fb3e010cff0 2023-10-18T12:26:03.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.541+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb3d8003a00 con 0x7fb3e010cff0 2023-10-18T12:26:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.542+0000 7fb3ddffb700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb3d00a14d0 0x7fb3d00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.543+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fb3d80cd690 con 0x7fb3e010cff0 2023-10-18T12:26:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.543+0000 7fb3e4b9b700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb3d00a14d0 0x7fb3d00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.543+0000 7fb3e4b9b700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb3d00a14d0 0x7fb3d00a3990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009c60 tx=0x7fb3d4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.545+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb3d8096930 con 0x7fb3e010cff0 2023-10-18T12:26:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:03 smithi070 ceph-mon[98639]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2023-10-18T12:26:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:03 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/384973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:03 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/756020584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.751+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb3e019a120 con 0x7fb3e010cff0 2023-10-18T12:26:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.752+0000 7fb3ddffb700 1 -- 172.21.15.70:0/1672549205 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb3d8099f50 con 0x7fb3e010cff0 2023-10-18T12:26:03.754 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb3d00a14d0 msgr2=0x7fb3d00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb3d00a14d0 0x7fb3d00a3990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009c60 tx=0x7fb3d4009260 comp rx=0 tx=0).stop 2023-10-18T12:26:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e010cff0 msgr2=0x7fb3e019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e010cff0 0x7fb3e019d700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800efc0 tx=0x7fb3d800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 shutdown_connections 2023-10-18T12:26:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb3d00a14d0 0x7fb3d00a3990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e010cff0 0x7fb3e019d700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3e0072ba0 0x7fb3e0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 --2- 172.21.15.70:0/1672549205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00734c0 0x7fb3e01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 >> 172.21.15.70:0/1672549205 conn(0x7fb3e00fc200 msgr2=0x7fb3e00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 shutdown_connections 2023-10-18T12:26:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.755+0000 7fb3e6dff700 1 -- 172.21.15.70:0/1672549205 wait complete. 2023-10-18T12:26:03.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:26:03.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 -- 172.21.15.70:0/575015781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 msgr2=0x7f8c5010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 --2- 172.21.15.70:0/575015781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5010d570 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b0a0 tx=0x7f8c4800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 -- 172.21.15.70:0/575015781 shutdown_connections 2023-10-18T12:26:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 --2- 172.21.15.70:0/575015781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5010d570 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 --2- 172.21.15.70:0/575015781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c501066b0 0x7f8c50106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 --2- 172.21.15.70:0/575015781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c501006a0 0x7f8c50100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 -- 172.21.15.70:0/575015781 >> 172.21.15.70:0/575015781 conn(0x7f8c500fc170 msgr2=0x7f8c500fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:03.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 -- 172.21.15.70:0/575015781 shutdown_connections 2023-10-18T12:26:03.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.972+0000 7f8c56dff700 1 -- 172.21.15.70:0/575015781 wait complete. 2023-10-18T12:26:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.973+0000 7f8c56dff700 1 Processor -- start 2023-10-18T12:26:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.973+0000 7f8c56dff700 1 -- start start 2023-10-18T12:26:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c56dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c501006a0 0x7f8c5019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c56dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c56dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c501066b0 0x7f8c50198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c56dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50199190 con 0x7f8c50101060 2023-10-18T12:26:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c56dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50199300 con 0x7f8c501066b0 2023-10-18T12:26:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c56dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501994c0 con 0x7f8c501006a0 2023-10-18T12:26:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49480/0 (socket says 172.21.15.70:49480) 2023-10-18T12:26:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 -- 172.21.15.70:0/3681062588 learned_addr learned my addr 172.21.15.70:0/3681062588 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c5539c700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c501066b0 0x7f8c50198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c54b9b700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c501006a0 0x7f8c5019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 -- 172.21.15.70:0/3681062588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c501006a0 msgr2=0x7f8c5019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c501006a0 0x7f8c5019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 -- 172.21.15.70:0/3681062588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c501066b0 msgr2=0x7f8c50198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:03.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c501066b0 0x7f8c50198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:03.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 -- 172.21.15.70:0/3681062588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48009d00 con 0x7f8c50101060 2023-10-18T12:26:03.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.974+0000 7f8c4ffff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5019eee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e9a0 tx=0x7f8c3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:03.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.975+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c009790 con 0x7f8c50101060 2023-10-18T12:26:03.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.975+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c010430 con 0x7f8c50101060 2023-10-18T12:26:03.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.975+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c501997a0 con 0x7f8c50101060 2023-10-18T12:26:03.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.975+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c018680 con 0x7f8c50101060 2023-10-18T12:26:03.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.975+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c501a5a40 con 0x7f8c50101060 2023-10-18T12:26:03.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.976+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50101de0 con 0x7f8c50101060 2023-10-18T12:26:03.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.978+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8c3c0105a0 con 0x7f8c50101060 2023-10-18T12:26:03.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.979+0000 7f8c4dffb700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8c400a1550 0x7f8c400a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:03.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.979+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f8c3c014070 con 0x7f8c50101060 2023-10-18T12:26:03.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.979+0000 7f8c54b9b700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8c400a1550 0x7f8c400a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:03.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.980+0000 7f8c54b9b700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8c400a1550 0x7f8c400a3a10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009be0 tx=0x7f8c44009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:03.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:03.983+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8c3c096510 con 0x7f8c50101060 2023-10-18T12:26:04.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.231+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c50067530 con 0x7f8c50101060 2023-10-18T12:26:04.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.231+0000 7f8c4dffb700 1 -- 172.21.15.70:0/3681062588 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f8c3c099b30 con 0x7f8c50101060 2023-10-18T12:26:04.233 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:04.234 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:26:04.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8c400a1550 msgr2=0x7f8c400a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8c400a1550 0x7f8c400a3a10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009be0 tx=0x7f8c44009260 comp rx=0 tx=0).stop 2023-10-18T12:26:04.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 msgr2=0x7f8c5019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5019eee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e9a0 tx=0x7f8c3c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 shutdown_connections 2023-10-18T12:26:04.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8c400a1550 0x7f8c400a3a10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c501066b0 0x7f8c50198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c501006a0 0x7f8c5019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 --2- 172.21.15.70:0/3681062588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c50101060 0x7f8c5019eee0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 >> 172.21.15.70:0/3681062588 conn(0x7f8c500fc170 msgr2=0x7f8c500fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 shutdown_connections 2023-10-18T12:26:04.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.234+0000 7f8c56dff700 1 -- 172.21.15.70:0/3681062588 wait complete. 2023-10-18T12:26:04.239 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:26:04.256 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:26:04.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.452+0000 7f9d8ffff700 1 -- 172.21.15.70:0/4108341816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881006a0 msgr2=0x7f9d88100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.452+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/4108341816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881006a0 0x7f9d88100b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009990 tx=0x7f9d78009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 -- 172.21.15.70:0/4108341816 shutdown_connections 2023-10-18T12:26:04.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/4108341816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/4108341816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d881066b0 0x7f9d88106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/4108341816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881006a0 0x7f9d88100b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 -- 172.21.15.70:0/4108341816 >> 172.21.15.70:0/4108341816 conn(0x7f9d880fc170 msgr2=0x7f9d880fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 -- 172.21.15.70:0/4108341816 shutdown_connections 2023-10-18T12:26:04.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.453+0000 7f9d8ffff700 1 -- 172.21.15.70:0/4108341816 wait complete. 2023-10-18T12:26:04.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 Processor -- start 2023-10-18T12:26:04.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 -- start start 2023-10-18T12:26:04.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d881006a0 0x7f9d8819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881066b0 0x7f9d88198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88199180 con 0x7f9d88101060 2023-10-18T12:26:04.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d881992f0 con 0x7f9d881066b0 2023-10-18T12:26:04.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d881994b0 con 0x7f9d881006a0 2023-10-18T12:26:04.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8d59a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8d59a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49508/0 (socket says 172.21.15.70:49508) 2023-10-18T12:26:04.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8d59a700 1 -- 172.21.15.70:0/1706765953 learned_addr learned my addr 172.21.15.70:0/1706765953 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:04.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.454+0000 7f9d8dd9b700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d881006a0 0x7f9d8819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8e59c700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881066b0 0x7f9d88198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8d59a700 1 -- 172.21.15.70:0/1706765953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d881006a0 msgr2=0x7f9d8819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8d59a700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d881006a0 0x7f9d8819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8d59a700 1 -- 172.21.15.70:0/1706765953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881066b0 msgr2=0x7f9d88198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8d59a700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881066b0 0x7f9d88198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8d59a700 1 -- 172.21.15.70:0/1706765953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80009c40 con 0x7f9d88101060 2023-10-18T12:26:04.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8e59c700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881066b0 0x7f9d88198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:04.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8d59a700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8819ef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801d070 con 0x7f9d88101060 2023-10-18T12:26:04.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d78022470 con 0x7f9d88101060 2023-10-18T12:26:04.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d780095f0 con 0x7f9d88101060 2023-10-18T12:26:04.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78017650 con 0x7f9d88101060 2023-10-18T12:26:04.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.455+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d881a5800 con 0x7f9d88101060 2023-10-18T12:26:04.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.456+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d88101de0 con 0x7f9d88101060 2023-10-18T12:26:04.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.458+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9d780225e0 con 0x7f9d88101060 2023-10-18T12:26:04.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.459+0000 7f9d7effd700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d740a14d0 0x7f9d740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.460+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9d780ce460 con 0x7f9d88101060 2023-10-18T12:26:04.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.460+0000 7f9d8dd9b700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d740a14d0 0x7f9d740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.460+0000 7f9d8dd9b700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d740a14d0 0x7f9d740a3990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009be0 tx=0x7f9d84009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.464+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9d78097760 con 0x7f9d88101060 2023-10-18T12:26:04.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:04 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3082079818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:04 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1672549205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:04.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:04 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3082079818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:04 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1672549205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.669+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d88110620 con 0x7f9d88101060 2023-10-18T12:26:04.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:04 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3082079818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:04 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1672549205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:04.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.671+0000 7f9d7effd700 1 -- 172.21.15.70:0/1706765953 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f9d7809ad80 con 0x7f9d88101060 2023-10-18T12:26:04.673 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:04.673 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:26:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d740a14d0 msgr2=0x7f9d740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d740a14d0 0x7f9d740a3990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009be0 tx=0x7f9d84009260 comp rx=0 tx=0).stop 2023-10-18T12:26:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 msgr2=0x7f9d8819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8819ef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d78004730 comp rx=0 tx=0).stop 2023-10-18T12:26:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 shutdown_connections 2023-10-18T12:26:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d740a14d0 0x7f9d740a3990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d881066b0 0x7f9d88198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d881006a0 0x7f9d8819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 --2- 172.21.15.70:0/1706765953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d88101060 0x7f9d8819ef30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.673+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 >> 172.21.15.70:0/1706765953 conn(0x7f9d880fc170 msgr2=0x7f9d8810bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.674+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 shutdown_connections 2023-10-18T12:26:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.674+0000 7f9d8ffff700 1 -- 172.21.15.70:0/1706765953 wait complete. 2023-10-18T12:26:04.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-18T12:24:09.485042+0000', 'modified': '2023-10-18T12:24:34.631959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24719': {'gid': 24719, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.70:6839/3607185387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6838', 'nonce': 3607185387}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 3607185387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6839/1810263953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1810263953}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1810263953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:26:04.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:26:04.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T12:26:04.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 -- 172.21.15.70:0/2033472583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c01006a0 msgr2=0x7f19c0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 --2- 172.21.15.70:0/2033472583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c01006a0 0x7f19c0100b20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009990 tx=0x7f19b4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 -- 172.21.15.70:0/2033472583 shutdown_connections 2023-10-18T12:26:04.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 --2- 172.21.15.70:0/2033472583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19c0101060 0x7f19c010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 --2- 172.21.15.70:0/2033472583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19c01066b0 0x7f19c0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 --2- 172.21.15.70:0/2033472583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c01006a0 0x7f19c0100b20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 -- 172.21.15.70:0/2033472583 >> 172.21.15.70:0/2033472583 conn(0x7f19c00fc190 msgr2=0x7f19c00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 -- 172.21.15.70:0/2033472583 shutdown_connections 2023-10-18T12:26:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.892+0000 7f19c67ff700 1 -- 172.21.15.70:0/2033472583 wait complete. 2023-10-18T12:26:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.893+0000 7f19c67ff700 1 Processor -- start 2023-10-18T12:26:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.893+0000 7f19c67ff700 1 -- start start 2023-10-18T12:26:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.893+0000 7f19c67ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19c01006a0 0x7f19c019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.893+0000 7f19c67ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 0x7f19c019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.893+0000 7f19c67ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19c01066b0 0x7f19c0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.893+0000 7f19c67ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0199170 con 0x7f19c0101060 2023-10-18T12:26:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19c67ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01992e0 con 0x7f19c01006a0 2023-10-18T12:26:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19c67ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01994a0 con 0x7f19c01066b0 2023-10-18T12:26:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 0x7f19c019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19c01006a0 0x7f19c019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 0x7f19c019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49522/0 (socket says 172.21.15.70:49522) 2023-10-18T12:26:04.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3691201236 learned_addr learned my addr 172.21.15.70:0/3691201236 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:04.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19c01066b0 0x7f19c0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3691201236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19c01066b0 msgr2=0x7f19c0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19c01066b0 0x7f19c0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3691201236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19c01006a0 msgr2=0x7f19c019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19c01006a0 0x7f19c019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3691201236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b40095f0 con 0x7f19c0101060 2023-10-18T12:26:04.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.894+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 0x7f19c019ef20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f19b4005910 tx=0x7f19b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.895+0000 7f19bd7fa700 1 -- 172.21.15.70:0/3691201236 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b401d070 con 0x7f19c0101060 2023-10-18T12:26:04.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.895+0000 7f19bd7fa700 1 -- 172.21.15.70:0/3691201236 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b4022470 con 0x7f19c0101060 2023-10-18T12:26:04.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.895+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c0199720 con 0x7f19c0101060 2023-10-18T12:26:04.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.895+0000 7f19bd7fa700 1 -- 172.21.15.70:0/3691201236 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4017650 con 0x7f19c0101060 2023-10-18T12:26:04.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.895+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c01a59d0 con 0x7f19c0101060 2023-10-18T12:26:04.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.898+0000 7f19bd7fa700 1 -- 172.21.15.70:0/3691201236 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f19b40225e0 con 0x7f19c0101060 2023-10-18T12:26:04.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 Processor -- start 2023-10-18T12:26:04.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 -- start start 2023-10-18T12:26:04.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19bd7fa700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19a80a1550 0x7f19a80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19bd7fa700 1 -- 172.21.15.70:0/3691201236 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f19b40ce370 con 0x7f19c0101060 2023-10-18T12:26:04.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae67040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19a80a1550 0x7f19a80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36adda690 con 0x55b36ae6d4b0 2023-10-18T12:26:04.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36adc0c10 con 0x55b36ae6cb90 2023-10-18T12:26:04.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.899+0000 7f19da20fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36adbe530 con 0x55b36ae67580 2023-10-18T12:26:04.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae67040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae67040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49528/0 (socket says 172.21.15.70:49528) 2023-10-18T12:26:04.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3363863210 learned_addr learned my addr 172.21.15.70:0/3363863210 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:04.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bffff700 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3363863210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 msgr2=0x55b36ae67a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae67a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3363863210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 msgr2=0x55b36ae6cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 -- 172.21.15.70:0/3363863210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b36ad773a0 con 0x55b36ae6d4b0 2023-10-18T12:26:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae67a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:26:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19a80a1550 0x7f19a80a3a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f19b800f540 tx=0x7f19b8009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae67040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19b40978f0 tx=0x7f19b4022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1937eb50eb1fdb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.900+0000 7f19a67fc700 1 -- 172.21.15.70:0/3363863210 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b409ad50 con 0x55b36ae6d4b0 2023-10-18T12:26:04.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19a67fc700 1 -- 172.21.15.70:0/3363863210 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b40b0030 con 0x55b36ae6d4b0 2023-10-18T12:26:04.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19a67fc700 1 -- 172.21.15.70:0/3363863210 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40a36f0 con 0x55b36ae6d4b0 2023-10-18T12:26:04.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 -- 172.21.15.70:0/3363863210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 msgr2=0x55b36ae67040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae67040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19b40978f0 tx=0x7f19b4022f90 comp rx=0 tx=0).stop 2023-10-18T12:26:04.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 -- 172.21.15.70:0/3363863210 shutdown_connections 2023-10-18T12:26:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae67a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3363863210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae67040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 -- 172.21.15.70:0/3363863210 >> 172.21.15.70:0/3363863210 conn(0x55b36addb6d0 msgr2=0x55b36ae61b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 -- 172.21.15.70:0/3363863210 shutdown_connections 2023-10-18T12:26:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.901+0000 7f19da20fb80 1 -- 172.21.15.70:0/3363863210 wait complete. 2023-10-18T12:26:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.902+0000 7f19da20fb80 1 -- start start 2023-10-18T12:26:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.902+0000 7f19da20fb80 1 Processor -- start 2023-10-18T12:26:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36adb9e20 con 0x55b36ae67580 2023-10-18T12:26:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ad566c0 con 0x55b36ae6cb90 2023-10-18T12:26:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ad56360 con 0x55b36ae6d4b0 2023-10-18T12:26:04.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19da20fb80 10 client.0 _ll_register_callbacks cb 0x55b36aca0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:04.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49540/0 (socket says 172.21.15.70:49540) 2023-10-18T12:26:04.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 -- 172.21.15.70:0/2792609588 learned_addr learned my addr 172.21.15.70:0/2792609588 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:04.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 -- 172.21.15.70:0/2792609588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 msgr2=0x55b36ae70d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 -- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 msgr2=0x55b36ae706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bffff700 1 -- 172.21.15.70:0/2792609588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c01a59d0 con 0x55b36ae67580 2023-10-18T12:26:04.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.903+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:04.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f19bffff700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae733d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005c50 tx=0x7f19b0012a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e638f1032fcdf227 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f199f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:04.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f199f7fe700 1 -- 172.21.15.70:0/2792609588 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017070 con 0x55b36ae67580 2023-10-18T12:26:04.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b36adbeea0 con 0x55b36ae67580 2023-10-18T12:26:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f199f7fe700 1 -- 172.21.15.70:0/2792609588 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f19b0010430 con 0x55b36ae67580 2023-10-18T12:26:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f199f7fe700 1 -- 172.21.15.70:0/2792609588 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b00105f0 con 0x55b36ae67580 2023-10-18T12:26:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f19da20fb80 10 client.15417 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b36abe4d90 con 0x55b36ae67580 2023-10-18T12:26:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.904+0000 7f199f7fe700 1 -- 172.21.15.70:0/2792609588 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b36adbeea0 con 0x55b36ae67580 2023-10-18T12:26:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f199f7fe700 1 -- 172.21.15.70:0/2792609588 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f19b001c840 con 0x55b36ae67580 2023-10-18T12:26:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19da20fb80 10 client.15417 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19da20fb80 10 client.15417 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T12:26:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19da20fb80 20 client.15417 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19da20fb80 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36ae80be0 0x55b36ae7f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19da20fb80 4 client.15417 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:04.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b36ae36860 con 0x55b36ae80be0 2023-10-18T12:26:04.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.905+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36ae80be0 0x55b36ae7f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.906+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36ae80be0 0x55b36ae7f660 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8362dc33914453b3 server_cookie=bc698b6bf8dd5fac in_seq=0 out_seq=0 2023-10-18T12:26:04.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.906+0000 7f199f7fe700 10 client.15417 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:04.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f199f7fe700 1 -- 172.21.15.70:0/2792609588 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b36ae36860 con 0x55b36ae80be0 2023-10-18T12:26:04.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f199f7fe700 10 client.15417 handle_command_reply: tid=0 2023-10-18T12:26:04.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 client.15417 shutdown 2023-10-18T12:26:04.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 msgr2=0x55b36ae733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae733d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005c50 tx=0x7f19b0012a20 comp rx=0 tx=0).stop 2023-10-18T12:26:04.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 shutdown_connections 2023-10-18T12:26:04.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36ae80be0 0x55b36ae7f660 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae733d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 --2- 172.21.15.70:0/2792609588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f199f7fe700 0 client.15417 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:04.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 >> 172.21.15.70:0/2792609588 conn(0x55b36addb6d0 msgr2=0x55b36ae66030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 shutdown_connections 2023-10-18T12:26:04.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 1 -- 172.21.15.70:0/2792609588 wait complete. 2023-10-18T12:26:04.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.907+0000 7f19da20fb80 20 client.15417 trim_cache size 0 max 16384 2023-10-18T12:26:04.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 Processor -- start 2023-10-18T12:26:04.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 -- start start 2023-10-18T12:26:04.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ac3a8d0 con 0x55b36ae67580 2023-10-18T12:26:04.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36adbb0b0 con 0x55b36ae6cb90 2023-10-18T12:26:04.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.925+0000 7f19da20fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ad78590 con 0x55b36ae6d4b0 2023-10-18T12:26:04.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae7cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49556/0 (socket says 172.21.15.70:49556) 2023-10-18T12:26:04.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 -- 172.21.15.70:0/4234938831 learned_addr learned my addr 172.21.15.70:0/4234938831 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 -- 172.21.15.70:0/4234938831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 msgr2=0x55b36ae70c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 -- 172.21.15.70:0/4234938831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 msgr2=0x55b36ae6ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 -- 172.21.15.70:0/4234938831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b36abcdc10 con 0x55b36ae67580 2023-10-18T12:26:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19bffff700 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae7cc00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009340 tx=0x7f19b0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd28b9ebc4f4a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.926+0000 7f19a6ffd700 1 -- 172.21.15.70:0/4234938831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017040 con 0x55b36ae67580 2023-10-18T12:26:04.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19a6ffd700 1 -- 172.21.15.70:0/4234938831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b002a440 con 0x55b36ae67580 2023-10-18T12:26:04.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19a6ffd700 1 -- 172.21.15.70:0/4234938831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0010690 con 0x55b36ae67580 2023-10-18T12:26:04.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 -- 172.21.15.70:0/4234938831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 msgr2=0x55b36ae7cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae7cc00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009340 tx=0x7f19b0009480 comp rx=0 tx=0).stop 2023-10-18T12:26:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 -- 172.21.15.70:0/4234938831 shutdown_connections 2023-10-18T12:26:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae67580 0x55b36ae7cc00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 --2- 172.21.15.70:0/4234938831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 -- 172.21.15.70:0/4234938831 >> 172.21.15.70:0/4234938831 conn(0x55b36addb6d0 msgr2=0x55b36ae6a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 -- 172.21.15.70:0/4234938831 shutdown_connections 2023-10-18T12:26:04.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.927+0000 7f19da20fb80 1 -- 172.21.15.70:0/4234938831 wait complete. 2023-10-18T12:26:04.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.928+0000 7f19da20fb80 1 -- start start 2023-10-18T12:26:04.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.928+0000 7f19da20fb80 1 Processor -- start 2023-10-18T12:26:04.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.928+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae89560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19da20fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19da20fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ae70680 con 0x55b36ae6d4b0 2023-10-18T12:26:04.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19da20fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ae707f0 con 0x55b36ae6cb90 2023-10-18T12:26:04.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19da20fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36ae709b0 con 0x55b36ae67580 2023-10-18T12:26:04.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19da20fb80 10 client.0 _ll_register_callbacks cb 0x55b36aca0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:04.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae89560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bf7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae89560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42166/0 (socket says 172.21.15.70:42166) 2023-10-18T12:26:04.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 -- 172.21.15.70:0/3879962879 learned_addr learned my addr 172.21.15.70:0/3879962879 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:04.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19c4d9c700 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 -- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 msgr2=0x55b36ae6fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 -- 172.21.15.70:0/3879962879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 msgr2=0x55b36ae70140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:04.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.929+0000 7f19bffff700 1 -- 172.21.15.70:0/3879962879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b36abe4d90 con 0x55b36ae67580 2023-10-18T12:26:04.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f19bffff700 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae89560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19b001cc70 tx=0x7f19b002abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd515a2bfabbde1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:04.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f199f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:26:04.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f199f7fe700 1 -- 172.21.15.70:0/3879962879 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017040 con 0x55b36ae67580 2023-10-18T12:26:04.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19b40b0030 con 0x55b36ae67580 2023-10-18T12:26:04.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f199f7fe700 1 -- 172.21.15.70:0/3879962879 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f19b001c630 con 0x55b36ae67580 2023-10-18T12:26:04.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f199f7fe700 1 -- 172.21.15.70:0/3879962879 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b0005c50 con 0x55b36ae67580 2023-10-18T12:26:04.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f19da20fb80 10 client.25447 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:04.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b36ac0b200 con 0x55b36ae67580 2023-10-18T12:26:04.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.930+0000 7f199f7fe700 1 -- 172.21.15.70:0/3879962879 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b40b0030 con 0x55b36ae67580 2023-10-18T12:26:04.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f199f7fe700 1 -- 172.21.15.70:0/3879962879 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f19b003d070 con 0x55b36ae67580 2023-10-18T12:26:04.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19da20fb80 10 client.25447 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:04.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19da20fb80 10 client.25447 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T12:26:04.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19da20fb80 20 client.25447 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:04.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36aed91f0 0x55b36aed5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:04.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19da20fb80 4 client.25447 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:26:04.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b36adb2e00 con 0x55b36aed91f0 2023-10-18T12:26:04.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.931+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36aed91f0 0x55b36aed5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:04.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f199f7fe700 10 client.25447 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:04.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19bf7fe700 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36aed91f0 0x55b36aed5610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded18a51225d251b server_cookie=da93f7d0e699061c in_seq=0 out_seq=0 2023-10-18T12:26:04.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f199f7fe700 1 -- 172.21.15.70:0/3879962879 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55b36adb2e00 con 0x55b36aed91f0 2023-10-18T12:26:04.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f199f7fe700 10 client.25447 handle_command_reply: tid=0 2023-10-18T12:26:04.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 client.25447 shutdown 2023-10-18T12:26:04.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 msgr2=0x55b36ae89560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae89560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19b001cc70 tx=0x7f19b002abf0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 shutdown_connections 2023-10-18T12:26:04.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b36aed91f0 0x55b36aed5610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b36ae6d4b0 0x55b36ae70140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b36ae67580 0x55b36ae89560 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f199f7fe700 0 client.25447 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:04.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 --2- 172.21.15.70:0/3879962879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b36ae6cb90 0x55b36ae6fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 >> 172.21.15.70:0/3879962879 conn(0x55b36addb6d0 msgr2=0x55b36ae5b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 shutdown_connections 2023-10-18T12:26:04.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 1 -- 172.21.15.70:0/3879962879 wait complete. 2023-10-18T12:26:04.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.932+0000 7f19da20fb80 20 client.25447 trim_cache size 0 max 16384 2023-10-18T12:26:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:26:04.951 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "f94fde2a-6daf-11ee-8db6-212e2dc638e7", 2023-10-18T12:26:04.951 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-18T12:26:04.952 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 24743, 2023-10-18T12:26:04.952 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-18T12:26:04.952 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-18T12:26:04.952 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-18T12:26:04.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 114.812324789, 2023-10-18T12:26:04.953 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 20, 2023-10-18T12:26:04.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 91, 2023-10-18T12:26:04.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 89, 2023-10-18T12:26:04.953 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 121.788628604 2023-10-18T12:26:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:26:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19a80a1550 msgr2=0x7f19a80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19a80a1550 0x7f19a80a3a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f19b800f540 tx=0x7f19b8009920 comp rx=0 tx=0).stop 2023-10-18T12:26:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 msgr2=0x7f19c019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 0x7f19c019ef20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f19b4005910 tx=0x7f19b4004730 comp rx=0 tx=0).stop 2023-10-18T12:26:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 shutdown_connections 2023-10-18T12:26:04.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19c01066b0 0x7f19c0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f19a80a1550 0x7f19a80a3a10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19c01006a0 0x7f19c019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 --2- 172.21.15.70:0/3691201236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c0101060 0x7f19c019ef20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:04.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 >> 172.21.15.70:0/3691201236 conn(0x7f19c00fc190 msgr2=0x7f19c0072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:04.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 shutdown_connections 2023-10-18T12:26:04.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:04.933+0000 7f19c67ff700 1 -- 172.21.15.70:0/3691201236 wait complete. 2023-10-18T12:26:04.957 INFO:teuthology.run_tasks:Running task kclient... 2023-10-18T12:26:04.969 INFO:tasks.kclient:Mounting kernel clients... 2023-10-18T12:26:04.969 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-18T12:26:04.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:04.970 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-18T12:26:05.013 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-18T12:26:05.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:05.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T12:26:05.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:05.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-10-18T12:26:05.228 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:05 smithi070 ceph-mon[98639]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:05.229 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:05 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3681062588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:05.229 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:05 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1706765953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:05.238 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-10-18T12:26:05.238 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T12:26:05.239 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:05.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-18T12:26:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:05 smithi186 ceph-mon[135253]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:05.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:05 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3681062588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:05.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:05 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1706765953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:05 smithi152 ceph-mon[134767]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:05.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:05 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3681062588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:05.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:05 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1706765953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:05.975 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:26:07.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:07 smithi070 ceph-mon[98639]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:07.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:07 smithi186 ceph-mon[135253]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:07.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 -- 172.21.15.70:0/1701477021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84641057c0 msgr2=0x7f8464105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:07.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 --2- 172.21.15.70:0/1701477021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84641057c0 0x7f8464105ba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8454009990 tx=0x7f8454009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 -- 172.21.15.70:0/1701477021 shutdown_connections 2023-10-18T12:26:07.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 --2- 172.21.15.70:0/1701477021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8464101260 0x7f846410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 --2- 172.21.15.70:0/1701477021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84641057c0 0x7f8464105ba0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 --2- 172.21.15.70:0/1701477021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84641008a0 0x7f8464100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 -- 172.21.15.70:0/1701477021 >> 172.21.15.70:0/1701477021 conn(0x7f84640779e0 msgr2=0x7f8464077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:07.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 -- 172.21.15.70:0/1701477021 shutdown_connections 2023-10-18T12:26:07.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.592+0000 7f8469222700 1 -- 172.21.15.70:0/1701477021 wait complete. 2023-10-18T12:26:07.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.593+0000 7f8469222700 1 Processor -- start 2023-10-18T12:26:07.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.593+0000 7f8469222700 1 -- start start 2023-10-18T12:26:07.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8469222700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 0x7f8464072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:07.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8469222700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 0x7f8464073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:07.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8469222700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84641057c0 0x7f8464073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:07.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8469222700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641a5030 con 0x7f8464101260 2023-10-18T12:26:07.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8469222700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641a51a0 con 0x7f84641008a0 2023-10-18T12:26:07.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8469222700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641a5360 con 0x7f84641057c0 2023-10-18T12:26:07.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f846259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 0x7f8464073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:07.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 0x7f8464072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:07.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 0x7f8464072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:33698/0 (socket says 172.21.15.70:33698) 2023-10-18T12:26:07.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f846259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 0x7f8464073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49598/0 (socket says 172.21.15.70:49598) 2023-10-18T12:26:07.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 -- 172.21.15.70:0/2944013390 learned_addr learned my addr 172.21.15.70:0/2944013390 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:07.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f846359e700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84641057c0 0x7f8464073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:07.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 -- 172.21.15.70:0/2944013390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84641057c0 msgr2=0x7f8464073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:07.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84641057c0 0x7f8464073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 -- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 msgr2=0x7f8464073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:07.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 0x7f8464073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f8462d9d700 1 -- 172.21.15.70:0/2944013390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540095f0 con 0x7f84641008a0 2023-10-18T12:26:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.594+0000 7f846259c700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 0x7f8464073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.595+0000 7f8462d9d700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 0x7f8464072af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.595+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845401d070 con 0x7f84641008a0 2023-10-18T12:26:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.595+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84641a55e0 con 0x7f84641008a0 2023-10-18T12:26:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.595+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8454022470 con 0x7f84641008a0 2023-10-18T12:26:07.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.595+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454017650 con 0x7f84641008a0 2023-10-18T12:26:07.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.595+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84641a5a00 con 0x7f84641008a0 2023-10-18T12:26:07.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.596+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8464108d80 con 0x7f84641008a0 2023-10-18T12:26:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.598+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f845400fad0 con 0x7f84641008a0 2023-10-18T12:26:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.599+0000 7f844bfff700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f844c0a14d0 0x7f844c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.600+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f84540ceb80 con 0x7f84641008a0 2023-10-18T12:26:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.600+0000 7f846259c700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f844c0a14d0 0x7f844c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.600+0000 7f846259c700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f844c0a14d0 0x7f844c0a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f84640741d0 tx=0x7f8458009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.602+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8454097e20 con 0x7f84641008a0 2023-10-18T12:26:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:07 smithi152 ceph-mon[134767]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:07.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.845+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8464073f80 con 0x7f84641008a0 2023-10-18T12:26:07.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.845+0000 7f844bfff700 1 -- 172.21.15.70:0/2944013390 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+88 (secure 0 0 0) 0x7f845409b440 con 0x7f84641008a0 2023-10-18T12:26:07.847 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T12:26:07.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f844c0a14d0 msgr2=0x7f844c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f844c0a14d0 0x7f844c0a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f84640741d0 tx=0x7f8458009260 comp rx=0 tx=0).stop 2023-10-18T12:26:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 msgr2=0x7f8464072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 0x7f8464072af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454005740 comp rx=0 tx=0).stop 2023-10-18T12:26:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 shutdown_connections 2023-10-18T12:26:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84641057c0 0x7f8464073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f844c0a14d0 0x7f844c0a3990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84641008a0 0x7f8464072af0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 --2- 172.21.15.70:0/2944013390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8464101260 0x7f8464073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 >> 172.21.15.70:0/2944013390 conn(0x7f84640779e0 msgr2=0x7f84640fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:07.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 shutdown_connections 2023-10-18T12:26:07.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:07.848+0000 7f8469222700 1 -- 172.21.15.70:0/2944013390 wait complete. 2023-10-18T12:26:08.356 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T12:26:08.356 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T12:26:08.356 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-18T12:26:08.357 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T12:26:08.357 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:08.357 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T12:26:08.357 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T12:26:08.358 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T12:26:08.358 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:26:08.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:08.359 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-18T12:26:08.405 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T12:26:08.405 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T12:26:08.405 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T12:26:08.406 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:26:08.406 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-18T12:26:08.406 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:26:08.406 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:26:08.407 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:26:08.407 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-18T12:26:08.407 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:26:08.407 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:26:08.407 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-18T12:26:08.408 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:26:08.408 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:26:08.408 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T12:26:08.408 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:26:08.409 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T12:26:08.409 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41568sec preferred_lft 41568sec 2023-10-18T12:26:08.409 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::6f76:96b5:cb68:c2e6/64 scope link noprefixroute 2023-10-18T12:26:08.409 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:26:08.410 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T12:26:08.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:08.411 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:26:08.411 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:26:08.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2023-10-18T12:26:08.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2023-10-18T12:26:08.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2023-10-18T12:26:08.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T12:26:08.413 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:26:08.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:08 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2944013390' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T12:26:08.554 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:08 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2944013390' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T12:26:08.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:08.658 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T12:26:08.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:08 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2944013390' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T12:26:08.727 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-18T12:26:08.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:08.731 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-18T12:26:08.777 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-18T12:26:08.778 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T12:26:08.778 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T12:26:08.778 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T12:26:08.778 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T12:26:08.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:08.780 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:26:08.780 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:26:08.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T12:26:08.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T12:26:08.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T12:26:08.781 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:26:08.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:08.955 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-18T12:26:09.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:09.001 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-10-18T12:26:09.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:09.085 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:26:09.086 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:26:09.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:26:09.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-18T12:26:09.087 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:26:09.270 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-18T12:26:09.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:09.271 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:26:09.272 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:26:09.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-18T12:26:09.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T12:26:09.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-18T12:26:09.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-18T12:26:09.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-18T12:26:09.273 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:26:09.446 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:09 smithi070 ceph-mon[98639]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:09.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:09 smithi186 ceph-mon[135253]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:09.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:09.521 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-18T12:26:09.522 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-18T12:26:09.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-18T12:26:09.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T12:26:09.523 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-18T12:26:09.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:09 smithi152 ceph-mon[134767]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:09.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:26:09.697 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:09.742 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:26:09.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:26:09.743 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:09.824 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-18T12:26:09.825 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-18T12:26:09.825 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/ceb5577f-b957-4f73-96f3-adc6f44d9343 2023-10-18T12:26:09.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:09.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/ceb5577f-b957-4f73-96f3-adc6f44d9343 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-10-18T12:26:09.970 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-10-18T12:26:09.970 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2023-10-18T12:26:09.970 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2023-10-18T12:26:10.474 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-10-18T12:26:10.474 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2023-10-18T12:26:10.474 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2023-10-18T12:26:10.475 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:6789,172.21.15.152:6789,172.21.15.186:6789" 2023-10-18T12:26:10.475 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.70:6789,172.21.15.152:6789,172.21.15.186:6789:/volumes/_nogroup/sv_1/ceb5577f-b957-4f73-96f3-adc6f44d9343 2023-10-18T12:26:10.475 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=f94fde2a-6daf-11ee-8db6-212e2dc638e7" will pass to kernel 2023-10-18T12:26:10.481 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T12:26:10.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:10.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:10.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:10.555 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T12:26:10.642 INFO:teuthology.orchestra.run.smithi070.stdout:client15432 2023-10-18T12:26:10.643 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:26:10.643 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:26:10.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:10.688 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T12:26:10.814 INFO:teuthology.orchestra.run.smithi070.stdout:client15432 2023-10-18T12:26:10.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:10.815 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T12:26:10.902 INFO:teuthology.orchestra.run.smithi070.stdout:client15432 2023-10-18T12:26:10.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:26:10.903 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7.client15432/status') 2023-10-18T12:26:11.010 INFO:teuthology.orchestra.run.smithi070.stdout:instance: client.15432 (3)192.168.0.1:0/2584154621 2023-10-18T12:26:11.010 INFO:teuthology.orchestra.run.smithi070.stdout:blocklisted: false 2023-10-18T12:26:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records in 2023-10-18T12:26:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records out 2023-10-18T12:26:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:70 bytes copied, 5.2401e-05 s, 1.3 MB/s 2023-10-18T12:26:11.013 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-18T12:26:11.025 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-18T12:26:11.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:26:11.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 -- 172.21.15.70:0/2705173222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 msgr2=0x7f4c4c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2705173222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c106a90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009990 tx=0x7f4c40009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 -- 172.21.15.70:0/2705173222 shutdown_connections 2023-10-18T12:26:11.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2705173222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 0x7f4c4c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2705173222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c106a90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2705173222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 0x7f4c4c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 -- 172.21.15.70:0/2705173222 >> 172.21.15.70:0/2705173222 conn(0x7f4c4c0fc1d0 msgr2=0x7f4c4c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:11.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 -- 172.21.15.70:0/2705173222 shutdown_connections 2023-10-18T12:26:11.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.240+0000 7f4c52bff700 1 -- 172.21.15.70:0/2705173222 wait complete. 2023-10-18T12:26:11.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.241+0000 7f4c52bff700 1 Processor -- start 2023-10-18T12:26:11.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.241+0000 7f4c52bff700 1 -- start start 2023-10-18T12:26:11.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c52bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 0x7f4c4c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c52bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 0x7f4c4c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c52bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c52bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c19cb60 con 0x7f4c4c1066b0 2023-10-18T12:26:11.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c52bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c19ccd0 con 0x7f4c4c1006a0 2023-10-18T12:26:11.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c52bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c19ce90 con 0x7f4c4c101060 2023-10-18T12:26:11.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c5119c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c5119c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39878/0 (socket says 172.21.15.70:39878) 2023-10-18T12:26:11.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c5099b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 0x7f4c4c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c5119c700 1 -- 172.21.15.70:0/2608780740 learned_addr learned my addr 172.21.15.70:0/2608780740 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:11.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.242+0000 7f4c4bfff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 0x7f4c4c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5119c700 1 -- 172.21.15.70:0/2608780740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 msgr2=0x7f4c4c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5119c700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 0x7f4c4c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5119c700 1 -- 172.21.15.70:0/2608780740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 msgr2=0x7f4c4c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5119c700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 0x7f4c4c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5119c700 1 -- 172.21.15.70:0/2608780740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c400095f0 con 0x7f4c4c1066b0 2023-10-18T12:26:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c4bfff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 0x7f4c4c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5099b700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 0x7f4c4c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:11.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c5119c700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c19c560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400ef30 tx=0x7f4c4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:11.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44018070 con 0x7f4c4c1066b0 2023-10-18T12:26:11.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c4400f040 con 0x7f4c4c1066b0 2023-10-18T12:26:11.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c19d110 con 0x7f4c4c1066b0 2023-10-18T12:26:11.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400cd00 con 0x7f4c4c1066b0 2023-10-18T12:26:11.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.243+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c1a5ad0 con 0x7f4c4c1066b0 2023-10-18T12:26:11.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.244+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c101de0 con 0x7f4c4c1066b0 2023-10-18T12:26:11.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.246+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4c44003a00 con 0x7f4c4c1066b0 2023-10-18T12:26:11.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.248+0000 7f4c49ffb700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c3c0a1550 0x7f4c3c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.248+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f4c440ccc30 con 0x7f4c4c1066b0 2023-10-18T12:26:11.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.248+0000 7f4c5099b700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c3c0a1550 0x7f4c3c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.248+0000 7f4c5099b700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c3c0a1550 0x7f4c3c0a3a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4c40006010 tx=0x7f4c4000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:11.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.252+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4c44095ed0 con 0x7f4c4c1066b0 2023-10-18T12:26:11.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:11 smithi070 ceph-mon[98639]: pgmap v317: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:11.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.456+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c4c067530 con 0x7f4c4c1066b0 2023-10-18T12:26:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.458+0000 7f4c49ffb700 1 -- 172.21.15.70:0/2608780740 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f4c440994f0 con 0x7f4c4c1066b0 2023-10-18T12:26:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:11.462 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:26:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.460+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c3c0a1550 msgr2=0x7f4c3c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c3c0a1550 0x7f4c3c0a3a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4c40006010 tx=0x7f4c4000b260 comp rx=0 tx=0).stop 2023-10-18T12:26:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 msgr2=0x7f4c4c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c19c560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400ef30 tx=0x7f4c4400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 shutdown_connections 2023-10-18T12:26:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c4c1066b0 0x7f4c4c19c560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c3c0a1550 0x7f4c3c0a3a10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c4c1006a0 0x7f4c4c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 --2- 172.21.15.70:0/2608780740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c4c101060 0x7f4c4c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 >> 172.21.15.70:0/2608780740 conn(0x7f4c4c0fc1d0 msgr2=0x7f4c4c10be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:11.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 shutdown_connections 2023-10-18T12:26:11.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.461+0000 7f4c52bff700 1 -- 172.21.15.70:0/2608780740 wait complete. 2023-10-18T12:26:11.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:11 smithi186 ceph-mon[135253]: pgmap v317: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:11.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:26:11.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:26:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:11 smithi152 ceph-mon[134767]: pgmap v317: 129 pgs: 129 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.683+0000 7fd5227ff700 1 -- 172.21.15.70:0/953231686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c073540 msgr2=0x7fd51c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.683+0000 7fd5227ff700 1 --2- 172.21.15.70:0/953231686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c073540 0x7fd51c10ca20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098f0 tx=0x7fd510009c00 comp rx=0 tx=0).stop 2023-10-18T12:26:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 -- 172.21.15.70:0/953231686 shutdown_connections 2023-10-18T12:26:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 --2- 172.21.15.70:0/953231686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c10cf60 0x7fd51c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 --2- 172.21.15.70:0/953231686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c072c20 0x7fd51c073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 --2- 172.21.15.70:0/953231686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c073540 0x7fd51c10ca20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 -- 172.21.15.70:0/953231686 >> 172.21.15.70:0/953231686 conn(0x7fd51c0fc1b0 msgr2=0x7fd51c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:11.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 -- 172.21.15.70:0/953231686 shutdown_connections 2023-10-18T12:26:11.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.684+0000 7fd5227ff700 1 -- 172.21.15.70:0/953231686 wait complete. 2023-10-18T12:26:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 Processor -- start 2023-10-18T12:26:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 -- start start 2023-10-18T12:26:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c072c20 0x7fd51c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 0x7fd51c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c10cf60 0x7fd51c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c19ddb0 con 0x7fd51c073540 2023-10-18T12:26:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c19df20 con 0x7fd51c10cf60 2023-10-18T12:26:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd5227ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c19e0e0 con 0x7fd51c072c20 2023-10-18T12:26:11.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd51b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 0x7fd51c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd51b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 0x7fd51c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39902/0 (socket says 172.21.15.70:39902) 2023-10-18T12:26:11.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd51bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c072c20 0x7fd51c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd51b7fe700 1 -- 172.21.15.70:0/3587353234 learned_addr learned my addr 172.21.15.70:0/3587353234 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:11.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.685+0000 7fd520d9c700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c10cf60 0x7fd51c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd51b7fe700 1 -- 172.21.15.70:0/3587353234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c072c20 msgr2=0x7fd51c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd51b7fe700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c072c20 0x7fd51c198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd51b7fe700 1 -- 172.21.15.70:0/3587353234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c10cf60 msgr2=0x7fd51c19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd51b7fe700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c10cf60 0x7fd51c19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd51b7fe700 1 -- 172.21.15.70:0/3587353234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0095f0 con 0x7fd51c073540 2023-10-18T12:26:11.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd51b7fe700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 0x7fd51c199420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd510005950 tx=0x7fd51000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:11.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51001c070 con 0x7fd51c073540 2023-10-18T12:26:11.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51000baa0 con 0x7fd51c073540 2023-10-18T12:26:11.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd510009520 con 0x7fd51c073540 2023-10-18T12:26:11.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100038e0 con 0x7fd51c073540 2023-10-18T12:26:11.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.686+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c1a5a20 con 0x7fd51c073540 2023-10-18T12:26:11.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.687+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c108170 con 0x7fd51c073540 2023-10-18T12:26:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.689+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd510020030 con 0x7fd51c073540 2023-10-18T12:26:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.691+0000 7fd5197fa700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5040a14d0 0x7fd5040a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.691+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fd5100ce0d0 con 0x7fd51c073540 2023-10-18T12:26:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.691+0000 7fd51bfff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5040a14d0 0x7fd5040a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:11.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.691+0000 7fd51bfff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5040a14d0 0x7fd5040a3990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005fd0 tx=0x7fd50c009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:11.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.693+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd510097370 con 0x7fd51c073540 2023-10-18T12:26:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.891+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd51c19a190 con 0x7fd51c073540 2023-10-18T12:26:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.891+0000 7fd5197fa700 1 -- 172.21.15.70:0/3587353234 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd51009a990 con 0x7fd51c073540 2023-10-18T12:26:11.894 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:26:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.893+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5040a14d0 msgr2=0x7fd5040a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.893+0000 7fd5227ff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5040a14d0 0x7fd5040a3990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005fd0 tx=0x7fd50c009310 comp rx=0 tx=0).stop 2023-10-18T12:26:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.893+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 msgr2=0x7fd51c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.893+0000 7fd5227ff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 0x7fd51c199420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd510005950 tx=0x7fd51000fa00 comp rx=0 tx=0).stop 2023-10-18T12:26:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 shutdown_connections 2023-10-18T12:26:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd5040a14d0 0x7fd5040a3990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51c10cf60 0x7fd51c19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c072c20 0x7fd51c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 --2- 172.21.15.70:0/3587353234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51c073540 0x7fd51c199420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 >> 172.21.15.70:0/3587353234 conn(0x7fd51c0fc1b0 msgr2=0x7fd51c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 shutdown_connections 2023-10-18T12:26:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:11.894+0000 7fd5227ff700 1 -- 172.21.15.70:0/3587353234 wait complete. 2023-10-18T12:26:11.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:26:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/3843455127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f94100720 msgr2=0x7f4f94100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/3843455127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f94100720 0x7f4f94100ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009990 tx=0x7f4f7c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/3843455127 shutdown_connections 2023-10-18T12:26:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/3843455127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f941010e0 0x7f4f9410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/3843455127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94106730 0x7f4f94106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/3843455127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f94100720 0x7f4f94100ba0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/3843455127 >> 172.21.15.70:0/3843455127 conn(0x7f4f940fc1b0 msgr2=0x7f4f940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/3843455127 shutdown_connections 2023-10-18T12:26:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.115+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/3843455127 wait complete. 2023-10-18T12:26:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.116+0000 7f4f9a3ff700 1 Processor -- start 2023-10-18T12:26:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.116+0000 7f4f9a3ff700 1 -- start start 2023-10-18T12:26:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9a3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 0x7f4f94072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9a3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f941010e0 0x7f4f94073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9a3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94106730 0x7f4f94073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9a3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941a4ee0 con 0x7f4f94100720 2023-10-18T12:26:12.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9a3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941a5050 con 0x7f4f94106730 2023-10-18T12:26:12.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9a3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941a5210 con 0x7f4f941010e0 2023-10-18T12:26:12.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 0x7f4f94072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 0x7f4f94072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39926/0 (socket says 172.21.15.70:39926) 2023-10-18T12:26:12.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 -- 172.21.15.70:0/1147067168 learned_addr learned my addr 172.21.15.70:0/1147067168 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:12.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f8bfff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f941010e0 0x7f4f94073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f9899c700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94106730 0x7f4f94073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 -- 172.21.15.70:0/1147067168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f941010e0 msgr2=0x7f4f94073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f941010e0 0x7f4f94073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 -- 172.21.15.70:0/1147067168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94106730 msgr2=0x7f4f94073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94106730 0x7f4f94073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.117+0000 7f4f93fff700 1 -- 172.21.15.70:0/1147067168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c0095f0 con 0x7f4f94100720 2023-10-18T12:26:12.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.118+0000 7f4f93fff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 0x7f4f94072af0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400e9a0 tx=0x7f4f8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:12.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.118+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84009790 con 0x7f4f94100720 2023-10-18T12:26:12.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.118+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f941a54f0 con 0x7f4f94100720 2023-10-18T12:26:12.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.118+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f84010430 con 0x7f4f94100720 2023-10-18T12:26:12.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.118+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84018620 con 0x7f4f94100720 2023-10-18T12:26:12.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.118+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f941a5a50 con 0x7f4f94100720 2023-10-18T12:26:12.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.119+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f94108d20 con 0x7f4f94100720 2023-10-18T12:26:12.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.121+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4f840105a0 con 0x7f4f94100720 2023-10-18T12:26:12.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.122+0000 7f4f91ffb700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4f800a14d0 0x7f4f800a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.122+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f4f84014070 con 0x7f4f94100720 2023-10-18T12:26:12.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.123+0000 7f4f8bfff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4f800a14d0 0x7f4f800a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.123+0000 7f4f8bfff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4f800a14d0 0x7f4f800a3990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00b6f0 tx=0x7f4f7c00b280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:12.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.126+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4f84052100 con 0x7f4f94100720 2023-10-18T12:26:12.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.335+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f94073fd0 con 0x7f4f94100720 2023-10-18T12:26:12.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.335+0000 7f4f91ffb700 1 -- 172.21.15.70:0/1147067168 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f840964c0 con 0x7f4f94100720 2023-10-18T12:26:12.339 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.337+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4f800a14d0 msgr2=0x7f4f800a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.337+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4f800a14d0 0x7f4f800a3990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00b6f0 tx=0x7f4f7c00b280 comp rx=0 tx=0).stop 2023-10-18T12:26:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.337+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 msgr2=0x7f4f94072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.337+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 0x7f4f94072af0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400e9a0 tx=0x7f4f8400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 shutdown_connections 2023-10-18T12:26:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4f800a14d0 0x7f4f800a3990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94106730 0x7f4f94073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f94100720 0x7f4f94072af0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 --2- 172.21.15.70:0/1147067168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f941010e0 0x7f4f94073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 >> 172.21.15.70:0/1147067168 conn(0x7f4f940fc1b0 msgr2=0x7f4f940fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:12.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 shutdown_connections 2023-10-18T12:26:12.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.338+0000 7f4f9a3ff700 1 -- 172.21.15.70:0/1147067168 wait complete. 2023-10-18T12:26:12.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:26:12.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:12 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2608780740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:12.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:12 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3587353234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 -- 172.21.15.70:0/1380950488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd810cf60 msgr2=0x7f8bd810f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 --2- 172.21.15.70:0/1380950488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd810cf60 0x7f8bd810f3d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000b0a0 tx=0x7f8bd000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 -- 172.21.15.70:0/1380950488 shutdown_connections 2023-10-18T12:26:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 --2- 172.21.15.70:0/1380950488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd810cf60 0x7f8bd810f3d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 --2- 172.21.15.70:0/1380950488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd8072c20 0x7f8bd8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 --2- 172.21.15.70:0/1380950488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bd8073540 0x7f8bd810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 -- 172.21.15.70:0/1380950488 >> 172.21.15.70:0/1380950488 conn(0x7f8bd80fc1b0 msgr2=0x7f8bd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:12.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 -- 172.21.15.70:0/1380950488 shutdown_connections 2023-10-18T12:26:12.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.558+0000 7f8bdedff700 1 -- 172.21.15.70:0/1380950488 wait complete. 2023-10-18T12:26:12.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.559+0000 7f8bdedff700 1 Processor -- start 2023-10-18T12:26:12.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 -- start start 2023-10-18T12:26:12.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd8072c20 0x7f8bd8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 0x7f8bd8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bd810cf60 0x7f8bd819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819de50 con 0x7f8bd8073540 2023-10-18T12:26:12.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819dfc0 con 0x7f8bd8072c20 2023-10-18T12:26:12.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdedff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819e180 con 0x7f8bd810cf60 2023-10-18T12:26:12.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bd7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 0x7f8bd8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bd7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 0x7f8bd8199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39948/0 (socket says 172.21.15.70:39948) 2023-10-18T12:26:12.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bd7fff700 1 -- 172.21.15.70:0/4004144637 learned_addr learned my addr 172.21.15.70:0/4004144637 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:12.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdd39c700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bd810cf60 0x7f8bd819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.560+0000 7f8bdcb9b700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd8072c20 0x7f8bd8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd7fff700 1 -- 172.21.15.70:0/4004144637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bd810cf60 msgr2=0x7f8bd819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd7fff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bd810cf60 0x7f8bd819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd7fff700 1 -- 172.21.15.70:0/4004144637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd8072c20 msgr2=0x7f8bd8198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd7fff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd8072c20 0x7f8bd8198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd7fff700 1 -- 172.21.15.70:0/4004144637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009d00 con 0x7f8bd8073540 2023-10-18T12:26:12.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd7fff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 0x7f8bd8199470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e9a0 tx=0x7f8bc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:12.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4009790 con 0x7f8bd8073540 2023-10-18T12:26:12.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc4010430 con 0x7f8bd8073540 2023-10-18T12:26:12.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd819e460 con 0x7f8bd8073540 2023-10-18T12:26:12.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4018680 con 0x7f8bd8073540 2023-10-18T12:26:12.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.561+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81a5ca0 con 0x7f8bd8073540 2023-10-18T12:26:12.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.562+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd8108170 con 0x7f8bd8073540 2023-10-18T12:26:12.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.564+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8bc40105a0 con 0x7f8bd8073540 2023-10-18T12:26:12.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.566+0000 7f8bd5ffb700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8bc80a1550 0x7f8bc80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:12.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.566+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f8bc4014070 con 0x7f8bd8073540 2023-10-18T12:26:12.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.566+0000 7f8bdcb9b700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8bc80a1550 0x7f8bc80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:12.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.566+0000 7f8bdcb9b700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8bc80a1550 0x7f8bc80a3a10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009c60 tx=0x7f8bcc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:12.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.569+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8bc4096470 con 0x7f8bd8073540 2023-10-18T12:26:12.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2608780740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:12.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:12 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3587353234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:12.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:12 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2608780740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:12.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:12 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3587353234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.772+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8bd819a230 con 0x7f8bd8073540 2023-10-18T12:26:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.773+0000 7f8bd5ffb700 1 -- 172.21.15.70:0/4004144637 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8bc4099a90 con 0x7f8bd8073540 2023-10-18T12:26:12.776 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:26:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8bc80a1550 msgr2=0x7f8bc80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8bc80a1550 0x7f8bc80a3a10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009c60 tx=0x7f8bcc009260 comp rx=0 tx=0).stop 2023-10-18T12:26:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 msgr2=0x7f8bd8199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 0x7f8bd8199470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e9a0 tx=0x7f8bc400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 shutdown_connections 2023-10-18T12:26:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bd810cf60 0x7f8bd819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8bc80a1550 0x7f8bc80a3a10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd8072c20 0x7f8bd8198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 --2- 172.21.15.70:0/4004144637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bd8073540 0x7f8bd8199470 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 >> 172.21.15.70:0/4004144637 conn(0x7f8bd80fc1b0 msgr2=0x7f8bd80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 shutdown_connections 2023-10-18T12:26:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.775+0000 7f8bdedff700 1 -- 172.21.15.70:0/4004144637 wait complete. 2023-10-18T12:26:12.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:26:13.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.996+0000 7fb940bff700 1 -- 172.21.15.70:0/1442770869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 msgr2=0x7fb93c10d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.996+0000 7fb940bff700 1 --2- 172.21.15.70:0/1442770869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c10d5e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b0a0 tx=0x7fb93400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 -- 172.21.15.70:0/1442770869 shutdown_connections 2023-10-18T12:26:13.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 --2- 172.21.15.70:0/1442770869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c10d5e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 --2- 172.21.15.70:0/1442770869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb93c106720 0x7fb93c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 --2- 172.21.15.70:0/1442770869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c100710 0x7fb93c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 -- 172.21.15.70:0/1442770869 >> 172.21.15.70:0/1442770869 conn(0x7fb93c0fc200 msgr2=0x7fb93c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:13.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 -- 172.21.15.70:0/1442770869 shutdown_connections 2023-10-18T12:26:13.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.997+0000 7fb940bff700 1 -- 172.21.15.70:0/1442770869 wait complete. 2023-10-18T12:26:13.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 Processor -- start 2023-10-18T12:26:13.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 -- start start 2023-10-18T12:26:13.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb93c100710 0x7fb93c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c106720 0x7fb93c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c199180 con 0x7fb93c1010d0 2023-10-18T12:26:13.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1992f0 con 0x7fb93c106720 2023-10-18T12:26:13.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb940bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1994b0 con 0x7fb93c100710 2023-10-18T12:26:13.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb939d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb939d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39960/0 (socket says 172.21.15.70:39960) 2023-10-18T12:26:13.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb939d9b700 1 -- 172.21.15.70:0/3797395035 learned_addr learned my addr 172.21.15.70:0/3797395035 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:13.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.998+0000 7fb93a59c700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb93c100710 0x7fb93c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb93ad9d700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c106720 0x7fb93c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb939d9b700 1 -- 172.21.15.70:0/3797395035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb93c100710 msgr2=0x7fb93c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb939d9b700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb93c100710 0x7fb93c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb939d9b700 1 -- 172.21.15.70:0/3797395035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c106720 msgr2=0x7fb93c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb939d9b700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c106720 0x7fb93c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb939d9b700 1 -- 172.21.15.70:0/3797395035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb934009d00 con 0x7fb93c1010d0 2023-10-18T12:26:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb93ad9d700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c106720 0x7fb93c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb939d9b700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c19ef30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb93000d980 tx=0x7fb93000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930004220 con 0x7fb93c1010d0 2023-10-18T12:26:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb930007640 con 0x7fb93c1010d0 2023-10-18T12:26:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930005610 con 0x7fb93c1010d0 2023-10-18T12:26:13.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c199790 con 0x7fb93c1010d0 2023-10-18T12:26:13.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:12.999+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c1a59e0 con 0x7fb93c1010d0 2023-10-18T12:26:13.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.000+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c101e50 con 0x7fb93c1010d0 2023-10-18T12:26:13.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.002+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb930004750 con 0x7fb93c1010d0 2023-10-18T12:26:13.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.004+0000 7fb92b7fe700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb9240a14d0 0x7fb9240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.004+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fb9300ccfe0 con 0x7fb93c1010d0 2023-10-18T12:26:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.004+0000 7fb93a59c700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb9240a14d0 0x7fb9240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.004+0000 7fb93a59c700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb9240a14d0 0x7fb9240a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009be0 tx=0x7fb92c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.008+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb930096230 con 0x7fb93c1010d0 2023-10-18T12:26:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.210+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb93c110680 con 0x7fb93c1010d0 2023-10-18T12:26:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.210+0000 7fb92b7fe700 1 -- 172.21.15.70:0/3797395035 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb930016020 con 0x7fb93c1010d0 2023-10-18T12:26:13.214 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb9240a14d0 msgr2=0x7fb9240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb9240a14d0 0x7fb9240a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009be0 tx=0x7fb92c009260 comp rx=0 tx=0).stop 2023-10-18T12:26:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 msgr2=0x7fb93c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c19ef30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb93000d980 tx=0x7fb93000dc90 comp rx=0 tx=0).stop 2023-10-18T12:26:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 shutdown_connections 2023-10-18T12:26:13.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb9240a14d0 0x7fb9240a3990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb93c106720 0x7fb93c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb93c100710 0x7fb93c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 --2- 172.21.15.70:0/3797395035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb93c1010d0 0x7fb93c19ef30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 >> 172.21.15.70:0/3797395035 conn(0x7fb93c0fc200 msgr2=0x7fb93c10c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 shutdown_connections 2023-10-18T12:26:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.213+0000 7fb940bff700 1 -- 172.21.15.70:0/3797395035 wait complete. 2023-10-18T12:26:13.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:26:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 -- 172.21.15.70:0/1438750550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 msgr2=0x7f245010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 --2- 172.21.15.70:0/1438750550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245010d570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f244800b0a0 tx=0x7f244800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 -- 172.21.15.70:0/1438750550 shutdown_connections 2023-10-18T12:26:13.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 --2- 172.21.15.70:0/1438750550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245010d570 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 --2- 172.21.15.70:0/1438750550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24501066b0 0x7f2450106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 --2- 172.21.15.70:0/1438750550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501006a0 0x7f2450100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 -- 172.21.15.70:0/1438750550 >> 172.21.15.70:0/1438750550 conn(0x7f24500fc170 msgr2=0x7f24500fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:13.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 -- 172.21.15.70:0/1438750550 shutdown_connections 2023-10-18T12:26:13.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.431+0000 7f2456bff700 1 -- 172.21.15.70:0/1438750550 wait complete. 2023-10-18T12:26:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.432+0000 7f2456bff700 1 Processor -- start 2023-10-18T12:26:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 -- start start 2023-10-18T12:26:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501006a0 0x7f245019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24501066b0 0x7f2450196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450196ee0 con 0x7f2450101060 2023-10-18T12:26:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450197050 con 0x7f24501006a0 2023-10-18T12:26:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f2456bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450197210 con 0x7f24501066b0 2023-10-18T12:26:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f244ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f244ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39974/0 (socket says 172.21.15.70:39974) 2023-10-18T12:26:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f244ffff700 1 -- 172.21.15.70:0/2914992871 learned_addr learned my addr 172.21.15.70:0/2914992871 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f245519c700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24501066b0 0x7f2450196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.433+0000 7f245499b700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501006a0 0x7f245019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244ffff700 1 -- 172.21.15.70:0/2914992871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24501066b0 msgr2=0x7f2450196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244ffff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24501066b0 0x7f2450196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244ffff700 1 -- 172.21.15.70:0/2914992871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501006a0 msgr2=0x7f245019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244ffff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501006a0 0x7f245019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244ffff700 1 -- 172.21.15.70:0/2914992871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448009d00 con 0x7f2450101060 2023-10-18T12:26:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244ffff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245019cc30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f243c00e9a0 tx=0x7f243c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c009790 con 0x7f2450101060 2023-10-18T12:26:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f243c010430 con 0x7f2450101060 2023-10-18T12:26:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24501974f0 con 0x7f2450101060 2023-10-18T12:26:13.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c018680 con 0x7f2450101060 2023-10-18T12:26:13.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.434+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24501a3790 con 0x7f2450101060 2023-10-18T12:26:13.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.435+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2450108cc0 con 0x7f2450101060 2023-10-18T12:26:13.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.437+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f243c0105a0 con 0x7f2450101060 2023-10-18T12:26:13.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.440+0000 7f244dffb700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f24400a1550 0x7f24400a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.440+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f243c014070 con 0x7f2450101060 2023-10-18T12:26:13.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.440+0000 7f245499b700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f24400a1550 0x7f24400a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.440+0000 7f245499b700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f24400a1550 0x7f24400a3a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24500fe1d0 tx=0x7f2444009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:13.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.444+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f243c096510 con 0x7f2450101060 2023-10-18T12:26:13.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:13 smithi186 ceph-mon[135253]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:26:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:13 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1147067168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:13.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:13 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4004144637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:13.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:13 smithi070 ceph-mon[98639]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:26:13.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:13 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1147067168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:13.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:13 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4004144637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:13.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.648+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2450101de0 con 0x7f2450101060 2023-10-18T12:26:13.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.650+0000 7f244dffb700 1 -- 172.21.15.70:0/2914992871 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f243c099b30 con 0x7f2450101060 2023-10-18T12:26:13.653 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:13.653 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:25:15.204744+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:26:13.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f24400a1550 msgr2=0x7f24400a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f24400a1550 0x7f24400a3a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24500fe1d0 tx=0x7f2444009260 comp rx=0 tx=0).stop 2023-10-18T12:26:13.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 msgr2=0x7f245019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245019cc30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f243c00e9a0 tx=0x7f243c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 shutdown_connections 2023-10-18T12:26:13.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24501066b0 0x7f2450196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f24400a1550 0x7f24400a3a10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501006a0 0x7f245019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 --2- 172.21.15.70:0/2914992871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2450101060 0x7f245019cc30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 >> 172.21.15.70:0/2914992871 conn(0x7f24500fc170 msgr2=0x7f24500fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:13.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 shutdown_connections 2023-10-18T12:26:13.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.652+0000 7f2456bff700 1 -- 172.21.15.70:0/2914992871 wait complete. 2023-10-18T12:26:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:13 smithi152 ceph-mon[134767]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2023-10-18T12:26:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:13 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1147067168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:13.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:13 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4004144637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:13.675 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:26:13.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:26:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 -- 172.21.15.70:0/166598182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e8100720 msgr2=0x7f30e8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 --2- 172.21.15.70:0/166598182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e8100720 0x7f30e8100ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30d80098f0 tx=0x7f30d8009c00 comp rx=0 tx=0).stop 2023-10-18T12:26:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 -- 172.21.15.70:0/166598182 shutdown_connections 2023-10-18T12:26:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 --2- 172.21.15.70:0/166598182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e81010e0 0x7f30e810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 --2- 172.21.15.70:0/166598182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30e8106730 0x7f30e8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 --2- 172.21.15.70:0/166598182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e8100720 0x7f30e8100ba0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 -- 172.21.15.70:0/166598182 >> 172.21.15.70:0/166598182 conn(0x7f30e80fc1b0 msgr2=0x7f30e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 -- 172.21.15.70:0/166598182 shutdown_connections 2023-10-18T12:26:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.876+0000 7f30f01ff700 1 -- 172.21.15.70:0/166598182 wait complete. 2023-10-18T12:26:13.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.877+0000 7f30f01ff700 1 Processor -- start 2023-10-18T12:26:13.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.877+0000 7f30f01ff700 1 -- start start 2023-10-18T12:26:13.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30f01ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e8100720 0x7f30e819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30f01ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 0x7f30e819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30f01ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30e8106730 0x7f30e8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30f01ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8197220 con 0x7f30e81010e0 2023-10-18T12:26:13.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30f01ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8197390 con 0x7f30e8106730 2023-10-18T12:26:13.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30f01ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8197550 con 0x7f30e8100720 2023-10-18T12:26:13.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 0x7f30e819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 0x7f30e819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39992/0 (socket says 172.21.15.70:39992) 2023-10-18T12:26:13.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 -- 172.21.15.70:0/401083133 learned_addr learned my addr 172.21.15.70:0/401083133 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:13.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ee79c700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30e8106730 0x7f30e8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30edf9b700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e8100720 0x7f30e819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 -- 172.21.15.70:0/401083133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e8100720 msgr2=0x7f30e819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e8100720 0x7f30e819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 -- 172.21.15.70:0/401083133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30e8106730 msgr2=0x7f30e8196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:13.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30e8106730 0x7f30e8196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:13.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 -- 172.21.15.70:0/401083133 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40095f0 con 0x7f30e81010e0 2023-10-18T12:26:13.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30edf9b700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e8100720 0x7f30e819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:13.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.878+0000 7f30ed79a700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 0x7f30e819cef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f30d8005950 tx=0x7f30d800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:13.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.879+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d801c070 con 0x7f30e81010e0 2023-10-18T12:26:13.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.879+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30d800bb20 con 0x7f30e81010e0 2023-10-18T12:26:13.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.879+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d8009520 con 0x7f30e81010e0 2023-10-18T12:26:13.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.879+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80038e0 con 0x7f30e81010e0 2023-10-18T12:26:13.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.879+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e81978e0 con 0x7f30e81010e0 2023-10-18T12:26:13.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.880+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e8108cb0 con 0x7f30e81010e0 2023-10-18T12:26:13.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.882+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f30d8020030 con 0x7f30e81010e0 2023-10-18T12:26:13.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.883+0000 7f30deffd700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30d40a1550 0x7f30d40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:13.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.883+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f30d80cdee0 con 0x7f30e81010e0 2023-10-18T12:26:13.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.883+0000 7f30edf9b700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30d40a1550 0x7f30d40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:13.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.884+0000 7f30edf9b700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30d40a1550 0x7f30d40a3a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f30e4005fd0 tx=0x7f30e4009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:13.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:13.887+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f30d8097180 con 0x7f30e81010e0 2023-10-18T12:26:14.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.086+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30e8197f30 con 0x7f30e81010e0 2023-10-18T12:26:14.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.086+0000 7f30deffd700 1 -- 172.21.15.70:0/401083133 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30d809a7a0 con 0x7f30e81010e0 2023-10-18T12:26:14.090 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:26:14.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30d40a1550 msgr2=0x7f30d40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30d40a1550 0x7f30d40a3a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f30e4005fd0 tx=0x7f30e4009310 comp rx=0 tx=0).stop 2023-10-18T12:26:14.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 msgr2=0x7f30e819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 0x7f30e819cef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f30d8005950 tx=0x7f30d800fa00 comp rx=0 tx=0).stop 2023-10-18T12:26:14.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 shutdown_connections 2023-10-18T12:26:14.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f30d40a1550 0x7f30d40a3a10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30e8106730 0x7f30e8196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30e8100720 0x7f30e819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 --2- 172.21.15.70:0/401083133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e81010e0 0x7f30e819cef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 >> 172.21.15.70:0/401083133 conn(0x7f30e80fc1b0 msgr2=0x7f30e80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:14.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 shutdown_connections 2023-10-18T12:26:14.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.089+0000 7f30f01ff700 1 -- 172.21.15.70:0/401083133 wait complete. 2023-10-18T12:26:14.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:26:14.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.313+0000 7f36a53ff700 1 -- 172.21.15.70:0/2499146409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0106730 msgr2=0x7f36a0106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.313+0000 7f36a53ff700 1 --2- 172.21.15.70:0/2499146409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0106730 0x7f36a0106b10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3690009990 tx=0x7f3690009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 -- 172.21.15.70:0/2499146409 shutdown_connections 2023-10-18T12:26:14.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 --2- 172.21.15.70:0/2499146409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a01010e0 0x7f36a010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 --2- 172.21.15.70:0/2499146409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0106730 0x7f36a0106b10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 --2- 172.21.15.70:0/2499146409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36a0100720 0x7f36a0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 -- 172.21.15.70:0/2499146409 >> 172.21.15.70:0/2499146409 conn(0x7f36a00fc1b0 msgr2=0x7f36a00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 -- 172.21.15.70:0/2499146409 shutdown_connections 2023-10-18T12:26:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.314+0000 7f36a53ff700 1 -- 172.21.15.70:0/2499146409 wait complete. 2023-10-18T12:26:14.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 Processor -- start 2023-10-18T12:26:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 -- start start 2023-10-18T12:26:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 0x7f36a019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36a01010e0 0x7f36a019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0106730 0x7f36a0196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0196ff0 con 0x7f36a0100720 2023-10-18T12:26:14.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0197160 con 0x7f36a0106730 2023-10-18T12:26:14.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f36a53ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0197320 con 0x7f36a01010e0 2023-10-18T12:26:14.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f369effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 0x7f36a019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f369effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 0x7f36a019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40002/0 (socket says 172.21.15.70:40002) 2023-10-18T12:26:14.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.315+0000 7f369effd700 1 -- 172.21.15.70:0/1247960613 learned_addr learned my addr 172.21.15.70:0/1247960613 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369e7fc700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36a01010e0 0x7f36a019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369f7fe700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0106730 0x7f36a0196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369effd700 1 -- 172.21.15.70:0/1247960613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36a01010e0 msgr2=0x7f36a019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369effd700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36a01010e0 0x7f36a019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369effd700 1 -- 172.21.15.70:0/1247960613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0106730 msgr2=0x7f36a0196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369effd700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0106730 0x7f36a0196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369effd700 1 -- 172.21.15.70:0/1247960613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36900095f0 con 0x7f36a0100720 2023-10-18T12:26:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369f7fe700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0106730 0x7f36a0196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f369effd700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 0x7f36a019c860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3690000c00 tx=0x7f3690004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369001d070 con 0x7f36a0100720 2023-10-18T12:26:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a01975a0 con 0x7f36a0100720 2023-10-18T12:26:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3690022470 con 0x7f36a0100720 2023-10-18T12:26:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690017650 con 0x7f36a0100720 2023-10-18T12:26:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.316+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a01a3850 con 0x7f36a0100720 2023-10-18T12:26:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.317+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a0108d20 con 0x7f36a0100720 2023-10-18T12:26:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.321+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f36900225e0 con 0x7f36a0100720 2023-10-18T12:26:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.322+0000 7f3687fff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36880a14d0 0x7f36880a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.322+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f36900ceb80 con 0x7f36a0100720 2023-10-18T12:26:14.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.322+0000 7f369e7fc700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36880a14d0 0x7f36880a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.323+0000 7f369e7fc700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36880a14d0 0x7f36880a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36a0197b50 tx=0x7f3694009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:14.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.324+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3690097e20 con 0x7f36a0100720 2023-10-18T12:26:14.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:14 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3797395035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:14 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2914992871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:14.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:14 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/401083133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:14.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.526+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36a0197e50 con 0x7f36a0100720 2023-10-18T12:26:14.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.527+0000 7f3687fff700 1 -- 172.21.15.70:0/1247960613 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f369009b440 con 0x7f36a0100720 2023-10-18T12:26:14.530 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:14.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36880a14d0 msgr2=0x7f36880a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36880a14d0 0x7f36880a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36a0197b50 tx=0x7f3694009260 comp rx=0 tx=0).stop 2023-10-18T12:26:14.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 msgr2=0x7f36a019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 0x7f36a019c860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3690000c00 tx=0x7f3690004730 comp rx=0 tx=0).stop 2023-10-18T12:26:14.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 shutdown_connections 2023-10-18T12:26:14.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f36880a14d0 0x7f36880a3990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0106730 0x7f36a0196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a0100720 0x7f36a019c860 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 --2- 172.21.15.70:0/1247960613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36a01010e0 0x7f36a019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 >> 172.21.15.70:0/1247960613 conn(0x7f36a00fc1b0 msgr2=0x7f36a00fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:14.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 shutdown_connections 2023-10-18T12:26:14.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.529+0000 7f36a53ff700 1 -- 172.21.15.70:0/1247960613 wait complete. 2023-10-18T12:26:14.548 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3797395035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:14.549 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2914992871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:14.549 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:14 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/401083133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:14.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:26:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:14 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3797395035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:14 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2914992871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:14.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:14 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/401083133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:14.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 -- 172.21.15.70:0/2527525697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 msgr2=0x7f170c106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 --2- 172.21.15.70:0/2527525697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c106a20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1700009990 tx=0x7f1700009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 -- 172.21.15.70:0/2527525697 shutdown_connections 2023-10-18T12:26:14.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 --2- 172.21.15.70:0/2527525697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c10d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 --2- 172.21.15.70:0/2527525697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c106a20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 --2- 172.21.15.70:0/2527525697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f170c100630 0x7f170c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 -- 172.21.15.70:0/2527525697 >> 172.21.15.70:0/2527525697 conn(0x7f170c0fc160 msgr2=0x7f170c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:14.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 -- 172.21.15.70:0/2527525697 shutdown_connections 2023-10-18T12:26:14.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.749+0000 7f1712fff700 1 -- 172.21.15.70:0/2527525697 wait complete. 2023-10-18T12:26:14.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.750+0000 7f1712fff700 1 Processor -- start 2023-10-18T12:26:14.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.750+0000 7f1712fff700 1 -- start start 2023-10-18T12:26:14.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.750+0000 7f1712fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f170c100630 0x7f170c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f1712fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f1712fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f1712fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c19b920 con 0x7f170c100ff0 2023-10-18T12:26:14.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f1712fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c19ba90 con 0x7f170c106640 2023-10-18T12:26:14.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f1712fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c19bc50 con 0x7f170c100630 2023-10-18T12:26:14.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40028/0 (socket says 172.21.15.70:40028) 2023-10-18T12:26:14.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 -- 172.21.15.70:0/2275122394 learned_addr learned my addr 172.21.15.70:0/2275122394 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:14.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f1710d9b700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f170c100630 0x7f170c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f171159c700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 -- 172.21.15.70:0/2275122394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f170c100630 msgr2=0x7f170c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f170c100630 0x7f170c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 -- 172.21.15.70:0/2275122394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 msgr2=0x7f170c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 -- 172.21.15.70:0/2275122394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17000095f0 con 0x7f170c100ff0 2023-10-18T12:26:14.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f171159c700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:14.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.751+0000 7f170bfff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c196f80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9a0 tx=0x7f16f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:14.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.752+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8009790 con 0x7f170c100ff0 2023-10-18T12:26:14.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.752+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c19bf30 con 0x7f170c100ff0 2023-10-18T12:26:14.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.752+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16f800cbd0 con 0x7f170c100ff0 2023-10-18T12:26:14.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.752+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8018680 con 0x7f170c100ff0 2023-10-18T12:26:14.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.752+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c1a37e0 con 0x7f170c100ff0 2023-10-18T12:26:14.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.753+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c108bc0 con 0x7f170c100ff0 2023-10-18T12:26:14.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.755+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f16f800cd40 con 0x7f170c100ff0 2023-10-18T12:26:14.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.756+0000 7f1709ffb700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16fc0a14c0 0x7f16fc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:14.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.756+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f16f8014070 con 0x7f170c100ff0 2023-10-18T12:26:14.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.756+0000 7f1710d9b700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16fc0a14c0 0x7f16fc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.757+0000 7f1710d9b700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16fc0a14c0 0x7f16fc0a3980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1700000c00 tx=0x7f1700005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.758+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f16f8096910 con 0x7f170c100ff0 2023-10-18T12:26:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.960+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f170c197d40 con 0x7f170c100ff0 2023-10-18T12:26:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.960+0000 7f1709ffb700 1 -- 172.21.15.70:0/2275122394 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16f8099f30 con 0x7f170c100ff0 2023-10-18T12:26:14.964 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:26:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.962+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16fc0a14c0 msgr2=0x7f16fc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.962+0000 7f1712fff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16fc0a14c0 0x7f16fc0a3980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1700000c00 tx=0x7f1700005960 comp rx=0 tx=0).stop 2023-10-18T12:26:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.962+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 msgr2=0x7f170c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.962+0000 7f1712fff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c196f80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9a0 tx=0x7f16f800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 shutdown_connections 2023-10-18T12:26:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16fc0a14c0 0x7f16fc0a3980 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f170c106640 0x7f170c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f170c100630 0x7f170c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 --2- 172.21.15.70:0/2275122394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f170c100ff0 0x7f170c196f80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 >> 172.21.15.70:0/2275122394 conn(0x7f170c0fc160 msgr2=0x7f170c0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:14.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 shutdown_connections 2023-10-18T12:26:14.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:14.963+0000 7f1712fff700 1 -- 172.21.15.70:0/2275122394 wait complete. 2023-10-18T12:26:14.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:26:15.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.182+0000 7f35333ff700 1 -- 172.21.15.70:0/3485571220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c072c20 msgr2=0x7f352c073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.182+0000 7f35333ff700 1 --2- 172.21.15.70:0/3485571220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c072c20 0x7f352c073000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f352800b0a0 tx=0x7f352800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 -- 172.21.15.70:0/3485571220 shutdown_connections 2023-10-18T12:26:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 --2- 172.21.15.70:0/3485571220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f352c10cf50 0x7f352c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 --2- 172.21.15.70:0/3485571220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c072c20 0x7f352c073000 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 --2- 172.21.15.70:0/3485571220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c073540 0x7f352c10ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 -- 172.21.15.70:0/3485571220 >> 172.21.15.70:0/3485571220 conn(0x7f352c0fc200 msgr2=0x7f352c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 -- 172.21.15.70:0/3485571220 shutdown_connections 2023-10-18T12:26:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.183+0000 7f35333ff700 1 -- 172.21.15.70:0/3485571220 wait complete. 2023-10-18T12:26:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 Processor -- start 2023-10-18T12:26:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 -- start start 2023-10-18T12:26:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c072c20 0x7f352c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 0x7f352c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f352c10cf50 0x7f352c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19de30 con 0x7f352c073540 2023-10-18T12:26:15.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19dfa0 con 0x7f352c072c20 2023-10-18T12:26:15.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f35333ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19e160 con 0x7f352c10cf50 2023-10-18T12:26:15.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f353099a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 0x7f352c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f353099a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 0x7f352c199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40046/0 (socket says 172.21.15.70:40046) 2023-10-18T12:26:15.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.184+0000 7f353099a700 1 -- 172.21.15.70:0/3505015206 learned_addr learned my addr 172.21.15.70:0/3505015206 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:15.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353119b700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c072c20 0x7f352c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353199c700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f352c10cf50 0x7f352c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353099a700 1 -- 172.21.15.70:0/3505015206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f352c10cf50 msgr2=0x7f352c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353099a700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f352c10cf50 0x7f352c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353099a700 1 -- 172.21.15.70:0/3505015206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c072c20 msgr2=0x7f352c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353099a700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c072c20 0x7f352c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353099a700 1 -- 172.21.15.70:0/3505015206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528009d00 con 0x7f352c073540 2023-10-18T12:26:15.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f353099a700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 0x7f352c199450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e9a0 tx=0x7f351c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:15.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c009790 con 0x7f352c073540 2023-10-18T12:26:15.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351c00cbd0 con 0x7f352c073540 2023-10-18T12:26:15.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c19e440 con 0x7f352c073540 2023-10-18T12:26:15.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c018680 con 0x7f352c073540 2023-10-18T12:26:15.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.185+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c1a5c80 con 0x7f352c073540 2023-10-18T12:26:15.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.187+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c108160 con 0x7f352c073540 2023-10-18T12:26:15.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.188+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f351c00cd40 con 0x7f352c073540 2023-10-18T12:26:15.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.190+0000 7f35227fc700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f35180a14d0 0x7f35180a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.190+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f351c014070 con 0x7f352c073540 2023-10-18T12:26:15.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.190+0000 7f353119b700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f35180a14d0 0x7f35180a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.191+0000 7f353119b700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f35180a14d0 0x7f35180a3990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f352800b780 tx=0x7f3528006210 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:15.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.194+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f351c052100 con 0x7f352c073540 2023-10-18T12:26:15.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:15 smithi070 ceph-mon[98639]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:26:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-10-18T12:26:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:15 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-10-18T12:26:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:15 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1247960613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:15 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2275122394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.400+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f352c19a210 con 0x7f352c073540 2023-10-18T12:26:15.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.400+0000 7f35227fc700 1 -- 172.21.15.70:0/3505015206 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f351c096660 con 0x7f352c073540 2023-10-18T12:26:15.403 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:15.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.402+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f35180a14d0 msgr2=0x7f35180a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.402+0000 7f35333ff700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f35180a14d0 0x7f35180a3990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f352800b780 tx=0x7f3528006210 comp rx=0 tx=0).stop 2023-10-18T12:26:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.402+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 msgr2=0x7f352c199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.402+0000 7f35333ff700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 0x7f352c199450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e9a0 tx=0x7f351c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 shutdown_connections 2023-10-18T12:26:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f352c10cf50 0x7f352c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f35180a14d0 0x7f35180a3990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c072c20 0x7f352c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 --2- 172.21.15.70:0/3505015206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f352c073540 0x7f352c199450 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 >> 172.21.15.70:0/3505015206 conn(0x7f352c0fc200 msgr2=0x7f352c0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 shutdown_connections 2023-10-18T12:26:15.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.403+0000 7f35333ff700 1 -- 172.21.15.70:0/3505015206 wait complete. 2023-10-18T12:26:15.425 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-18T12:26:15.426 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:26:15.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:15 smithi186 ceph-mon[135253]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:26:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-10-18T12:26:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:15 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-10-18T12:26:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:15 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1247960613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:15.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:15 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2275122394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:15.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.620+0000 7f18b2fff700 1 -- 172.21.15.70:0/3889757172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac101060 msgr2=0x7f18ac10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.620+0000 7f18b2fff700 1 --2- 172.21.15.70:0/3889757172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac101060 0x7f18ac10d570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b0a0 tx=0x7f18a400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 -- 172.21.15.70:0/3889757172 shutdown_connections 2023-10-18T12:26:15.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 --2- 172.21.15.70:0/3889757172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac101060 0x7f18ac10d570 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 --2- 172.21.15.70:0/3889757172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac1066b0 0x7f18ac106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 --2- 172.21.15.70:0/3889757172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18ac1006a0 0x7f18ac100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 -- 172.21.15.70:0/3889757172 >> 172.21.15.70:0/3889757172 conn(0x7f18ac0fc1d0 msgr2=0x7f18ac0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:15.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 -- 172.21.15.70:0/3889757172 shutdown_connections 2023-10-18T12:26:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.621+0000 7f18b2fff700 1 -- 172.21.15.70:0/3889757172 wait complete. 2023-10-18T12:26:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 Processor -- start 2023-10-18T12:26:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 -- start start 2023-10-18T12:26:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac1006a0 0x7f18ac072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 0x7f18ac073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18ac1066b0 0x7f18ac073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac1a4ef0 con 0x7f18ac1006a0 2023-10-18T12:26:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac1a5060 con 0x7f18ac1066b0 2023-10-18T12:26:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b2fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac1a5220 con 0x7f18ac101060 2023-10-18T12:26:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18b0d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac1006a0 0x7f18ac072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18abfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 0x7f18ac073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18abfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 0x7f18ac073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:59018/0 (socket says 172.21.15.70:59018) 2023-10-18T12:26:15.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.622+0000 7f18abfff700 1 -- 172.21.15.70:0/1928102591 learned_addr learned my addr 172.21.15.70:0/1928102591 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:15.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18b159c700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18ac1066b0 0x7f18ac073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18abfff700 1 -- 172.21.15.70:0/1928102591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18ac1066b0 msgr2=0x7f18ac073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18abfff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18ac1066b0 0x7f18ac073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18abfff700 1 -- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac1006a0 msgr2=0x7f18ac072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18abfff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac1006a0 0x7f18ac072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18abfff700 1 -- 172.21.15.70:0/1928102591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a4009d00 con 0x7f18ac101060 2023-10-18T12:26:15.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18b0d9b700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac1006a0 0x7f18ac072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:15.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.623+0000 7f18abfff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 0x7f18ac073030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f189800e9a0 tx=0x7f189800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:15.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.624+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898009790 con 0x7f18ac101060 2023-10-18T12:26:15.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.624+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1898010430 con 0x7f18ac101060 2023-10-18T12:26:15.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.624+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac1a5500 con 0x7f18ac101060 2023-10-18T12:26:15.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.624+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898018680 con 0x7f18ac101060 2023-10-18T12:26:15.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.624+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac1a59e0 con 0x7f18ac101060 2023-10-18T12:26:15.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.625+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac108ca0 con 0x7f18ac101060 2023-10-18T12:26:15.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.627+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f18980105a0 con 0x7f18ac101060 2023-10-18T12:26:15.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.628+0000 7f18a9ffb700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f189c0a14d0 0x7f189c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:15.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.628+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1898014070 con 0x7f18ac101060 2023-10-18T12:26:15.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.629+0000 7f18b0d9b700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f189c0a14d0 0x7f189c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:15.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.629+0000 7f18b0d9b700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f189c0a14d0 0x7f189c0a3990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009fd0 tx=0x7f18a0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:15.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.631+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1898096760 con 0x7f18ac101060 2023-10-18T12:26:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:15 smithi152 ceph-mon[134767]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2023-10-18T12:26:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-10-18T12:26:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:15 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-10-18T12:26:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:15 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1247960613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:15 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2275122394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:15.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.876+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18ac073fd0 con 0x7f18ac101060 2023-10-18T12:26:15.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.877+0000 7f18a9ffb700 1 -- 172.21.15.70:0/1928102591 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f1898026090 con 0x7f18ac101060 2023-10-18T12:26:15.882 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:15.883 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:26:15.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f189c0a14d0 msgr2=0x7f189c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f189c0a14d0 0x7f189c0a3990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009fd0 tx=0x7f18a0009260 comp rx=0 tx=0).stop 2023-10-18T12:26:15.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 msgr2=0x7f18ac073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:15.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 0x7f18ac073030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f189800e9a0 tx=0x7f189800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 shutdown_connections 2023-10-18T12:26:15.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f189c0a14d0 0x7f189c0a3990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18ac1066b0 0x7f18ac073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac1006a0 0x7f18ac072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 --2- 172.21.15.70:0/1928102591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18ac101060 0x7f18ac073030 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:15.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 >> 172.21.15.70:0/1928102591 conn(0x7f18ac0fc1d0 msgr2=0x7f18ac0fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 shutdown_connections 2023-10-18T12:26:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:15.880+0000 7f18b2fff700 1 -- 172.21.15.70:0/1928102591 wait complete. 2023-10-18T12:26:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:26:15.902 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-18T12:26:15.902 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-18T12:26:15.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:26:16.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 -- 172.21.15.70:0/1636347702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab41066b0 msgr2=0x7faab4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 --2- 172.21.15.70:0/1636347702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab41066b0 0x7faab4106a90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faab0009990 tx=0x7faab0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 -- 172.21.15.70:0/1636347702 shutdown_connections 2023-10-18T12:26:16.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 --2- 172.21.15.70:0/1636347702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faab4101060 0x7faab410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 --2- 172.21.15.70:0/1636347702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab41066b0 0x7faab4106a90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 --2- 172.21.15.70:0/1636347702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab41006a0 0x7faab4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 -- 172.21.15.70:0/1636347702 >> 172.21.15.70:0/1636347702 conn(0x7faab40fc170 msgr2=0x7faab40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:16.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 -- 172.21.15.70:0/1636347702 shutdown_connections 2023-10-18T12:26:16.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.098+0000 7faabb3ff700 1 -- 172.21.15.70:0/1636347702 wait complete. 2023-10-18T12:26:16.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 Processor -- start 2023-10-18T12:26:16.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 -- start start 2023-10-18T12:26:16.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab41006a0 0x7faab419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 0x7faab419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faab41066b0 0x7faab4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4199120 con 0x7faab4101060 2023-10-18T12:26:16.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4199290 con 0x7faab41066b0 2023-10-18T12:26:16.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.099+0000 7faabb3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4199450 con 0x7faab41006a0 2023-10-18T12:26:16.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 0x7faab419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 0x7faab419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40076/0 (socket says 172.21.15.70:40076) 2023-10-18T12:26:16.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 -- 172.21.15.70:0/3800923721 learned_addr learned my addr 172.21.15.70:0/3800923721 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:16.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab919b700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab41006a0 0x7faab419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab999c700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faab41066b0 0x7faab4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 -- 172.21.15.70:0/3800923721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab41006a0 msgr2=0x7faab419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab41006a0 0x7faab419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 -- 172.21.15.70:0/3800923721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faab41066b0 msgr2=0x7faab4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faab41066b0 0x7faab4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 -- 172.21.15.70:0/3800923721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00095f0 con 0x7faab4101060 2023-10-18T12:26:16.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.100+0000 7faab899a700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 0x7faab419eed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e9a0 tx=0x7faaa400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:16.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.101+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4009790 con 0x7faab4101060 2023-10-18T12:26:16.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.101+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaa4010430 con 0x7faab4101060 2023-10-18T12:26:16.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.101+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4018680 con 0x7faab4101060 2023-10-18T12:26:16.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.101+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab4199730 con 0x7faab4101060 2023-10-18T12:26:16.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.101+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab41a5980 con 0x7faab4101060 2023-10-18T12:26:16.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.102+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab4101de0 con 0x7faab4101060 2023-10-18T12:26:16.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.104+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faaa40105a0 con 0x7faab4101060 2023-10-18T12:26:16.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.105+0000 7faaaa7fc700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaa00a1550 0x7faaa00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.105+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7faaa4014070 con 0x7faab4101060 2023-10-18T12:26:16.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.105+0000 7faab919b700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaa00a1550 0x7faaa00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.106+0000 7faab919b700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaa00a1550 0x7faaa00a3a10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faab00098e0 tx=0x7faab0005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:16.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.109+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faaa4096510 con 0x7faab4101060 2023-10-18T12:26:16.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.310+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faab41105c0 con 0x7faaa00a1550 2023-10-18T12:26:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.324+0000 7faaaa7fc700 1 -- 172.21.15.70:0/3800923721 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427323 (secure 0 0 0) 0x7faab41105c0 con 0x7faaa00a1550 2023-10-18T12:26:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:16.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.328+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaa00a1550 msgr2=0x7faaa00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.328+0000 7faabb3ff700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaa00a1550 0x7faaa00a3a10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faab00098e0 tx=0x7faab0005960 comp rx=0 tx=0).stop 2023-10-18T12:26:16.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.328+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 msgr2=0x7faab419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.328+0000 7faabb3ff700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 0x7faab419eed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e9a0 tx=0x7faaa400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 shutdown_connections 2023-10-18T12:26:16.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7faaa00a1550 0x7faaa00a3a10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faab41066b0 0x7faab4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faab41006a0 0x7faab419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 --2- 172.21.15.70:0/3800923721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab4101060 0x7faab419eed0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 >> 172.21.15.70:0/3800923721 conn(0x7faab40fc170 msgr2=0x7faab410b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:16.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 shutdown_connections 2023-10-18T12:26:16.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.329+0000 7faabb3ff700 1 -- 172.21.15.70:0/3800923721 wait complete. 2023-10-18T12:26:16.339 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:26:16.351 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-18T12:26:16.214128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652862,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":347,"num_read_kb":287,"num_write":264,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338200,"kb_used_data":14004,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603624,"statfs":{"total":1151940427776,"available":1151594110976,"internally_reserved":0,"allocated":14340096,"data_stored":8113568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.591689"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650976+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:25:43.650976+0000","last_peered":"2023-10-18T12:25:43.650976+0000","last_clean":"2023-10-18T12:25:43.650976+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:25:43.650976+0000","last_undegraded":"2023-10-18T12:25:43.650976+0000","last_fullsized":"2023-10-18T12:25:43.650976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649821+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:25:43.649821+0000","last_peered":"2023-10-18T12:25:43.649821+0000","last_clean":"2023-10-18T12:25:43.649821+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:25:43.649821+0000","last_undegraded":"2023-10-18T12:25:43.649821+0000","last_fullsized":"2023-10-18T12:25:43.649821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650939+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:25:43.650939+0000","last_peered":"2023-10-18T12:25:43.650939+0000","last_clean":"2023-10-18T12:25:43.650939+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:25:43.650939+0000","last_undegraded":"2023-10-18T12:25:43.650939+0000","last_fullsized":"2023-10-18T12:25:43.650939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.640858+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:25:43.640858+0000","last_peered":"2023-10-18T12:25:43.640858+0000","last_clean":"2023-10-18T12:25:43.640858+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:25:43.640858+0000","last_undegraded":"2023-10-18T12:25:43.640858+0000","last_fullsized":"2023-10-18T12:25:43.640858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.631268+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:25:43.631268+0000","last_peered":"2023-10-18T12:25:43.631268+0000","last_clean":"2023-10-18T12:25:43.631268+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:25:43.631268+0000","last_undegraded":"2023-10-18T12:25:43.631268+0000","last_fullsized":"2023-10-18T12:25:43.631268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643204+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:25:43.643204+0000","last_peered":"2023-10-18T12:25:43.643204+0000","last_clean":"2023-10-18T12:25:43.643204+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:25:43.643204+0000","last_undegraded":"2023-10-18T12:25:43.643204+0000","last_fullsized":"2023-10-18T12:25:43.643204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639863+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:43.639863+0000","last_peered":"2023-10-18T12:25:43.639863+0000","last_clean":"2023-10-18T12:25:43.639863+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:43.639863+0000","last_undegraded":"2023-10-18T12:25:43.639863+0000","last_fullsized":"2023-10-18T12:25:43.639863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639818+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:25:43.639818+0000","last_peered":"2023-10-18T12:25:43.639818+0000","last_clean":"2023-10-18T12:25:43.639818+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:25:43.639818+0000","last_undegraded":"2023-10-18T12:25:43.639818+0000","last_fullsized":"2023-10-18T12:25:43.639818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642671+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:25:43.642671+0000","last_peered":"2023-10-18T12:25:43.642671+0000","last_clean":"2023-10-18T12:25:43.642671+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:25:43.642671+0000","last_undegraded":"2023-10-18T12:25:43.642671+0000","last_fullsized":"2023-10-18T12:25:43.642671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.645379+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:25:43.645379+0000","last_peered":"2023-10-18T12:25:43.645379+0000","last_clean":"2023-10-18T12:25:43.645379+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:25:43.645379+0000","last_undegraded":"2023-10-18T12:25:43.645379+0000","last_fullsized":"2023-10-18T12:25:43.645379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644261+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:43.644261+0000","last_peered":"2023-10-18T12:25:43.644261+0000","last_clean":"2023-10-18T12:25:43.644261+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:43.644261+0000","last_undegraded":"2023-10-18T12:25:43.644261+0000","last_fullsized":"2023-10-18T12:25:43.644261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633544+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:43.633544+0000","last_peered":"2023-10-18T12:25:43.633544+0000","last_clean":"2023-10-18T12:25:43.633544+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:43.633544+0000","last_undegraded":"2023-10-18T12:25:43.633544+0000","last_fullsized":"2023-10-18T12:25:43.633544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.658873+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:25:43.658873+0000","last_peered":"2023-10-18T12:25:43.658873+0000","last_clean":"2023-10-18T12:25:43.658873+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:25:43.658873+0000","last_undegraded":"2023-10-18T12:25:43.658873+0000","last_fullsized":"2023-10-18T12:25:43.658873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646839+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:25:43.646839+0000","last_peered":"2023-10-18T12:25:43.646839+0000","last_clean":"2023-10-18T12:25:43.646839+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:25:43.646839+0000","last_undegraded":"2023-10-18T12:25:43.646839+0000","last_fullsized":"2023-10-18T12:25:43.646839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649694+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:25:43.649694+0000","last_peered":"2023-10-18T12:25:43.649694+0000","last_clean":"2023-10-18T12:25:43.649694+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:25:43.649694+0000","last_undegraded":"2023-10-18T12:25:43.649694+0000","last_fullsized":"2023-10-18T12:25:43.649694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648146+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:25:43.648146+0000","last_peered":"2023-10-18T12:25:43.648146+0000","last_clean":"2023-10-18T12:25:43.648146+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:25:43.648146+0000","last_undegraded":"2023-10-18T12:25:43.648146+0000","last_fullsized":"2023-10-18T12:25:43.648146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652031+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:25:43.652031+0000","last_peered":"2023-10-18T12:25:43.652031+0000","last_clean":"2023-10-18T12:25:43.652031+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:25:43.652031+0000","last_undegraded":"2023-10-18T12:25:43.652031+0000","last_fullsized":"2023-10-18T12:25:43.652031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639903+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:25:43.639903+0000","last_peered":"2023-10-18T12:25:43.639903+0000","last_clean":"2023-10-18T12:25:43.639903+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:25:43.639903+0000","last_undegraded":"2023-10-18T12:25:43.639903+0000","last_fullsized":"2023-10-18T12:25:43.639903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643796+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:25:43.643796+0000","last_peered":"2023-10-18T12:25:43.643796+0000","last_clean":"2023-10-18T12:25:43.643796+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:25:43.643796+0000","last_undegraded":"2023-10-18T12:25:43.643796+0000","last_fullsized":"2023-10-18T12:25:43.643796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212928+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:25:14.212928+0000","last_peered":"2023-10-18T12:25:14.212928+0000","last_clean":"2023-10-18T12:25:14.212928+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:25:14.212928+0000","last_undegraded":"2023-10-18T12:25:14.212928+0000","last_fullsized":"2023-10-18T12:25:14.212928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213691+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:26:15.213691+0000","last_peered":"2023-10-18T12:26:15.213691+0000","last_clean":"2023-10-18T12:26:15.213691+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:26:15.213691+0000","last_undegraded":"2023-10-18T12:26:15.213691+0000","last_fullsized":"2023-10-18T12:26:15.213691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639283+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:25:43.639283+0000","last_peered":"2023-10-18T12:25:43.639283+0000","last_clean":"2023-10-18T12:25:43.639283+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:25:43.639283+0000","last_undegraded":"2023-10-18T12:25:43.639283+0000","last_fullsized":"2023-10-18T12:25:43.639283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210737+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:14.210737+0000","last_peered":"2023-10-18T12:25:14.210737+0000","last_clean":"2023-10-18T12:25:14.210737+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:14.210737+0000","last_undegraded":"2023-10-18T12:25:14.210737+0000","last_fullsized":"2023-10-18T12:25:14.210737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.216358+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:25:15.216358+0000","last_peered":"2023-10-18T12:25:15.216358+0000","last_clean":"2023-10-18T12:25:15.216358+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:25:15.216358+0000","last_undegraded":"2023-10-18T12:25:15.216358+0000","last_fullsized":"2023-10-18T12:25:15.216358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656048+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:25:43.656048+0000","last_peered":"2023-10-18T12:25:43.656048+0000","last_clean":"2023-10-18T12:25:43.656048+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:25:43.656048+0000","last_undegraded":"2023-10-18T12:25:43.656048+0000","last_fullsized":"2023-10-18T12:25:43.656048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211459+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:14.211459+0000","last_peered":"2023-10-18T12:25:14.211459+0000","last_clean":"2023-10-18T12:25:14.211459+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:14.211459+0000","last_undegraded":"2023-10-18T12:25:14.211459+0000","last_fullsized":"2023-10-18T12:25:14.211459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215742+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:25:14.215742+0000","last_peered":"2023-10-18T12:25:14.215742+0000","last_clean":"2023-10-18T12:25:14.215742+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:25:14.215742+0000","last_undegraded":"2023-10-18T12:25:14.215742+0000","last_fullsized":"2023-10-18T12:25:14.215742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210540+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:14.210540+0000","last_peered":"2023-10-18T12:25:14.210540+0000","last_clean":"2023-10-18T12:25:14.210540+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:14.210540+0000","last_undegraded":"2023-10-18T12:25:14.210540+0000","last_fullsized":"2023-10-18T12:25:14.210540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216726+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:25:14.216726+0000","last_peered":"2023-10-18T12:25:14.216726+0000","last_clean":"2023-10-18T12:25:14.216726+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:25:14.216726+0000","last_undegraded":"2023-10-18T12:25:14.216726+0000","last_fullsized":"2023-10-18T12:25:14.216726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644740+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:25:43.644740+0000","last_peered":"2023-10-18T12:25:43.644740+0000","last_clean":"2023-10-18T12:25:43.644740+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:25:43.644740+0000","last_undegraded":"2023-10-18T12:25:43.644740+0000","last_fullsized":"2023-10-18T12:25:43.644740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217322+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:25:15.217322+0000","last_peered":"2023-10-18T12:25:15.217322+0000","last_clean":"2023-10-18T12:25:15.217322+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:25:15.217322+0000","last_undegraded":"2023-10-18T12:25:15.217322+0000","last_fullsized":"2023-10-18T12:25:15.217322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211310+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:25:14.211310+0000","last_peered":"2023-10-18T12:25:14.211310+0000","last_clean":"2023-10-18T12:25:14.211310+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:25:14.211310+0000","last_undegraded":"2023-10-18T12:25:14.211310+0000","last_fullsized":"2023-10-18T12:25:14.211310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633052+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:25:43.633052+0000","last_peered":"2023-10-18T12:25:43.633052+0000","last_clean":"2023-10-18T12:25:43.633052+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:25:43.633052+0000","last_undegraded":"2023-10-18T12:25:43.633052+0000","last_fullsized":"2023-10-18T12:25:43.633052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211137+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:25:14.211137+0000","last_peered":"2023-10-18T12:25:14.211137+0000","last_clean":"2023-10-18T12:25:14.211137+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:25:14.211137+0000","last_undegraded":"2023-10-18T12:25:14.211137+0000","last_fullsized":"2023-10-18T12:25:14.211137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210401+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:25:14.210401+0000","last_peered":"2023-10-18T12:25:14.210401+0000","last_clean":"2023-10-18T12:25:14.210401+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:25:14.210401+0000","last_undegraded":"2023-10-18T12:25:14.210401+0000","last_fullsized":"2023-10-18T12:25:14.210401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.645401+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:25:43.645401+0000","last_peered":"2023-10-18T12:25:43.645401+0000","last_clean":"2023-10-18T12:25:43.645401+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:25:43.645401+0000","last_undegraded":"2023-10-18T12:25:43.645401+0000","last_fullsized":"2023-10-18T12:25:43.645401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215355+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:25:14.215355+0000","last_peered":"2023-10-18T12:25:14.215355+0000","last_clean":"2023-10-18T12:25:14.215355+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:25:14.215355+0000","last_undegraded":"2023-10-18T12:25:14.215355+0000","last_fullsized":"2023-10-18T12:25:14.215355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207597+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:25:14.207597+0000","last_peered":"2023-10-18T12:25:14.207597+0000","last_clean":"2023-10-18T12:25:14.207597+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:25:14.207597+0000","last_undegraded":"2023-10-18T12:25:14.207597+0000","last_fullsized":"2023-10-18T12:25:14.207597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653839+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:25:43.653839+0000","last_peered":"2023-10-18T12:25:43.653839+0000","last_clean":"2023-10-18T12:25:43.653839+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:25:43.653839+0000","last_undegraded":"2023-10-18T12:25:43.653839+0000","last_fullsized":"2023-10-18T12:25:43.653839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217590+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:25:15.217590+0000","last_peered":"2023-10-18T12:25:15.217590+0000","last_clean":"2023-10-18T12:25:15.217590+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:25:15.217590+0000","last_undegraded":"2023-10-18T12:25:15.217590+0000","last_fullsized":"2023-10-18T12:25:15.217590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210651+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:25:14.210651+0000","last_peered":"2023-10-18T12:25:14.210651+0000","last_clean":"2023-10-18T12:25:14.210651+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:25:14.210651+0000","last_undegraded":"2023-10-18T12:25:14.210651+0000","last_fullsized":"2023-10-18T12:25:14.210651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639762+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:25:43.639762+0000","last_peered":"2023-10-18T12:25:43.639762+0000","last_clean":"2023-10-18T12:25:43.639762+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:25:43.639762+0000","last_undegraded":"2023-10-18T12:25:43.639762+0000","last_fullsized":"2023-10-18T12:25:43.639762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642014+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:25:43.642014+0000","last_peered":"2023-10-18T12:25:43.642014+0000","last_clean":"2023-10-18T12:25:43.642014+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:25:43.642014+0000","last_undegraded":"2023-10-18T12:25:43.642014+0000","last_fullsized":"2023-10-18T12:25:43.642014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208294+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:25:15.208294+0000","last_peered":"2023-10-18T12:25:15.208294+0000","last_clean":"2023-10-18T12:25:15.208294+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:25:15.208294+0000","last_undegraded":"2023-10-18T12:25:15.208294+0000","last_fullsized":"2023-10-18T12:25:15.208294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211053+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:14.211053+0000","last_peered":"2023-10-18T12:25:14.211053+0000","last_clean":"2023-10-18T12:25:14.211053+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:14.211053+0000","last_undegraded":"2023-10-18T12:25:14.211053+0000","last_fullsized":"2023-10-18T12:25:14.211053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652650+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:25:43.652650+0000","last_peered":"2023-10-18T12:25:43.652650+0000","last_clean":"2023-10-18T12:25:43.652650+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:25:43.652650+0000","last_undegraded":"2023-10-18T12:25:43.652650+0000","last_fullsized":"2023-10-18T12:25:43.652650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211621+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:14.211621+0000","last_peered":"2023-10-18T12:25:14.211621+0000","last_clean":"2023-10-18T12:25:14.211621+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:14.211621+0000","last_undegraded":"2023-10-18T12:25:14.211621+0000","last_fullsized":"2023-10-18T12:25:14.211621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211404+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:25:14.211404+0000","last_peered":"2023-10-18T12:25:14.211404+0000","last_clean":"2023-10-18T12:25:14.211404+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:25:14.211404+0000","last_undegraded":"2023-10-18T12:25:14.211404+0000","last_fullsized":"2023-10-18T12:25:14.211404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653417+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:25:43.653417+0000","last_peered":"2023-10-18T12:25:43.653417+0000","last_clean":"2023-10-18T12:25:43.653417+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:25:43.653417+0000","last_undegraded":"2023-10-18T12:25:43.653417+0000","last_fullsized":"2023-10-18T12:25:43.653417+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208356+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:25:15.208356+0000","last_peered":"2023-10-18T12:25:15.208356+0000","last_clean":"2023-10-18T12:25:15.208356+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:25:15.208356+0000","last_undegraded":"2023-10-18T12:25:15.208356+0000","last_fullsized":"2023-10-18T12:25:15.208356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:11.324634+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:26:11.324634+0000","last_peered":"2023-10-18T12:26:11.324634+0000","last_clean":"2023-10-18T12:26:11.324634+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:26:11.324634+0000","last_undegraded":"2023-10-18T12:26:11.324634+0000","last_fullsized":"2023-10-18T12:26:11.324634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61450,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":206,"num_read_kb":163,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636586+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:25:43.636586+0000","last_peered":"2023-10-18T12:25:43.636586+0000","last_clean":"2023-10-18T12:25:43.636586+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:25:43.636586+0000","last_undegraded":"2023-10-18T12:25:43.636586+0000","last_fullsized":"2023-10-18T12:25:43.636586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211801+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:25:14.211801+0000","last_peered":"2023-10-18T12:25:14.211801+0000","last_clean":"2023-10-18T12:25:14.211801+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:25:14.211801+0000","last_undegraded":"2023-10-18T12:25:14.211801+0000","last_fullsized":"2023-10-18T12:25:14.211801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211560+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:25:14.211560+0000","last_peered":"2023-10-18T12:25:14.211560+0000","last_clean":"2023-10-18T12:25:14.211560+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:25:14.211560+0000","last_undegraded":"2023-10-18T12:25:14.211560+0000","last_fullsized":"2023-10-18T12:25:14.211560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649605+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:25:43.649605+0000","last_peered":"2023-10-18T12:25:43.649605+0000","last_clean":"2023-10-18T12:25:43.649605+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:25:43.649605+0000","last_undegraded":"2023-10-18T12:25:43.649605+0000","last_fullsized":"2023-10-18T12:25:43.649605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214581+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:25:14.214581+0000","last_peered":"2023-10-18T12:25:14.214581+0000","last_clean":"2023-10-18T12:25:14.214581+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:25:14.214581+0000","last_undegraded":"2023-10-18T12:25:14.214581+0000","last_fullsized":"2023-10-18T12:25:14.214581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211113+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:25:14.211113+0000","last_peered":"2023-10-18T12:25:14.211113+0000","last_clean":"2023-10-18T12:25:14.211113+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:25:14.211113+0000","last_undegraded":"2023-10-18T12:25:14.211113+0000","last_fullsized":"2023-10-18T12:25:14.211113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.654349+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:43.654349+0000","last_peered":"2023-10-18T12:25:43.654349+0000","last_clean":"2023-10-18T12:25:43.654349+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:43.654349+0000","last_undegraded":"2023-10-18T12:25:43.654349+0000","last_fullsized":"2023-10-18T12:25:43.654349+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208931+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:25:15.208931+0000","last_peered":"2023-10-18T12:25:15.208931+0000","last_clean":"2023-10-18T12:25:15.208931+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:25:15.208931+0000","last_undegraded":"2023-10-18T12:25:15.208931+0000","last_fullsized":"2023-10-18T12:25:15.208931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210996+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:14.210996+0000","last_peered":"2023-10-18T12:25:14.210996+0000","last_clean":"2023-10-18T12:25:14.210996+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:14.210996+0000","last_undegraded":"2023-10-18T12:25:14.210996+0000","last_fullsized":"2023-10-18T12:25:14.210996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642268+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:43.642268+0000","last_peered":"2023-10-18T12:25:43.642268+0000","last_clean":"2023-10-18T12:25:43.642268+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:43.642268+0000","last_undegraded":"2023-10-18T12:25:43.642268+0000","last_fullsized":"2023-10-18T12:25:43.642268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208699+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:25:14.208699+0000","last_peered":"2023-10-18T12:25:14.208699+0000","last_clean":"2023-10-18T12:25:14.208699+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:25:14.208699+0000","last_undegraded":"2023-10-18T12:25:14.208699+0000","last_fullsized":"2023-10-18T12:25:14.208699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225793+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:25:15.225793+0000","last_peered":"2023-10-18T12:25:15.225793+0000","last_clean":"2023-10-18T12:25:15.225793+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:25:15.225793+0000","last_undegraded":"2023-10-18T12:25:15.225793+0000","last_fullsized":"2023-10-18T12:25:15.225793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643090+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:25:43.643090+0000","last_peered":"2023-10-18T12:25:43.643090+0000","last_clean":"2023-10-18T12:25:43.643090+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:25:43.643090+0000","last_undegraded":"2023-10-18T12:25:43.643090+0000","last_fullsized":"2023-10-18T12:25:43.643090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211886+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:25:14.211886+0000","last_peered":"2023-10-18T12:25:14.211886+0000","last_clean":"2023-10-18T12:25:14.211886+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:25:14.211886+0000","last_undegraded":"2023-10-18T12:25:14.211886+0000","last_fullsized":"2023-10-18T12:25:14.211886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.224004+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:25:15.224004+0000","last_peered":"2023-10-18T12:25:15.224004+0000","last_clean":"2023-10-18T12:25:15.224004+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:25:15.224004+0000","last_undegraded":"2023-10-18T12:25:15.224004+0000","last_fullsized":"2023-10-18T12:25:15.224004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644328+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:25:43.644328+0000","last_peered":"2023-10-18T12:25:43.644328+0000","last_clean":"2023-10-18T12:25:43.644328+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:25:43.644328+0000","last_undegraded":"2023-10-18T12:25:43.644328+0000","last_fullsized":"2023-10-18T12:25:43.644328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215695+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:25:14.215695+0000","last_peered":"2023-10-18T12:25:14.215695+0000","last_clean":"2023-10-18T12:25:14.215695+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:25:14.215695+0000","last_undegraded":"2023-10-18T12:25:14.215695+0000","last_fullsized":"2023-10-18T12:25:14.215695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213449+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:26:15.213449+0000","last_peered":"2023-10-18T12:26:15.213449+0000","last_clean":"2023-10-18T12:26:15.213449+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:26:15.213449+0000","last_undegraded":"2023-10-18T12:26:15.213449+0000","last_fullsized":"2023-10-18T12:26:15.213449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208378+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:25:14.208378+0000","last_peered":"2023-10-18T12:25:14.208378+0000","last_clean":"2023-10-18T12:25:14.208378+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:25:14.208378+0000","last_undegraded":"2023-10-18T12:25:14.208378+0000","last_fullsized":"2023-10-18T12:25:14.208378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213184+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:25:15.213184+0000","last_peered":"2023-10-18T12:25:15.213184+0000","last_clean":"2023-10-18T12:25:15.213184+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:25:15.213184+0000","last_undegraded":"2023-10-18T12:25:15.213184+0000","last_fullsized":"2023-10-18T12:25:15.213184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649880+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:25:43.649880+0000","last_peered":"2023-10-18T12:25:43.649880+0000","last_clean":"2023-10-18T12:25:43.649880+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:25:43.649880+0000","last_undegraded":"2023-10-18T12:25:43.649880+0000","last_fullsized":"2023-10-18T12:25:43.649880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209020+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:25:14.209020+0000","last_peered":"2023-10-18T12:25:14.209020+0000","last_clean":"2023-10-18T12:25:14.209020+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:25:14.209020+0000","last_undegraded":"2023-10-18T12:25:14.209020+0000","last_fullsized":"2023-10-18T12:25:14.209020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644627+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:25:43.644627+0000","last_peered":"2023-10-18T12:25:43.644627+0000","last_clean":"2023-10-18T12:25:43.644627+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:25:43.644627+0000","last_undegraded":"2023-10-18T12:25:43.644627+0000","last_fullsized":"2023-10-18T12:25:43.644627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210906+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:25:15.210906+0000","last_peered":"2023-10-18T12:25:15.210906+0000","last_clean":"2023-10-18T12:25:15.210906+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:25:15.210906+0000","last_undegraded":"2023-10-18T12:25:15.210906+0000","last_fullsized":"2023-10-18T12:25:15.210906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210590+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:25:14.210590+0000","last_peered":"2023-10-18T12:25:14.210590+0000","last_clean":"2023-10-18T12:25:14.210590+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:25:14.210590+0000","last_undegraded":"2023-10-18T12:25:14.210590+0000","last_fullsized":"2023-10-18T12:25:14.210590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-10-18T12:24:09.961298+0000","last_change":"2023-10-18T12:26:15.661239+0000","last_active":"2023-10-18T12:24:08.948983+0000","last_peered":"2023-10-18T12:24:08.948983+0000","last_clean":"2023-10-18T12:24:08.948983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:24:09.961298+0000","last_undegraded":"2023-10-18T12:24:09.961298+0000","last_fullsized":"2023-10-18T12:24:09.961298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211540+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:25:14.211540+0000","last_peered":"2023-10-18T12:25:14.211540+0000","last_clean":"2023-10-18T12:25:14.211540+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:25:14.211540+0000","last_undegraded":"2023-10-18T12:25:14.211540+0000","last_fullsized":"2023-10-18T12:25:14.211540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.212426+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:15.212426+0000","last_peered":"2023-10-18T12:25:15.212426+0000","last_clean":"2023-10-18T12:25:15.212426+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:15.212426+0000","last_undegraded":"2023-10-18T12:25:15.212426+0000","last_fullsized":"2023-10-18T12:25:15.212426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223137+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:25:15.223137+0000","last_peered":"2023-10-18T12:25:15.223137+0000","last_clean":"2023-10-18T12:25:15.223137+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:25:15.223137+0000","last_undegraded":"2023-10-18T12:25:15.223137+0000","last_fullsized":"2023-10-18T12:25:15.223137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649534+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:25:43.649534+0000","last_peered":"2023-10-18T12:25:43.649534+0000","last_clean":"2023-10-18T12:25:43.649534+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:25:43.649534+0000","last_undegraded":"2023-10-18T12:25:43.649534+0000","last_fullsized":"2023-10-18T12:25:43.649534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217523+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:25:15.217523+0000","last_peered":"2023-10-18T12:25:15.217523+0000","last_clean":"2023-10-18T12:25:15.217523+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:25:15.217523+0000","last_undegraded":"2023-10-18T12:25:15.217523+0000","last_fullsized":"2023-10-18T12:25:15.217523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214480+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:25:14.214480+0000","last_peered":"2023-10-18T12:25:14.214480+0000","last_clean":"2023-10-18T12:25:14.214480+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:25:14.214480+0000","last_undegraded":"2023-10-18T12:25:14.214480+0000","last_fullsized":"2023-10-18T12:25:14.214480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213970+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:25:15.213970+0000","last_peered":"2023-10-18T12:25:15.213970+0000","last_clean":"2023-10-18T12:25:15.213970+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:25:15.213970+0000","last_undegraded":"2023-10-18T12:25:15.213970+0000","last_fullsized":"2023-10-18T12:25:15.213970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649689+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:25:43.649689+0000","last_peered":"2023-10-18T12:25:43.649689+0000","last_clean":"2023-10-18T12:25:43.649689+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:25:43.649689+0000","last_undegraded":"2023-10-18T12:25:43.649689+0000","last_fullsized":"2023-10-18T12:25:43.649689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.214742+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:25:15.214742+0000","last_peered":"2023-10-18T12:25:15.214742+0000","last_clean":"2023-10-18T12:25:15.214742+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:25:15.214742+0000","last_undegraded":"2023-10-18T12:25:15.214742+0000","last_fullsized":"2023-10-18T12:25:15.214742+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:13.149946+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:26:13.149946+0000","last_peered":"2023-10-18T12:26:13.149946+0000","last_clean":"2023-10-18T12:26:13.149946+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:26:13.149946+0000","last_undegraded":"2023-10-18T12:26:13.149946+0000","last_fullsized":"2023-10-18T12:26:13.149946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643505+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:25:43.643505+0000","last_peered":"2023-10-18T12:25:43.643505+0000","last_clean":"2023-10-18T12:25:43.643505+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:25:43.643505+0000","last_undegraded":"2023-10-18T12:25:43.643505+0000","last_fullsized":"2023-10-18T12:25:43.643505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212653+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:25:14.212653+0000","last_peered":"2023-10-18T12:25:14.212653+0000","last_clean":"2023-10-18T12:25:14.212653+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:25:14.212653+0000","last_undegraded":"2023-10-18T12:25:14.212653+0000","last_fullsized":"2023-10-18T12:25:14.212653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213075+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:14.213075+0000","last_peered":"2023-10-18T12:25:14.213075+0000","last_clean":"2023-10-18T12:25:14.213075+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:14.213075+0000","last_undegraded":"2023-10-18T12:25:14.213075+0000","last_fullsized":"2023-10-18T12:25:14.213075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649714+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:25:43.649714+0000","last_peered":"2023-10-18T12:25:43.649714+0000","last_clean":"2023-10-18T12:25:43.649714+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:25:43.649714+0000","last_undegraded":"2023-10-18T12:25:43.649714+0000","last_fullsized":"2023-10-18T12:25:43.649714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223250+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223250+0000","last_peered":"2023-10-18T12:25:15.223250+0000","last_clean":"2023-10-18T12:25:15.223250+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223250+0000","last_undegraded":"2023-10-18T12:25:15.223250+0000","last_fullsized":"2023-10-18T12:25:15.223250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214524+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:25:14.214524+0000","last_peered":"2023-10-18T12:25:14.214524+0000","last_clean":"2023-10-18T12:25:14.214524+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:25:14.214524+0000","last_undegraded":"2023-10-18T12:25:14.214524+0000","last_fullsized":"2023-10-18T12:25:14.214524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.604171+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:25:43.604171+0000","last_peered":"2023-10-18T12:25:43.604171+0000","last_clean":"2023-10-18T12:25:43.604171+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:25:43.604171+0000","last_undegraded":"2023-10-18T12:25:43.604171+0000","last_fullsized":"2023-10-18T12:25:43.604171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.220890+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:25:15.220890+0000","last_peered":"2023-10-18T12:25:15.220890+0000","last_clean":"2023-10-18T12:25:15.220890+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:25:15.220890+0000","last_undegraded":"2023-10-18T12:25:15.220890+0000","last_fullsized":"2023-10-18T12:25:15.220890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210834+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:25:14.210834+0000","last_peered":"2023-10-18T12:25:14.210834+0000","last_clean":"2023-10-18T12:25:14.210834+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:25:14.210834+0000","last_undegraded":"2023-10-18T12:25:14.210834+0000","last_fullsized":"2023-10-18T12:25:14.210834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.651462+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:25:43.651462+0000","last_peered":"2023-10-18T12:25:43.651462+0000","last_clean":"2023-10-18T12:25:43.651462+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:25:43.651462+0000","last_undegraded":"2023-10-18T12:25:43.651462+0000","last_fullsized":"2023-10-18T12:25:43.651462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211350+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:14.211350+0000","last_peered":"2023-10-18T12:25:14.211350+0000","last_clean":"2023-10-18T12:25:14.211350+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:14.211350+0000","last_undegraded":"2023-10-18T12:25:14.211350+0000","last_fullsized":"2023-10-18T12:25:14.211350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210884+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:25:14.210884+0000","last_peered":"2023-10-18T12:25:14.210884+0000","last_clean":"2023-10-18T12:25:14.210884+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:25:14.210884+0000","last_undegraded":"2023-10-18T12:25:14.210884+0000","last_fullsized":"2023-10-18T12:25:14.210884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648198+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:25:43.648198+0000","last_peered":"2023-10-18T12:25:43.648198+0000","last_clean":"2023-10-18T12:25:43.648198+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:25:43.648198+0000","last_undegraded":"2023-10-18T12:25:43.648198+0000","last_fullsized":"2023-10-18T12:25:43.648198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212507+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:14.212507+0000","last_peered":"2023-10-18T12:25:14.212507+0000","last_clean":"2023-10-18T12:25:14.212507+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:14.212507+0000","last_undegraded":"2023-10-18T12:25:14.212507+0000","last_fullsized":"2023-10-18T12:25:14.212507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216327+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:25:14.216327+0000","last_peered":"2023-10-18T12:25:14.216327+0000","last_clean":"2023-10-18T12:25:14.216327+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:25:14.216327+0000","last_undegraded":"2023-10-18T12:25:14.216327+0000","last_fullsized":"2023-10-18T12:25:14.216327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652574+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:25:43.652574+0000","last_peered":"2023-10-18T12:25:43.652574+0000","last_clean":"2023-10-18T12:25:43.652574+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:25:43.652574+0000","last_undegraded":"2023-10-18T12:25:43.652574+0000","last_fullsized":"2023-10-18T12:25:43.652574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214357+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:25:14.214357+0000","last_peered":"2023-10-18T12:25:14.214357+0000","last_clean":"2023-10-18T12:25:14.214357+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:25:14.214357+0000","last_undegraded":"2023-10-18T12:25:14.214357+0000","last_fullsized":"2023-10-18T12:25:14.214357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211315+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:25:14.211315+0000","last_peered":"2023-10-18T12:25:14.211315+0000","last_clean":"2023-10-18T12:25:14.211315+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:25:14.211315+0000","last_undegraded":"2023-10-18T12:25:14.211315+0000","last_fullsized":"2023-10-18T12:25:14.211315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644409+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:43.644409+0000","last_peered":"2023-10-18T12:25:43.644409+0000","last_clean":"2023-10-18T12:25:43.644409+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:43.644409+0000","last_undegraded":"2023-10-18T12:25:43.644409+0000","last_fullsized":"2023-10-18T12:25:43.644409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218308+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:25:14.218308+0000","last_peered":"2023-10-18T12:25:14.218308+0000","last_clean":"2023-10-18T12:25:14.218308+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:25:14.218308+0000","last_undegraded":"2023-10-18T12:25:14.218308+0000","last_fullsized":"2023-10-18T12:25:14.218308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211578+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:25:14.211578+0000","last_peered":"2023-10-18T12:25:14.211578+0000","last_clean":"2023-10-18T12:25:14.211578+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:25:14.211578+0000","last_undegraded":"2023-10-18T12:25:14.211578+0000","last_fullsized":"2023-10-18T12:25:14.211578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639071+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:43.639071+0000","last_peered":"2023-10-18T12:25:43.639071+0000","last_clean":"2023-10-18T12:25:43.639071+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:43.639071+0000","last_undegraded":"2023-10-18T12:25:43.639071+0000","last_fullsized":"2023-10-18T12:25:43.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210493+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:25:14.210493+0000","last_peered":"2023-10-18T12:25:14.210493+0000","last_clean":"2023-10-18T12:25:14.210493+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:25:14.210493+0000","last_undegraded":"2023-10-18T12:25:14.210493+0000","last_fullsized":"2023-10-18T12:25:14.210493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225441+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:25:15.225441+0000","last_peered":"2023-10-18T12:25:15.225441+0000","last_clean":"2023-10-18T12:25:15.225441+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:25:15.225441+0000","last_undegraded":"2023-10-18T12:25:15.225441+0000","last_fullsized":"2023-10-18T12:25:15.225441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639284+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:43.639284+0000","last_peered":"2023-10-18T12:25:43.639284+0000","last_clean":"2023-10-18T12:25:43.639284+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:43.639284+0000","last_undegraded":"2023-10-18T12:25:43.639284+0000","last_fullsized":"2023-10-18T12:25:43.639284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.638348+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:25:43.638348+0000","last_peered":"2023-10-18T12:25:43.638348+0000","last_clean":"2023-10-18T12:25:43.638348+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:25:43.638348+0000","last_undegraded":"2023-10-18T12:25:43.638348+0000","last_fullsized":"2023-10-18T12:25:43.638348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643095+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:25:43.643095+0000","last_peered":"2023-10-18T12:25:43.643095+0000","last_clean":"2023-10-18T12:25:43.643095+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:25:43.643095+0000","last_undegraded":"2023-10-18T12:25:43.643095+0000","last_fullsized":"2023-10-18T12:25:43.643095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646655+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:25:43.646655+0000","last_peered":"2023-10-18T12:25:43.646655+0000","last_clean":"2023-10-18T12:25:43.646655+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:25:43.646655+0000","last_undegraded":"2023-10-18T12:25:43.646655+0000","last_fullsized":"2023-10-18T12:25:43.646655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656634+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:25:43.656634+0000","last_peered":"2023-10-18T12:25:43.656634+0000","last_clean":"2023-10-18T12:25:43.656634+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:25:43.656634+0000","last_undegraded":"2023-10-18T12:25:43.656634+0000","last_fullsized":"2023-10-18T12:25:43.656634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.655509+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:25:43.655509+0000","last_peered":"2023-10-18T12:25:43.655509+0000","last_clean":"2023-10-18T12:25:43.655509+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:25:43.655509+0000","last_undegraded":"2023-10-18T12:25:43.655509+0000","last_fullsized":"2023-10-18T12:25:43.655509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639525+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:25:43.639525+0000","last_peered":"2023-10-18T12:25:43.639525+0000","last_clean":"2023-10-18T12:25:43.639525+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:25:43.639525+0000","last_undegraded":"2023-10-18T12:25:43.639525+0000","last_fullsized":"2023-10-18T12:25:43.639525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644102+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:25:43.644102+0000","last_peered":"2023-10-18T12:25:43.644102+0000","last_clean":"2023-10-18T12:25:43.644102+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:25:43.644102+0000","last_undegraded":"2023-10-18T12:25:43.644102+0000","last_fullsized":"2023-10-18T12:25:43.644102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644112+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:25:43.644112+0000","last_peered":"2023-10-18T12:25:43.644112+0000","last_clean":"2023-10-18T12:25:43.644112+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:25:43.644112+0000","last_undegraded":"2023-10-18T12:25:43.644112+0000","last_fullsized":"2023-10-18T12:25:43.644112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636676+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:25:43.636676+0000","last_peered":"2023-10-18T12:25:43.636676+0000","last_clean":"2023-10-18T12:25:43.636676+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:25:43.636676+0000","last_undegraded":"2023-10-18T12:25:43.636676+0000","last_fullsized":"2023-10-18T12:25:43.636676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636709+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:25:43.636709+0000","last_peered":"2023-10-18T12:25:43.636709+0000","last_clean":"2023-10-18T12:25:43.636709+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:25:43.636709+0000","last_undegraded":"2023-10-18T12:25:43.636709+0000","last_fullsized":"2023-10-18T12:25:43.636709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643394+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:25:43.643394+0000","last_peered":"2023-10-18T12:25:43.643394+0000","last_clean":"2023-10-18T12:25:43.643394+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:25:43.643394+0000","last_undegraded":"2023-10-18T12:25:43.643394+0000","last_fullsized":"2023-10-18T12:25:43.643394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.657911+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:25:43.657911+0000","last_peered":"2023-10-18T12:25:43.657911+0000","last_clean":"2023-10-18T12:25:43.657911+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:25:43.657911+0000","last_undegraded":"2023-10-18T12:25:43.657911+0000","last_fullsized":"2023-10-18T12:25:43.657911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643480+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:25:43.643480+0000","last_peered":"2023-10-18T12:25:43.643480+0000","last_clean":"2023-10-18T12:25:43.643480+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:25:43.643480+0000","last_undegraded":"2023-10-18T12:25:43.643480+0000","last_fullsized":"2023-10-18T12:25:43.643480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62248,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":211,"num_read_kb":168,"num_write":81,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":197502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":1.147,"15min":1.147},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.745}]},{"osd":2,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88400000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":1.238,"15min":1.238},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75600000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":1.661,"15min":1.661},"last":0.84599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90400000000000003,"5min":1.361,"15min":1.361},"last":0.872},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":1,"15min":1},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.91100000000000003,"5min":1.726,"15min":1.726},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.012,"5min":1.401,"15min":1.401},"last":0.95899999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89000000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.92400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1161040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.54500000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.435,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.499},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63300000000000001,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.999,"15min":0.999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72499999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.432}]},{"osd":5,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84599999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.876,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.77600000000000002,"5min":1.099,"15min":1.099},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82799999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90400000000000003,"5min":1.071,"15min":1.071},"last":0.62},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.044,"15min":1.044},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63700000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.67500000000000004}]}]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":527643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0089999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.92700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.753},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.86199999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1739999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1519999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.91500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.871},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.94599999999999995,"15min":0.95199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":6.2220000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.69699999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.89100000000000001},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2430000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78000000000000003}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.432},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.501}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0149999999999999,"5min":1.044,"15min":1.044},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.72399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0640000000000001,"5min":1.26,"15min":1.26},"last":0.754}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.68300000000000005},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76800000000000002,"15min":0.72799999999999998},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.107,"15min":1.107},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66700000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1108170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.433,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.501},{"interface":"front","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60099999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35799999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.999,"15min":0.999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.137,"15min":1.137},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81299999999999994,"5min":1.071,"15min":1.071},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.66500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.98199999999999998,"5min":1.02,"15min":1.02},"last":0.72799999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95399999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69199999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70699999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":509028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.53000000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.622},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.753}]},{"osd":3,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.68799999999999994,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.9199999999999999,"5min":6.9199999999999999,"15min":6.9199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.026,"5min":0.81899999999999995,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":7.2270000000000003,"5min":7.2270000000000003,"15min":7.2270000000000003},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.016,"5min":1.234,"15min":1.234},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74299999999999999,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.28,"5min":1.849,"15min":1.849},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84999999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.88,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":7.7400000000000002,"5min":7.7400000000000002,"15min":7.7400000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.69699999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.90800000000000003,"15min":0.88900000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.4969999999999999,"5min":7.4969999999999999,"15min":7.4969999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95499999999999996,"15min":0.90300000000000002},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3240000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":570478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.161,"15min":1.161},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94699999999999995,"5min":1.454,"15min":1.454},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.57299999999999995},"min":{"1min":0.52000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88500000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71899999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.64800000000000002,"15min":0.59099999999999997},"min":{"1min":0.63200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":1.298,"15min":1.298},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.52700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89400000000000002,"5min":1.016,"15min":1.016},"last":0.88200000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.002}]}]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":509118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.65200000000000002,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.30299999999999999},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65600000000000003,"15min":0.55200000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.32400000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.60699999999999998},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.84099999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1479999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.024,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.65800000000000003,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.69699999999999995,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.498}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.66200000000000003},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.65700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.755}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69899999999999995,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.751,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":0.92100000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86799999999999999,"15min":0.84599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.61199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.89300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.876},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.89300000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.03,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93899999999999995,"15min":0.92400000000000004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":0.90000000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":575618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.26100000000000001},"max":{"1min":0.92100000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.54600000000000004},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.626},{"interface":"front","average":{"1min":0.745,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.86899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.34000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73799999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.42299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.58399999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.753},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85399999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76500000000000001,"15min":0.74099999999999999},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.254,"15min":1.254},"last":0.995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.74299999999999999,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91600000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":509132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.51400000000000001,"15min":0.436},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7880000000000003,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.52700000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.73599999999999999,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.622},{"interface":"front","average":{"1min":1.026,"5min":0.68999999999999995,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":5.0579999999999998,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.73699999999999999,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.085,"5min":0.77800000000000002,"15min":0.70599999999999996},"min":{"1min":0.60999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.78400000000000003,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.60099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.752,"15min":0.69499999999999995},"min":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.83299999999999996,"15min":0.753},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.984,"5min":4.984,"15min":4.984},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3789999999999996,"5min":5.3789999999999996,"15min":5.3789999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.86199999999999999},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93799999999999994,"15min":0.90500000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.83099999999999996}]},{"osd":11,"last update":"Wed Oct 18 12:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.016,"15min":1.052},"min":{"1min":0.61899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2030000000000001,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.111,"15min":1.155},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2170000000000001,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54800000000000004,"5min":0.621,"15min":0.621},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57099999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.876,"5min":1.04,"15min":1.04},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.52200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":1.22,"15min":1.22},"last":0.75}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61099999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80200000000000005,"15min":0.81100000000000005},"min":{"1min":0.65900000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.02,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.55700000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65000000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.501}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.88}]}]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1103343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":0.999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88100000000000001}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94099999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.752},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.01},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":4.548,"15min":4.548},"last":0.99399999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.745,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73999999999999999,"15min":0.70799999999999996},"min":{"1min":0.60699999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:26:16.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:26:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-10-18T12:26:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:16 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-10-18T12:26:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:16 smithi186 ceph-mon[135253]: osdmap e92: 12 total, 12 up, 12 in 2023-10-18T12:26:16.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:16 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3505015206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:16.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:16 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1928102591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:16.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 -- 172.21.15.70:0/914427213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 msgr2=0x7fef4810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/914427213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef4810d570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b0a0 tx=0x7fef4000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 -- 172.21.15.70:0/914427213 shutdown_connections 2023-10-18T12:26:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/914427213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef4810d570 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/914427213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef481066b0 0x7fef48106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/914427213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481006a0 0x7fef48100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.586+0000 7fef4d1ff700 1 -- 172.21.15.70:0/914427213 >> 172.21.15.70:0/914427213 conn(0x7fef480fc170 msgr2=0x7fef480fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.587+0000 7fef4d1ff700 1 -- 172.21.15.70:0/914427213 shutdown_connections 2023-10-18T12:26:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.587+0000 7fef4d1ff700 1 -- 172.21.15.70:0/914427213 wait complete. 2023-10-18T12:26:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.587+0000 7fef4d1ff700 1 Processor -- start 2023-10-18T12:26:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 -- start start 2023-10-18T12:26:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef481006a0 0x7fef48196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef48196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481066b0 0x7fef4819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819b950 con 0x7fef48101060 2023-10-18T12:26:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819bac0 con 0x7fef481066b0 2023-10-18T12:26:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4d1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819bc80 con 0x7fef481006a0 2023-10-18T12:26:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef48196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef48196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40094/0 (socket says 172.21.15.70:40094) 2023-10-18T12:26:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef4659c700 1 -- 172.21.15.70:0/744571353 learned_addr learned my addr 172.21.15.70:0/744571353 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.588+0000 7fef46d9d700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef481006a0 0x7fef48196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4759e700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481066b0 0x7fef4819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4659c700 1 -- 172.21.15.70:0/744571353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef481006a0 msgr2=0x7fef48196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4659c700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef481006a0 0x7fef48196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4659c700 1 -- 172.21.15.70:0/744571353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481066b0 msgr2=0x7fef4819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4659c700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481066b0 0x7fef4819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4659c700 1 -- 172.21.15.70:0/744571353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40009d00 con 0x7fef48101060 2023-10-18T12:26:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4759e700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481066b0 0x7fef4819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4659c700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef48196fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e9a0 tx=0x7fef3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009790 con 0x7fef48101060 2023-10-18T12:26:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef3c00cbd0 con 0x7fef48101060 2023-10-18T12:26:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c018680 con 0x7fef48101060 2023-10-18T12:26:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4819bf60 con 0x7fef48101060 2023-10-18T12:26:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.589+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef481a3810 con 0x7fef48101060 2023-10-18T12:26:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.590+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef48108ca0 con 0x7fef48101060 2023-10-18T12:26:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.594+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fef3c00cd40 con 0x7fef48101060 2023-10-18T12:26:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.595+0000 7fef2ffff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fef300a14d0 0x7fef300a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.595+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fef3c014070 con 0x7fef48101060 2023-10-18T12:26:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.595+0000 7fef46d9d700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fef300a14d0 0x7fef300a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.596+0000 7fef46d9d700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fef300a14d0 0x7fef300a3990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fef38009be0 tx=0x7fef38009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.597+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef3c096910 con 0x7fef48101060 2023-10-18T12:26:16.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-10-18T12:26:16.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:16 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-10-18T12:26:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:16 smithi070 ceph-mon[98639]: osdmap e92: 12 total, 12 up, 12 in 2023-10-18T12:26:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:16 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3505015206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:16.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:16 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1928102591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:16.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-10-18T12:26:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:16 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-10-18T12:26:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:16 smithi152 ceph-mon[134767]: osdmap e92: 12 total, 12 up, 12 in 2023-10-18T12:26:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:16 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3505015206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:16.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:16 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1928102591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:16.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.796+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef48197d70 con 0x7fef300a14d0 2023-10-18T12:26:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.809+0000 7fef2ffff700 1 -- 172.21.15.70:0/744571353 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427323 (secure 0 0 0) 0x7fef48197d70 con 0x7fef300a14d0 2023-10-18T12:26:16.817 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:16.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fef300a14d0 msgr2=0x7fef300a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fef300a14d0 0x7fef300a3990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fef38009be0 tx=0x7fef38009260 comp rx=0 tx=0).stop 2023-10-18T12:26:16.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 msgr2=0x7fef48196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef48196fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e9a0 tx=0x7fef3c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 shutdown_connections 2023-10-18T12:26:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fef300a14d0 0x7fef300a3990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef481066b0 0x7fef4819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef481006a0 0x7fef48196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 --2- 172.21.15.70:0/744571353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef48101060 0x7fef48196fb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 >> 172.21.15.70:0/744571353 conn(0x7fef480fc170 msgr2=0x7fef480fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.814+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 shutdown_connections 2023-10-18T12:26:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:16.815+0000 7fef4d1ff700 1 -- 172.21.15.70:0/744571353 wait complete. 2023-10-18T12:26:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:26:16.836 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-18T12:26:16.214128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652862,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":347,"num_read_kb":287,"num_write":264,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338200,"kb_used_data":14004,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603624,"statfs":{"total":1151940427776,"available":1151594110976,"internally_reserved":0,"allocated":14340096,"data_stored":8113568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.591689"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650976+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:25:43.650976+0000","last_peered":"2023-10-18T12:25:43.650976+0000","last_clean":"2023-10-18T12:25:43.650976+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:25:43.650976+0000","last_undegraded":"2023-10-18T12:25:43.650976+0000","last_fullsized":"2023-10-18T12:25:43.650976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649821+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:25:43.649821+0000","last_peered":"2023-10-18T12:25:43.649821+0000","last_clean":"2023-10-18T12:25:43.649821+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:25:43.649821+0000","last_undegraded":"2023-10-18T12:25:43.649821+0000","last_fullsized":"2023-10-18T12:25:43.649821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650939+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:25:43.650939+0000","last_peered":"2023-10-18T12:25:43.650939+0000","last_clean":"2023-10-18T12:25:43.650939+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:25:43.650939+0000","last_undegraded":"2023-10-18T12:25:43.650939+0000","last_fullsized":"2023-10-18T12:25:43.650939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.640858+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:25:43.640858+0000","last_peered":"2023-10-18T12:25:43.640858+0000","last_clean":"2023-10-18T12:25:43.640858+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:25:43.640858+0000","last_undegraded":"2023-10-18T12:25:43.640858+0000","last_fullsized":"2023-10-18T12:25:43.640858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.631268+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:25:43.631268+0000","last_peered":"2023-10-18T12:25:43.631268+0000","last_clean":"2023-10-18T12:25:43.631268+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:25:43.631268+0000","last_undegraded":"2023-10-18T12:25:43.631268+0000","last_fullsized":"2023-10-18T12:25:43.631268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643204+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:25:43.643204+0000","last_peered":"2023-10-18T12:25:43.643204+0000","last_clean":"2023-10-18T12:25:43.643204+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:25:43.643204+0000","last_undegraded":"2023-10-18T12:25:43.643204+0000","last_fullsized":"2023-10-18T12:25:43.643204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639863+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:43.639863+0000","last_peered":"2023-10-18T12:25:43.639863+0000","last_clean":"2023-10-18T12:25:43.639863+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:43.639863+0000","last_undegraded":"2023-10-18T12:25:43.639863+0000","last_fullsized":"2023-10-18T12:25:43.639863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639818+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:25:43.639818+0000","last_peered":"2023-10-18T12:25:43.639818+0000","last_clean":"2023-10-18T12:25:43.639818+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:25:43.639818+0000","last_undegraded":"2023-10-18T12:25:43.639818+0000","last_fullsized":"2023-10-18T12:25:43.639818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642671+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:25:43.642671+0000","last_peered":"2023-10-18T12:25:43.642671+0000","last_clean":"2023-10-18T12:25:43.642671+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:25:43.642671+0000","last_undegraded":"2023-10-18T12:25:43.642671+0000","last_fullsized":"2023-10-18T12:25:43.642671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.645379+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:25:43.645379+0000","last_peered":"2023-10-18T12:25:43.645379+0000","last_clean":"2023-10-18T12:25:43.645379+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:25:43.645379+0000","last_undegraded":"2023-10-18T12:25:43.645379+0000","last_fullsized":"2023-10-18T12:25:43.645379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644261+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:43.644261+0000","last_peered":"2023-10-18T12:25:43.644261+0000","last_clean":"2023-10-18T12:25:43.644261+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:43.644261+0000","last_undegraded":"2023-10-18T12:25:43.644261+0000","last_fullsized":"2023-10-18T12:25:43.644261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633544+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:43.633544+0000","last_peered":"2023-10-18T12:25:43.633544+0000","last_clean":"2023-10-18T12:25:43.633544+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:43.633544+0000","last_undegraded":"2023-10-18T12:25:43.633544+0000","last_fullsized":"2023-10-18T12:25:43.633544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.658873+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:25:43.658873+0000","last_peered":"2023-10-18T12:25:43.658873+0000","last_clean":"2023-10-18T12:25:43.658873+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:25:43.658873+0000","last_undegraded":"2023-10-18T12:25:43.658873+0000","last_fullsized":"2023-10-18T12:25:43.658873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646839+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:25:43.646839+0000","last_peered":"2023-10-18T12:25:43.646839+0000","last_clean":"2023-10-18T12:25:43.646839+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:25:43.646839+0000","last_undegraded":"2023-10-18T12:25:43.646839+0000","last_fullsized":"2023-10-18T12:25:43.646839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649694+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:25:43.649694+0000","last_peered":"2023-10-18T12:25:43.649694+0000","last_clean":"2023-10-18T12:25:43.649694+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:25:43.649694+0000","last_undegraded":"2023-10-18T12:25:43.649694+0000","last_fullsized":"2023-10-18T12:25:43.649694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648146+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:25:43.648146+0000","last_peered":"2023-10-18T12:25:43.648146+0000","last_clean":"2023-10-18T12:25:43.648146+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:25:43.648146+0000","last_undegraded":"2023-10-18T12:25:43.648146+0000","last_fullsized":"2023-10-18T12:25:43.648146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652031+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:25:43.652031+0000","last_peered":"2023-10-18T12:25:43.652031+0000","last_clean":"2023-10-18T12:25:43.652031+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:25:43.652031+0000","last_undegraded":"2023-10-18T12:25:43.652031+0000","last_fullsized":"2023-10-18T12:25:43.652031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639903+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:25:43.639903+0000","last_peered":"2023-10-18T12:25:43.639903+0000","last_clean":"2023-10-18T12:25:43.639903+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:25:43.639903+0000","last_undegraded":"2023-10-18T12:25:43.639903+0000","last_fullsized":"2023-10-18T12:25:43.639903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643796+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:25:43.643796+0000","last_peered":"2023-10-18T12:25:43.643796+0000","last_clean":"2023-10-18T12:25:43.643796+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:25:43.643796+0000","last_undegraded":"2023-10-18T12:25:43.643796+0000","last_fullsized":"2023-10-18T12:25:43.643796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212928+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:25:14.212928+0000","last_peered":"2023-10-18T12:25:14.212928+0000","last_clean":"2023-10-18T12:25:14.212928+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:25:14.212928+0000","last_undegraded":"2023-10-18T12:25:14.212928+0000","last_fullsized":"2023-10-18T12:25:14.212928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213691+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:26:15.213691+0000","last_peered":"2023-10-18T12:26:15.213691+0000","last_clean":"2023-10-18T12:26:15.213691+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:26:15.213691+0000","last_undegraded":"2023-10-18T12:26:15.213691+0000","last_fullsized":"2023-10-18T12:26:15.213691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639283+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:25:43.639283+0000","last_peered":"2023-10-18T12:25:43.639283+0000","last_clean":"2023-10-18T12:25:43.639283+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:25:43.639283+0000","last_undegraded":"2023-10-18T12:25:43.639283+0000","last_fullsized":"2023-10-18T12:25:43.639283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210737+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:14.210737+0000","last_peered":"2023-10-18T12:25:14.210737+0000","last_clean":"2023-10-18T12:25:14.210737+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:14.210737+0000","last_undegraded":"2023-10-18T12:25:14.210737+0000","last_fullsized":"2023-10-18T12:25:14.210737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.216358+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:25:15.216358+0000","last_peered":"2023-10-18T12:25:15.216358+0000","last_clean":"2023-10-18T12:25:15.216358+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:25:15.216358+0000","last_undegraded":"2023-10-18T12:25:15.216358+0000","last_fullsized":"2023-10-18T12:25:15.216358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656048+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:25:43.656048+0000","last_peered":"2023-10-18T12:25:43.656048+0000","last_clean":"2023-10-18T12:25:43.656048+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:25:43.656048+0000","last_undegraded":"2023-10-18T12:25:43.656048+0000","last_fullsized":"2023-10-18T12:25:43.656048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211459+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:14.211459+0000","last_peered":"2023-10-18T12:25:14.211459+0000","last_clean":"2023-10-18T12:25:14.211459+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:14.211459+0000","last_undegraded":"2023-10-18T12:25:14.211459+0000","last_fullsized":"2023-10-18T12:25:14.211459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215742+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:25:14.215742+0000","last_peered":"2023-10-18T12:25:14.215742+0000","last_clean":"2023-10-18T12:25:14.215742+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:25:14.215742+0000","last_undegraded":"2023-10-18T12:25:14.215742+0000","last_fullsized":"2023-10-18T12:25:14.215742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210540+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:14.210540+0000","last_peered":"2023-10-18T12:25:14.210540+0000","last_clean":"2023-10-18T12:25:14.210540+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:14.210540+0000","last_undegraded":"2023-10-18T12:25:14.210540+0000","last_fullsized":"2023-10-18T12:25:14.210540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216726+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:25:14.216726+0000","last_peered":"2023-10-18T12:25:14.216726+0000","last_clean":"2023-10-18T12:25:14.216726+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:25:14.216726+0000","last_undegraded":"2023-10-18T12:25:14.216726+0000","last_fullsized":"2023-10-18T12:25:14.216726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644740+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:25:43.644740+0000","last_peered":"2023-10-18T12:25:43.644740+0000","last_clean":"2023-10-18T12:25:43.644740+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:25:43.644740+0000","last_undegraded":"2023-10-18T12:25:43.644740+0000","last_fullsized":"2023-10-18T12:25:43.644740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217322+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:25:15.217322+0000","last_peered":"2023-10-18T12:25:15.217322+0000","last_clean":"2023-10-18T12:25:15.217322+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:25:15.217322+0000","last_undegraded":"2023-10-18T12:25:15.217322+0000","last_fullsized":"2023-10-18T12:25:15.217322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211310+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:25:14.211310+0000","last_peered":"2023-10-18T12:25:14.211310+0000","last_clean":"2023-10-18T12:25:14.211310+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:25:14.211310+0000","last_undegraded":"2023-10-18T12:25:14.211310+0000","last_fullsized":"2023-10-18T12:25:14.211310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633052+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:25:43.633052+0000","last_peered":"2023-10-18T12:25:43.633052+0000","last_clean":"2023-10-18T12:25:43.633052+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:25:43.633052+0000","last_undegraded":"2023-10-18T12:25:43.633052+0000","last_fullsized":"2023-10-18T12:25:43.633052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211137+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:25:14.211137+0000","last_peered":"2023-10-18T12:25:14.211137+0000","last_clean":"2023-10-18T12:25:14.211137+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:25:14.211137+0000","last_undegraded":"2023-10-18T12:25:14.211137+0000","last_fullsized":"2023-10-18T12:25:14.211137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210401+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:25:14.210401+0000","last_peered":"2023-10-18T12:25:14.210401+0000","last_clean":"2023-10-18T12:25:14.210401+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:25:14.210401+0000","last_undegraded":"2023-10-18T12:25:14.210401+0000","last_fullsized":"2023-10-18T12:25:14.210401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.645401+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:25:43.645401+0000","last_peered":"2023-10-18T12:25:43.645401+0000","last_clean":"2023-10-18T12:25:43.645401+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:25:43.645401+0000","last_undegraded":"2023-10-18T12:25:43.645401+0000","last_fullsized":"2023-10-18T12:25:43.645401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215355+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:25:14.215355+0000","last_peered":"2023-10-18T12:25:14.215355+0000","last_clean":"2023-10-18T12:25:14.215355+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:25:14.215355+0000","last_undegraded":"2023-10-18T12:25:14.215355+0000","last_fullsized":"2023-10-18T12:25:14.215355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207597+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:25:14.207597+0000","last_peered":"2023-10-18T12:25:14.207597+0000","last_clean":"2023-10-18T12:25:14.207597+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:25:14.207597+0000","last_undegraded":"2023-10-18T12:25:14.207597+0000","last_fullsized":"2023-10-18T12:25:14.207597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653839+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:25:43.653839+0000","last_peered":"2023-10-18T12:25:43.653839+0000","last_clean":"2023-10-18T12:25:43.653839+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:25:43.653839+0000","last_undegraded":"2023-10-18T12:25:43.653839+0000","last_fullsized":"2023-10-18T12:25:43.653839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217590+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:25:15.217590+0000","last_peered":"2023-10-18T12:25:15.217590+0000","last_clean":"2023-10-18T12:25:15.217590+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:25:15.217590+0000","last_undegraded":"2023-10-18T12:25:15.217590+0000","last_fullsized":"2023-10-18T12:25:15.217590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210651+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:25:14.210651+0000","last_peered":"2023-10-18T12:25:14.210651+0000","last_clean":"2023-10-18T12:25:14.210651+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:25:14.210651+0000","last_undegraded":"2023-10-18T12:25:14.210651+0000","last_fullsized":"2023-10-18T12:25:14.210651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639762+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:25:43.639762+0000","last_peered":"2023-10-18T12:25:43.639762+0000","last_clean":"2023-10-18T12:25:43.639762+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:25:43.639762+0000","last_undegraded":"2023-10-18T12:25:43.639762+0000","last_fullsized":"2023-10-18T12:25:43.639762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642014+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:25:43.642014+0000","last_peered":"2023-10-18T12:25:43.642014+0000","last_clean":"2023-10-18T12:25:43.642014+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:25:43.642014+0000","last_undegraded":"2023-10-18T12:25:43.642014+0000","last_fullsized":"2023-10-18T12:25:43.642014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208294+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:25:15.208294+0000","last_peered":"2023-10-18T12:25:15.208294+0000","last_clean":"2023-10-18T12:25:15.208294+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:25:15.208294+0000","last_undegraded":"2023-10-18T12:25:15.208294+0000","last_fullsized":"2023-10-18T12:25:15.208294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211053+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:14.211053+0000","last_peered":"2023-10-18T12:25:14.211053+0000","last_clean":"2023-10-18T12:25:14.211053+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:14.211053+0000","last_undegraded":"2023-10-18T12:25:14.211053+0000","last_fullsized":"2023-10-18T12:25:14.211053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652650+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:25:43.652650+0000","last_peered":"2023-10-18T12:25:43.652650+0000","last_clean":"2023-10-18T12:25:43.652650+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:25:43.652650+0000","last_undegraded":"2023-10-18T12:25:43.652650+0000","last_fullsized":"2023-10-18T12:25:43.652650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211621+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:14.211621+0000","last_peered":"2023-10-18T12:25:14.211621+0000","last_clean":"2023-10-18T12:25:14.211621+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:14.211621+0000","last_undegraded":"2023-10-18T12:25:14.211621+0000","last_fullsized":"2023-10-18T12:25:14.211621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211404+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:25:14.211404+0000","last_peered":"2023-10-18T12:25:14.211404+0000","last_clean":"2023-10-18T12:25:14.211404+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:25:14.211404+0000","last_undegraded":"2023-10-18T12:25:14.211404+0000","last_fullsized":"2023-10-18T12:25:14.211404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653417+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:25:43.653417+0000","last_peered":"2023-10-18T12:25:43.653417+0000","last_clean":"2023-10-18T12:25:43.653417+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:25:43.653417+0000","last_undegraded":"2023-10-18T12:25:43.653417+0000","last_fullsized":"2023-10-18T12:25:43.653417+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208356+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:25:15.208356+0000","last_peered":"2023-10-18T12:25:15.208356+0000","last_clean":"2023-10-18T12:25:15.208356+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:25:15.208356+0000","last_undegraded":"2023-10-18T12:25:15.208356+0000","last_fullsized":"2023-10-18T12:25:15.208356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:11.324634+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:26:11.324634+0000","last_peered":"2023-10-18T12:26:11.324634+0000","last_clean":"2023-10-18T12:26:11.324634+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:26:11.324634+0000","last_undegraded":"2023-10-18T12:26:11.324634+0000","last_fullsized":"2023-10-18T12:26:11.324634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61450,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":206,"num_read_kb":163,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636586+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:25:43.636586+0000","last_peered":"2023-10-18T12:25:43.636586+0000","last_clean":"2023-10-18T12:25:43.636586+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:25:43.636586+0000","last_undegraded":"2023-10-18T12:25:43.636586+0000","last_fullsized":"2023-10-18T12:25:43.636586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211801+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:25:14.211801+0000","last_peered":"2023-10-18T12:25:14.211801+0000","last_clean":"2023-10-18T12:25:14.211801+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:25:14.211801+0000","last_undegraded":"2023-10-18T12:25:14.211801+0000","last_fullsized":"2023-10-18T12:25:14.211801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211560+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:25:14.211560+0000","last_peered":"2023-10-18T12:25:14.211560+0000","last_clean":"2023-10-18T12:25:14.211560+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:25:14.211560+0000","last_undegraded":"2023-10-18T12:25:14.211560+0000","last_fullsized":"2023-10-18T12:25:14.211560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649605+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:25:43.649605+0000","last_peered":"2023-10-18T12:25:43.649605+0000","last_clean":"2023-10-18T12:25:43.649605+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:25:43.649605+0000","last_undegraded":"2023-10-18T12:25:43.649605+0000","last_fullsized":"2023-10-18T12:25:43.649605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214581+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:25:14.214581+0000","last_peered":"2023-10-18T12:25:14.214581+0000","last_clean":"2023-10-18T12:25:14.214581+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:25:14.214581+0000","last_undegraded":"2023-10-18T12:25:14.214581+0000","last_fullsized":"2023-10-18T12:25:14.214581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211113+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:25:14.211113+0000","last_peered":"2023-10-18T12:25:14.211113+0000","last_clean":"2023-10-18T12:25:14.211113+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:25:14.211113+0000","last_undegraded":"2023-10-18T12:25:14.211113+0000","last_fullsized":"2023-10-18T12:25:14.211113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.654349+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:43.654349+0000","last_peered":"2023-10-18T12:25:43.654349+0000","last_clean":"2023-10-18T12:25:43.654349+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:43.654349+0000","last_undegraded":"2023-10-18T12:25:43.654349+0000","last_fullsized":"2023-10-18T12:25:43.654349+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208931+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:25:15.208931+0000","last_peered":"2023-10-18T12:25:15.208931+0000","last_clean":"2023-10-18T12:25:15.208931+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:25:15.208931+0000","last_undegraded":"2023-10-18T12:25:15.208931+0000","last_fullsized":"2023-10-18T12:25:15.208931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210996+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:14.210996+0000","last_peered":"2023-10-18T12:25:14.210996+0000","last_clean":"2023-10-18T12:25:14.210996+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:14.210996+0000","last_undegraded":"2023-10-18T12:25:14.210996+0000","last_fullsized":"2023-10-18T12:25:14.210996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642268+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:43.642268+0000","last_peered":"2023-10-18T12:25:43.642268+0000","last_clean":"2023-10-18T12:25:43.642268+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:43.642268+0000","last_undegraded":"2023-10-18T12:25:43.642268+0000","last_fullsized":"2023-10-18T12:25:43.642268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208699+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:25:14.208699+0000","last_peered":"2023-10-18T12:25:14.208699+0000","last_clean":"2023-10-18T12:25:14.208699+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:25:14.208699+0000","last_undegraded":"2023-10-18T12:25:14.208699+0000","last_fullsized":"2023-10-18T12:25:14.208699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225793+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:25:15.225793+0000","last_peered":"2023-10-18T12:25:15.225793+0000","last_clean":"2023-10-18T12:25:15.225793+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:25:15.225793+0000","last_undegraded":"2023-10-18T12:25:15.225793+0000","last_fullsized":"2023-10-18T12:25:15.225793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643090+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:25:43.643090+0000","last_peered":"2023-10-18T12:25:43.643090+0000","last_clean":"2023-10-18T12:25:43.643090+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:25:43.643090+0000","last_undegraded":"2023-10-18T12:25:43.643090+0000","last_fullsized":"2023-10-18T12:25:43.643090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211886+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:25:14.211886+0000","last_peered":"2023-10-18T12:25:14.211886+0000","last_clean":"2023-10-18T12:25:14.211886+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:25:14.211886+0000","last_undegraded":"2023-10-18T12:25:14.211886+0000","last_fullsized":"2023-10-18T12:25:14.211886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.224004+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:25:15.224004+0000","last_peered":"2023-10-18T12:25:15.224004+0000","last_clean":"2023-10-18T12:25:15.224004+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:25:15.224004+0000","last_undegraded":"2023-10-18T12:25:15.224004+0000","last_fullsized":"2023-10-18T12:25:15.224004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644328+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:25:43.644328+0000","last_peered":"2023-10-18T12:25:43.644328+0000","last_clean":"2023-10-18T12:25:43.644328+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:25:43.644328+0000","last_undegraded":"2023-10-18T12:25:43.644328+0000","last_fullsized":"2023-10-18T12:25:43.644328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215695+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:25:14.215695+0000","last_peered":"2023-10-18T12:25:14.215695+0000","last_clean":"2023-10-18T12:25:14.215695+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:25:14.215695+0000","last_undegraded":"2023-10-18T12:25:14.215695+0000","last_fullsized":"2023-10-18T12:25:14.215695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213449+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:26:15.213449+0000","last_peered":"2023-10-18T12:26:15.213449+0000","last_clean":"2023-10-18T12:26:15.213449+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:26:15.213449+0000","last_undegraded":"2023-10-18T12:26:15.213449+0000","last_fullsized":"2023-10-18T12:26:15.213449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208378+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:25:14.208378+0000","last_peered":"2023-10-18T12:25:14.208378+0000","last_clean":"2023-10-18T12:25:14.208378+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:25:14.208378+0000","last_undegraded":"2023-10-18T12:25:14.208378+0000","last_fullsized":"2023-10-18T12:25:14.208378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213184+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:25:15.213184+0000","last_peered":"2023-10-18T12:25:15.213184+0000","last_clean":"2023-10-18T12:25:15.213184+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:25:15.213184+0000","last_undegraded":"2023-10-18T12:25:15.213184+0000","last_fullsized":"2023-10-18T12:25:15.213184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649880+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:25:43.649880+0000","last_peered":"2023-10-18T12:25:43.649880+0000","last_clean":"2023-10-18T12:25:43.649880+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:25:43.649880+0000","last_undegraded":"2023-10-18T12:25:43.649880+0000","last_fullsized":"2023-10-18T12:25:43.649880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209020+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:25:14.209020+0000","last_peered":"2023-10-18T12:25:14.209020+0000","last_clean":"2023-10-18T12:25:14.209020+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:25:14.209020+0000","last_undegraded":"2023-10-18T12:25:14.209020+0000","last_fullsized":"2023-10-18T12:25:14.209020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644627+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:25:43.644627+0000","last_peered":"2023-10-18T12:25:43.644627+0000","last_clean":"2023-10-18T12:25:43.644627+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:25:43.644627+0000","last_undegraded":"2023-10-18T12:25:43.644627+0000","last_fullsized":"2023-10-18T12:25:43.644627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210906+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:25:15.210906+0000","last_peered":"2023-10-18T12:25:15.210906+0000","last_clean":"2023-10-18T12:25:15.210906+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:25:15.210906+0000","last_undegraded":"2023-10-18T12:25:15.210906+0000","last_fullsized":"2023-10-18T12:25:15.210906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210590+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:25:14.210590+0000","last_peered":"2023-10-18T12:25:14.210590+0000","last_clean":"2023-10-18T12:25:14.210590+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:25:14.210590+0000","last_undegraded":"2023-10-18T12:25:14.210590+0000","last_fullsized":"2023-10-18T12:25:14.210590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-10-18T12:24:09.961298+0000","last_change":"2023-10-18T12:26:15.661239+0000","last_active":"2023-10-18T12:24:08.948983+0000","last_peered":"2023-10-18T12:24:08.948983+0000","last_clean":"2023-10-18T12:24:08.948983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:24:09.961298+0000","last_undegraded":"2023-10-18T12:24:09.961298+0000","last_fullsized":"2023-10-18T12:24:09.961298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211540+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:25:14.211540+0000","last_peered":"2023-10-18T12:25:14.211540+0000","last_clean":"2023-10-18T12:25:14.211540+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:25:14.211540+0000","last_undegraded":"2023-10-18T12:25:14.211540+0000","last_fullsized":"2023-10-18T12:25:14.211540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.212426+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:15.212426+0000","last_peered":"2023-10-18T12:25:15.212426+0000","last_clean":"2023-10-18T12:25:15.212426+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:15.212426+0000","last_undegraded":"2023-10-18T12:25:15.212426+0000","last_fullsized":"2023-10-18T12:25:15.212426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223137+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:25:15.223137+0000","last_peered":"2023-10-18T12:25:15.223137+0000","last_clean":"2023-10-18T12:25:15.223137+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:25:15.223137+0000","last_undegraded":"2023-10-18T12:25:15.223137+0000","last_fullsized":"2023-10-18T12:25:15.223137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649534+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:25:43.649534+0000","last_peered":"2023-10-18T12:25:43.649534+0000","last_clean":"2023-10-18T12:25:43.649534+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:25:43.649534+0000","last_undegraded":"2023-10-18T12:25:43.649534+0000","last_fullsized":"2023-10-18T12:25:43.649534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217523+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:25:15.217523+0000","last_peered":"2023-10-18T12:25:15.217523+0000","last_clean":"2023-10-18T12:25:15.217523+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:25:15.217523+0000","last_undegraded":"2023-10-18T12:25:15.217523+0000","last_fullsized":"2023-10-18T12:25:15.217523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214480+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:25:14.214480+0000","last_peered":"2023-10-18T12:25:14.214480+0000","last_clean":"2023-10-18T12:25:14.214480+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:25:14.214480+0000","last_undegraded":"2023-10-18T12:25:14.214480+0000","last_fullsized":"2023-10-18T12:25:14.214480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213970+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:25:15.213970+0000","last_peered":"2023-10-18T12:25:15.213970+0000","last_clean":"2023-10-18T12:25:15.213970+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:25:15.213970+0000","last_undegraded":"2023-10-18T12:25:15.213970+0000","last_fullsized":"2023-10-18T12:25:15.213970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649689+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:25:43.649689+0000","last_peered":"2023-10-18T12:25:43.649689+0000","last_clean":"2023-10-18T12:25:43.649689+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:25:43.649689+0000","last_undegraded":"2023-10-18T12:25:43.649689+0000","last_fullsized":"2023-10-18T12:25:43.649689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.214742+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:25:15.214742+0000","last_peered":"2023-10-18T12:25:15.214742+0000","last_clean":"2023-10-18T12:25:15.214742+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:25:15.214742+0000","last_undegraded":"2023-10-18T12:25:15.214742+0000","last_fullsized":"2023-10-18T12:25:15.214742+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:13.149946+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:26:13.149946+0000","last_peered":"2023-10-18T12:26:13.149946+0000","last_clean":"2023-10-18T12:26:13.149946+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:26:13.149946+0000","last_undegraded":"2023-10-18T12:26:13.149946+0000","last_fullsized":"2023-10-18T12:26:13.149946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643505+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:25:43.643505+0000","last_peered":"2023-10-18T12:25:43.643505+0000","last_clean":"2023-10-18T12:25:43.643505+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:25:43.643505+0000","last_undegraded":"2023-10-18T12:25:43.643505+0000","last_fullsized":"2023-10-18T12:25:43.643505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212653+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:25:14.212653+0000","last_peered":"2023-10-18T12:25:14.212653+0000","last_clean":"2023-10-18T12:25:14.212653+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:25:14.212653+0000","last_undegraded":"2023-10-18T12:25:14.212653+0000","last_fullsized":"2023-10-18T12:25:14.212653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213075+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:14.213075+0000","last_peered":"2023-10-18T12:25:14.213075+0000","last_clean":"2023-10-18T12:25:14.213075+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:14.213075+0000","last_undegraded":"2023-10-18T12:25:14.213075+0000","last_fullsized":"2023-10-18T12:25:14.213075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649714+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:25:43.649714+0000","last_peered":"2023-10-18T12:25:43.649714+0000","last_clean":"2023-10-18T12:25:43.649714+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:25:43.649714+0000","last_undegraded":"2023-10-18T12:25:43.649714+0000","last_fullsized":"2023-10-18T12:25:43.649714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223250+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223250+0000","last_peered":"2023-10-18T12:25:15.223250+0000","last_clean":"2023-10-18T12:25:15.223250+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223250+0000","last_undegraded":"2023-10-18T12:25:15.223250+0000","last_fullsized":"2023-10-18T12:25:15.223250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214524+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:25:14.214524+0000","last_peered":"2023-10-18T12:25:14.214524+0000","last_clean":"2023-10-18T12:25:14.214524+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:25:14.214524+0000","last_undegraded":"2023-10-18T12:25:14.214524+0000","last_fullsized":"2023-10-18T12:25:14.214524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.604171+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:25:43.604171+0000","last_peered":"2023-10-18T12:25:43.604171+0000","last_clean":"2023-10-18T12:25:43.604171+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:25:43.604171+0000","last_undegraded":"2023-10-18T12:25:43.604171+0000","last_fullsized":"2023-10-18T12:25:43.604171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.220890+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:25:15.220890+0000","last_peered":"2023-10-18T12:25:15.220890+0000","last_clean":"2023-10-18T12:25:15.220890+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:25:15.220890+0000","last_undegraded":"2023-10-18T12:25:15.220890+0000","last_fullsized":"2023-10-18T12:25:15.220890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210834+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:25:14.210834+0000","last_peered":"2023-10-18T12:25:14.210834+0000","last_clean":"2023-10-18T12:25:14.210834+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:25:14.210834+0000","last_undegraded":"2023-10-18T12:25:14.210834+0000","last_fullsized":"2023-10-18T12:25:14.210834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.651462+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:25:43.651462+0000","last_peered":"2023-10-18T12:25:43.651462+0000","last_clean":"2023-10-18T12:25:43.651462+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:25:43.651462+0000","last_undegraded":"2023-10-18T12:25:43.651462+0000","last_fullsized":"2023-10-18T12:25:43.651462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211350+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:14.211350+0000","last_peered":"2023-10-18T12:25:14.211350+0000","last_clean":"2023-10-18T12:25:14.211350+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:14.211350+0000","last_undegraded":"2023-10-18T12:25:14.211350+0000","last_fullsized":"2023-10-18T12:25:14.211350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210884+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:25:14.210884+0000","last_peered":"2023-10-18T12:25:14.210884+0000","last_clean":"2023-10-18T12:25:14.210884+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:25:14.210884+0000","last_undegraded":"2023-10-18T12:25:14.210884+0000","last_fullsized":"2023-10-18T12:25:14.210884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648198+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:25:43.648198+0000","last_peered":"2023-10-18T12:25:43.648198+0000","last_clean":"2023-10-18T12:25:43.648198+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:25:43.648198+0000","last_undegraded":"2023-10-18T12:25:43.648198+0000","last_fullsized":"2023-10-18T12:25:43.648198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212507+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:14.212507+0000","last_peered":"2023-10-18T12:25:14.212507+0000","last_clean":"2023-10-18T12:25:14.212507+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:14.212507+0000","last_undegraded":"2023-10-18T12:25:14.212507+0000","last_fullsized":"2023-10-18T12:25:14.212507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216327+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:25:14.216327+0000","last_peered":"2023-10-18T12:25:14.216327+0000","last_clean":"2023-10-18T12:25:14.216327+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:25:14.216327+0000","last_undegraded":"2023-10-18T12:25:14.216327+0000","last_fullsized":"2023-10-18T12:25:14.216327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652574+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:25:43.652574+0000","last_peered":"2023-10-18T12:25:43.652574+0000","last_clean":"2023-10-18T12:25:43.652574+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:25:43.652574+0000","last_undegraded":"2023-10-18T12:25:43.652574+0000","last_fullsized":"2023-10-18T12:25:43.652574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214357+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:25:14.214357+0000","last_peered":"2023-10-18T12:25:14.214357+0000","last_clean":"2023-10-18T12:25:14.214357+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:25:14.214357+0000","last_undegraded":"2023-10-18T12:25:14.214357+0000","last_fullsized":"2023-10-18T12:25:14.214357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211315+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:25:14.211315+0000","last_peered":"2023-10-18T12:25:14.211315+0000","last_clean":"2023-10-18T12:25:14.211315+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:25:14.211315+0000","last_undegraded":"2023-10-18T12:25:14.211315+0000","last_fullsized":"2023-10-18T12:25:14.211315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644409+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:43.644409+0000","last_peered":"2023-10-18T12:25:43.644409+0000","last_clean":"2023-10-18T12:25:43.644409+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:43.644409+0000","last_undegraded":"2023-10-18T12:25:43.644409+0000","last_fullsized":"2023-10-18T12:25:43.644409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218308+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:25:14.218308+0000","last_peered":"2023-10-18T12:25:14.218308+0000","last_clean":"2023-10-18T12:25:14.218308+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:25:14.218308+0000","last_undegraded":"2023-10-18T12:25:14.218308+0000","last_fullsized":"2023-10-18T12:25:14.218308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211578+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:25:14.211578+0000","last_peered":"2023-10-18T12:25:14.211578+0000","last_clean":"2023-10-18T12:25:14.211578+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:25:14.211578+0000","last_undegraded":"2023-10-18T12:25:14.211578+0000","last_fullsized":"2023-10-18T12:25:14.211578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639071+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:43.639071+0000","last_peered":"2023-10-18T12:25:43.639071+0000","last_clean":"2023-10-18T12:25:43.639071+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:43.639071+0000","last_undegraded":"2023-10-18T12:25:43.639071+0000","last_fullsized":"2023-10-18T12:25:43.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210493+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:25:14.210493+0000","last_peered":"2023-10-18T12:25:14.210493+0000","last_clean":"2023-10-18T12:25:14.210493+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:25:14.210493+0000","last_undegraded":"2023-10-18T12:25:14.210493+0000","last_fullsized":"2023-10-18T12:25:14.210493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225441+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:25:15.225441+0000","last_peered":"2023-10-18T12:25:15.225441+0000","last_clean":"2023-10-18T12:25:15.225441+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:25:15.225441+0000","last_undegraded":"2023-10-18T12:25:15.225441+0000","last_fullsized":"2023-10-18T12:25:15.225441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639284+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:43.639284+0000","last_peered":"2023-10-18T12:25:43.639284+0000","last_clean":"2023-10-18T12:25:43.639284+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:43.639284+0000","last_undegraded":"2023-10-18T12:25:43.639284+0000","last_fullsized":"2023-10-18T12:25:43.639284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.638348+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:25:43.638348+0000","last_peered":"2023-10-18T12:25:43.638348+0000","last_clean":"2023-10-18T12:25:43.638348+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:25:43.638348+0000","last_undegraded":"2023-10-18T12:25:43.638348+0000","last_fullsized":"2023-10-18T12:25:43.638348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643095+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:25:43.643095+0000","last_peered":"2023-10-18T12:25:43.643095+0000","last_clean":"2023-10-18T12:25:43.643095+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:25:43.643095+0000","last_undegraded":"2023-10-18T12:25:43.643095+0000","last_fullsized":"2023-10-18T12:25:43.643095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646655+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:25:43.646655+0000","last_peered":"2023-10-18T12:25:43.646655+0000","last_clean":"2023-10-18T12:25:43.646655+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:25:43.646655+0000","last_undegraded":"2023-10-18T12:25:43.646655+0000","last_fullsized":"2023-10-18T12:25:43.646655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656634+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:25:43.656634+0000","last_peered":"2023-10-18T12:25:43.656634+0000","last_clean":"2023-10-18T12:25:43.656634+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:25:43.656634+0000","last_undegraded":"2023-10-18T12:25:43.656634+0000","last_fullsized":"2023-10-18T12:25:43.656634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.655509+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:25:43.655509+0000","last_peered":"2023-10-18T12:25:43.655509+0000","last_clean":"2023-10-18T12:25:43.655509+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:25:43.655509+0000","last_undegraded":"2023-10-18T12:25:43.655509+0000","last_fullsized":"2023-10-18T12:25:43.655509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639525+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:25:43.639525+0000","last_peered":"2023-10-18T12:25:43.639525+0000","last_clean":"2023-10-18T12:25:43.639525+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:25:43.639525+0000","last_undegraded":"2023-10-18T12:25:43.639525+0000","last_fullsized":"2023-10-18T12:25:43.639525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644102+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:25:43.644102+0000","last_peered":"2023-10-18T12:25:43.644102+0000","last_clean":"2023-10-18T12:25:43.644102+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:25:43.644102+0000","last_undegraded":"2023-10-18T12:25:43.644102+0000","last_fullsized":"2023-10-18T12:25:43.644102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644112+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:25:43.644112+0000","last_peered":"2023-10-18T12:25:43.644112+0000","last_clean":"2023-10-18T12:25:43.644112+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:25:43.644112+0000","last_undegraded":"2023-10-18T12:25:43.644112+0000","last_fullsized":"2023-10-18T12:25:43.644112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636676+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:25:43.636676+0000","last_peered":"2023-10-18T12:25:43.636676+0000","last_clean":"2023-10-18T12:25:43.636676+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:25:43.636676+0000","last_undegraded":"2023-10-18T12:25:43.636676+0000","last_fullsized":"2023-10-18T12:25:43.636676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636709+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:25:43.636709+0000","last_peered":"2023-10-18T12:25:43.636709+0000","last_clean":"2023-10-18T12:25:43.636709+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:25:43.636709+0000","last_undegraded":"2023-10-18T12:25:43.636709+0000","last_fullsized":"2023-10-18T12:25:43.636709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643394+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:25:43.643394+0000","last_peered":"2023-10-18T12:25:43.643394+0000","last_clean":"2023-10-18T12:25:43.643394+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:25:43.643394+0000","last_undegraded":"2023-10-18T12:25:43.643394+0000","last_fullsized":"2023-10-18T12:25:43.643394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.657911+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:25:43.657911+0000","last_peered":"2023-10-18T12:25:43.657911+0000","last_clean":"2023-10-18T12:25:43.657911+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:25:43.657911+0000","last_undegraded":"2023-10-18T12:25:43.657911+0000","last_fullsized":"2023-10-18T12:25:43.657911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643480+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:25:43.643480+0000","last_peered":"2023-10-18T12:25:43.643480+0000","last_clean":"2023-10-18T12:25:43.643480+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:25:43.643480+0000","last_undegraded":"2023-10-18T12:25:43.643480+0000","last_fullsized":"2023-10-18T12:25:43.643480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62248,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":211,"num_read_kb":168,"num_write":81,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":197502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":1.147,"15min":1.147},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.745}]},{"osd":2,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88400000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":1.238,"15min":1.238},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75600000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":1.661,"15min":1.661},"last":0.84599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90400000000000003,"5min":1.361,"15min":1.361},"last":0.872},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":1,"15min":1},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.91100000000000003,"5min":1.726,"15min":1.726},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.012,"5min":1.401,"15min":1.401},"last":0.95899999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89000000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.92400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1161040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.54500000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.435,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.499},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63300000000000001,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.999,"15min":0.999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72499999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.432}]},{"osd":5,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84599999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.876,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.77600000000000002,"5min":1.099,"15min":1.099},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82799999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90400000000000003,"5min":1.071,"15min":1.071},"last":0.62},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.044,"15min":1.044},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63700000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.67500000000000004}]}]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":527643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0089999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.92700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.753},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.86199999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1739999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1519999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.91500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.871},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.94599999999999995,"15min":0.95199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":6.2220000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.69699999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.89100000000000001},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2430000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78000000000000003}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.432},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.501}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0149999999999999,"5min":1.044,"15min":1.044},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.72399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0640000000000001,"5min":1.26,"15min":1.26},"last":0.754}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.68300000000000005},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76800000000000002,"15min":0.72799999999999998},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.107,"15min":1.107},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66700000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1108170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.433,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.501},{"interface":"front","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60099999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35799999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.999,"15min":0.999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.137,"15min":1.137},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81299999999999994,"5min":1.071,"15min":1.071},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.66500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.98199999999999998,"5min":1.02,"15min":1.02},"last":0.72799999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95399999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69199999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70699999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":509028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.53000000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.622},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.753}]},{"osd":3,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.68799999999999994,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.9199999999999999,"5min":6.9199999999999999,"15min":6.9199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.026,"5min":0.81899999999999995,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":7.2270000000000003,"5min":7.2270000000000003,"15min":7.2270000000000003},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.016,"5min":1.234,"15min":1.234},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74299999999999999,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.28,"5min":1.849,"15min":1.849},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84999999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.88,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":7.7400000000000002,"5min":7.7400000000000002,"15min":7.7400000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.69699999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.90800000000000003,"15min":0.88900000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.4969999999999999,"5min":7.4969999999999999,"15min":7.4969999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95499999999999996,"15min":0.90300000000000002},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3240000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":570478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.161,"15min":1.161},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94699999999999995,"5min":1.454,"15min":1.454},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.57299999999999995},"min":{"1min":0.52000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88500000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71899999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.64800000000000002,"15min":0.59099999999999997},"min":{"1min":0.63200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":1.298,"15min":1.298},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.52700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89400000000000002,"5min":1.016,"15min":1.016},"last":0.88200000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.002}]}]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":509118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.65200000000000002,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.30299999999999999},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65600000000000003,"15min":0.55200000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.32400000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.60699999999999998},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.84099999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1479999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.024,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.65800000000000003,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.69699999999999995,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.498}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.66200000000000003},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.65700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.755}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69899999999999995,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.751,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":0.92100000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86799999999999999,"15min":0.84599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.61199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.89300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.876},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.89300000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.03,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93899999999999995,"15min":0.92400000000000004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":0.90000000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":575618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.26100000000000001},"max":{"1min":0.92100000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.54600000000000004},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.626},{"interface":"front","average":{"1min":0.745,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.86899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.34000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73799999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.42299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.58399999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.753},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85399999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76500000000000001,"15min":0.74099999999999999},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.254,"15min":1.254},"last":0.995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.74299999999999999,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91600000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":509132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.51400000000000001,"15min":0.436},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7880000000000003,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.52700000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.73599999999999999,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.622},{"interface":"front","average":{"1min":1.026,"5min":0.68999999999999995,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":5.0579999999999998,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.73699999999999999,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.085,"5min":0.77800000000000002,"15min":0.70599999999999996},"min":{"1min":0.60999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.78400000000000003,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.60099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.752,"15min":0.69499999999999995},"min":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.83299999999999996,"15min":0.753},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.984,"5min":4.984,"15min":4.984},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3789999999999996,"5min":5.3789999999999996,"15min":5.3789999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.86199999999999999},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93799999999999994,"15min":0.90500000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.83099999999999996}]},{"osd":11,"last update":"Wed Oct 18 12:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.016,"15min":1.052},"min":{"1min":0.61899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2030000000000001,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.111,"15min":1.155},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2170000000000001,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54800000000000004,"5min":0.621,"15min":0.621},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57099999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.876,"5min":1.04,"15min":1.04},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.52200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":1.22,"15min":1.22},"last":0.75}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61099999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80200000000000005,"15min":0.81100000000000005},"min":{"1min":0.65900000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.02,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.55700000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65000000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.501}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.88}]}]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1103343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":0.999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88100000000000001}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94099999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.752},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.01},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":4.548,"15min":4.548},"last":0.99399999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.745,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73999999999999999,"15min":0.70799999999999996},"min":{"1min":0.60699999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:26:16.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:26:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 -- 172.21.15.70:0/493894172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05841006a0 msgr2=0x7f0584100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 --2- 172.21.15.70:0/493894172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05841006a0 0x7f0584100b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f056c009990 tx=0x7f056c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 -- 172.21.15.70:0/493894172 shutdown_connections 2023-10-18T12:26:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 --2- 172.21.15.70:0/493894172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0584101060 0x7f058410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 --2- 172.21.15.70:0/493894172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05841066b0 0x7f0584106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 --2- 172.21.15.70:0/493894172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05841006a0 0x7f0584100b20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 -- 172.21.15.70:0/493894172 >> 172.21.15.70:0/493894172 conn(0x7f05840fc170 msgr2=0x7f05840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 -- 172.21.15.70:0/493894172 shutdown_connections 2023-10-18T12:26:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.073+0000 7f05897ff700 1 -- 172.21.15.70:0/493894172 wait complete. 2023-10-18T12:26:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 Processor -- start 2023-10-18T12:26:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 -- start start 2023-10-18T12:26:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05841006a0 0x7f058419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 0x7f058419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05841066b0 0x7f0584196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584196f90 con 0x7f0584101060 2023-10-18T12:26:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584197100 con 0x7f05841006a0 2023-10-18T12:26:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.075+0000 7f05897ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841972c0 con 0x7f05841066b0 2023-10-18T12:26:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 0x7f058419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f0582ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05841006a0 0x7f058419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 0x7f058419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40112/0 (socket says 172.21.15.70:40112) 2023-10-18T12:26:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 -- 172.21.15.70:0/448188085 learned_addr learned my addr 172.21.15.70:0/448188085 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f05837fe700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05841066b0 0x7f0584196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 -- 172.21.15.70:0/448188085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05841066b0 msgr2=0x7f0584196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05841066b0 0x7f0584196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 -- 172.21.15.70:0/448188085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05841006a0 msgr2=0x7f058419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05841006a0 0x7f058419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 -- 172.21.15.70:0/448188085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0095f0 con 0x7f0584101060 2023-10-18T12:26:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f0582ffd700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05841006a0 0x7f058419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f057bfff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 0x7f058419cd40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.076+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c01d070 con 0x7f0584101060 2023-10-18T12:26:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.077+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056c022470 con 0x7f0584101060 2023-10-18T12:26:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.077+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0584197540 con 0x7f0584101060 2023-10-18T12:26:17.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.077+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c017650 con 0x7f0584101060 2023-10-18T12:26:17.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.077+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05841a37f0 con 0x7f0584101060 2023-10-18T12:26:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.078+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584108c30 con 0x7f0584101060 2023-10-18T12:26:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.080+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f056c0225e0 con 0x7f0584101060 2023-10-18T12:26:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.082+0000 7f0580ff9700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05700a14d0 0x7f05700a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.082+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f056c0ceba0 con 0x7f0584101060 2023-10-18T12:26:17.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.082+0000 7f0582ffd700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05700a14d0 0x7f05700a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.083+0000 7f0582ffd700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05700a14d0 0x7f05700a3990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0574009fd0 tx=0x7f0574009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:17.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.085+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f056c097e70 con 0x7f0584101060 2023-10-18T12:26:17.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.286+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0584197df0 con 0x7f05700a14d0 2023-10-18T12:26:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.299+0000 7f0580ff9700 1 -- 172.21.15.70:0/448188085 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427323 (secure 0 0 0) 0x7f0584197df0 con 0x7f05700a14d0 2023-10-18T12:26:17.307 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:17.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05700a14d0 msgr2=0x7f05700a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05700a14d0 0x7f05700a3990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0574009fd0 tx=0x7f0574009260 comp rx=0 tx=0).stop 2023-10-18T12:26:17.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 msgr2=0x7f058419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 0x7f058419cd40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c005740 comp rx=0 tx=0).stop 2023-10-18T12:26:17.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 shutdown_connections 2023-10-18T12:26:17.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05841066b0 0x7f0584196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05700a14d0 0x7f05700a3990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05841006a0 0x7f058419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 --2- 172.21.15.70:0/448188085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0584101060 0x7f058419cd40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 >> 172.21.15.70:0/448188085 conn(0x7f05840fc170 msgr2=0x7f05840fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:17.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 shutdown_connections 2023-10-18T12:26:17.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.304+0000 7f05897ff700 1 -- 172.21.15.70:0/448188085 wait complete. 2023-10-18T12:26:17.316 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:26:17.326 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-18T12:26:16.214128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652862,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":347,"num_read_kb":287,"num_write":264,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338200,"kb_used_data":14004,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603624,"statfs":{"total":1151940427776,"available":1151594110976,"internally_reserved":0,"allocated":14340096,"data_stored":8113568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.591689"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650976+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:25:43.650976+0000","last_peered":"2023-10-18T12:25:43.650976+0000","last_clean":"2023-10-18T12:25:43.650976+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:25:43.650976+0000","last_undegraded":"2023-10-18T12:25:43.650976+0000","last_fullsized":"2023-10-18T12:25:43.650976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649821+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:25:43.649821+0000","last_peered":"2023-10-18T12:25:43.649821+0000","last_clean":"2023-10-18T12:25:43.649821+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:25:43.649821+0000","last_undegraded":"2023-10-18T12:25:43.649821+0000","last_fullsized":"2023-10-18T12:25:43.649821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650939+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:25:43.650939+0000","last_peered":"2023-10-18T12:25:43.650939+0000","last_clean":"2023-10-18T12:25:43.650939+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:25:43.650939+0000","last_undegraded":"2023-10-18T12:25:43.650939+0000","last_fullsized":"2023-10-18T12:25:43.650939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.640858+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:25:43.640858+0000","last_peered":"2023-10-18T12:25:43.640858+0000","last_clean":"2023-10-18T12:25:43.640858+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:25:43.640858+0000","last_undegraded":"2023-10-18T12:25:43.640858+0000","last_fullsized":"2023-10-18T12:25:43.640858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.631268+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:25:43.631268+0000","last_peered":"2023-10-18T12:25:43.631268+0000","last_clean":"2023-10-18T12:25:43.631268+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:25:43.631268+0000","last_undegraded":"2023-10-18T12:25:43.631268+0000","last_fullsized":"2023-10-18T12:25:43.631268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643204+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:25:43.643204+0000","last_peered":"2023-10-18T12:25:43.643204+0000","last_clean":"2023-10-18T12:25:43.643204+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:25:43.643204+0000","last_undegraded":"2023-10-18T12:25:43.643204+0000","last_fullsized":"2023-10-18T12:25:43.643204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639863+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:43.639863+0000","last_peered":"2023-10-18T12:25:43.639863+0000","last_clean":"2023-10-18T12:25:43.639863+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:43.639863+0000","last_undegraded":"2023-10-18T12:25:43.639863+0000","last_fullsized":"2023-10-18T12:25:43.639863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639818+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:25:43.639818+0000","last_peered":"2023-10-18T12:25:43.639818+0000","last_clean":"2023-10-18T12:25:43.639818+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:25:43.639818+0000","last_undegraded":"2023-10-18T12:25:43.639818+0000","last_fullsized":"2023-10-18T12:25:43.639818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642671+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:25:43.642671+0000","last_peered":"2023-10-18T12:25:43.642671+0000","last_clean":"2023-10-18T12:25:43.642671+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:25:43.642671+0000","last_undegraded":"2023-10-18T12:25:43.642671+0000","last_fullsized":"2023-10-18T12:25:43.642671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.645379+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:25:43.645379+0000","last_peered":"2023-10-18T12:25:43.645379+0000","last_clean":"2023-10-18T12:25:43.645379+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:25:43.645379+0000","last_undegraded":"2023-10-18T12:25:43.645379+0000","last_fullsized":"2023-10-18T12:25:43.645379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644261+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:43.644261+0000","last_peered":"2023-10-18T12:25:43.644261+0000","last_clean":"2023-10-18T12:25:43.644261+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:43.644261+0000","last_undegraded":"2023-10-18T12:25:43.644261+0000","last_fullsized":"2023-10-18T12:25:43.644261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633544+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:43.633544+0000","last_peered":"2023-10-18T12:25:43.633544+0000","last_clean":"2023-10-18T12:25:43.633544+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:43.633544+0000","last_undegraded":"2023-10-18T12:25:43.633544+0000","last_fullsized":"2023-10-18T12:25:43.633544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.658873+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:25:43.658873+0000","last_peered":"2023-10-18T12:25:43.658873+0000","last_clean":"2023-10-18T12:25:43.658873+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:25:43.658873+0000","last_undegraded":"2023-10-18T12:25:43.658873+0000","last_fullsized":"2023-10-18T12:25:43.658873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646839+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:25:43.646839+0000","last_peered":"2023-10-18T12:25:43.646839+0000","last_clean":"2023-10-18T12:25:43.646839+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:25:43.646839+0000","last_undegraded":"2023-10-18T12:25:43.646839+0000","last_fullsized":"2023-10-18T12:25:43.646839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649694+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:25:43.649694+0000","last_peered":"2023-10-18T12:25:43.649694+0000","last_clean":"2023-10-18T12:25:43.649694+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:25:43.649694+0000","last_undegraded":"2023-10-18T12:25:43.649694+0000","last_fullsized":"2023-10-18T12:25:43.649694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648146+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:25:43.648146+0000","last_peered":"2023-10-18T12:25:43.648146+0000","last_clean":"2023-10-18T12:25:43.648146+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:25:43.648146+0000","last_undegraded":"2023-10-18T12:25:43.648146+0000","last_fullsized":"2023-10-18T12:25:43.648146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652031+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:25:43.652031+0000","last_peered":"2023-10-18T12:25:43.652031+0000","last_clean":"2023-10-18T12:25:43.652031+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:25:43.652031+0000","last_undegraded":"2023-10-18T12:25:43.652031+0000","last_fullsized":"2023-10-18T12:25:43.652031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639903+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:25:43.639903+0000","last_peered":"2023-10-18T12:25:43.639903+0000","last_clean":"2023-10-18T12:25:43.639903+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:25:43.639903+0000","last_undegraded":"2023-10-18T12:25:43.639903+0000","last_fullsized":"2023-10-18T12:25:43.639903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643796+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:25:43.643796+0000","last_peered":"2023-10-18T12:25:43.643796+0000","last_clean":"2023-10-18T12:25:43.643796+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:25:43.643796+0000","last_undegraded":"2023-10-18T12:25:43.643796+0000","last_fullsized":"2023-10-18T12:25:43.643796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212928+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:25:14.212928+0000","last_peered":"2023-10-18T12:25:14.212928+0000","last_clean":"2023-10-18T12:25:14.212928+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:25:14.212928+0000","last_undegraded":"2023-10-18T12:25:14.212928+0000","last_fullsized":"2023-10-18T12:25:14.212928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213691+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:26:15.213691+0000","last_peered":"2023-10-18T12:26:15.213691+0000","last_clean":"2023-10-18T12:26:15.213691+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:26:15.213691+0000","last_undegraded":"2023-10-18T12:26:15.213691+0000","last_fullsized":"2023-10-18T12:26:15.213691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639283+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:25:43.639283+0000","last_peered":"2023-10-18T12:25:43.639283+0000","last_clean":"2023-10-18T12:25:43.639283+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:25:43.639283+0000","last_undegraded":"2023-10-18T12:25:43.639283+0000","last_fullsized":"2023-10-18T12:25:43.639283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210737+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:14.210737+0000","last_peered":"2023-10-18T12:25:14.210737+0000","last_clean":"2023-10-18T12:25:14.210737+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:14.210737+0000","last_undegraded":"2023-10-18T12:25:14.210737+0000","last_fullsized":"2023-10-18T12:25:14.210737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.216358+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:25:15.216358+0000","last_peered":"2023-10-18T12:25:15.216358+0000","last_clean":"2023-10-18T12:25:15.216358+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:25:15.216358+0000","last_undegraded":"2023-10-18T12:25:15.216358+0000","last_fullsized":"2023-10-18T12:25:15.216358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656048+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:25:43.656048+0000","last_peered":"2023-10-18T12:25:43.656048+0000","last_clean":"2023-10-18T12:25:43.656048+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:25:43.656048+0000","last_undegraded":"2023-10-18T12:25:43.656048+0000","last_fullsized":"2023-10-18T12:25:43.656048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211459+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:14.211459+0000","last_peered":"2023-10-18T12:25:14.211459+0000","last_clean":"2023-10-18T12:25:14.211459+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:14.211459+0000","last_undegraded":"2023-10-18T12:25:14.211459+0000","last_fullsized":"2023-10-18T12:25:14.211459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215742+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:25:14.215742+0000","last_peered":"2023-10-18T12:25:14.215742+0000","last_clean":"2023-10-18T12:25:14.215742+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:25:14.215742+0000","last_undegraded":"2023-10-18T12:25:14.215742+0000","last_fullsized":"2023-10-18T12:25:14.215742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210540+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:14.210540+0000","last_peered":"2023-10-18T12:25:14.210540+0000","last_clean":"2023-10-18T12:25:14.210540+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:14.210540+0000","last_undegraded":"2023-10-18T12:25:14.210540+0000","last_fullsized":"2023-10-18T12:25:14.210540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216726+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:25:14.216726+0000","last_peered":"2023-10-18T12:25:14.216726+0000","last_clean":"2023-10-18T12:25:14.216726+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:25:14.216726+0000","last_undegraded":"2023-10-18T12:25:14.216726+0000","last_fullsized":"2023-10-18T12:25:14.216726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644740+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:25:43.644740+0000","last_peered":"2023-10-18T12:25:43.644740+0000","last_clean":"2023-10-18T12:25:43.644740+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:25:43.644740+0000","last_undegraded":"2023-10-18T12:25:43.644740+0000","last_fullsized":"2023-10-18T12:25:43.644740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217322+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:25:15.217322+0000","last_peered":"2023-10-18T12:25:15.217322+0000","last_clean":"2023-10-18T12:25:15.217322+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:25:15.217322+0000","last_undegraded":"2023-10-18T12:25:15.217322+0000","last_fullsized":"2023-10-18T12:25:15.217322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211310+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:25:14.211310+0000","last_peered":"2023-10-18T12:25:14.211310+0000","last_clean":"2023-10-18T12:25:14.211310+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:25:14.211310+0000","last_undegraded":"2023-10-18T12:25:14.211310+0000","last_fullsized":"2023-10-18T12:25:14.211310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633052+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:25:43.633052+0000","last_peered":"2023-10-18T12:25:43.633052+0000","last_clean":"2023-10-18T12:25:43.633052+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:25:43.633052+0000","last_undegraded":"2023-10-18T12:25:43.633052+0000","last_fullsized":"2023-10-18T12:25:43.633052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211137+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:25:14.211137+0000","last_peered":"2023-10-18T12:25:14.211137+0000","last_clean":"2023-10-18T12:25:14.211137+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:25:14.211137+0000","last_undegraded":"2023-10-18T12:25:14.211137+0000","last_fullsized":"2023-10-18T12:25:14.211137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210401+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:25:14.210401+0000","last_peered":"2023-10-18T12:25:14.210401+0000","last_clean":"2023-10-18T12:25:14.210401+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:25:14.210401+0000","last_undegraded":"2023-10-18T12:25:14.210401+0000","last_fullsized":"2023-10-18T12:25:14.210401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.645401+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:25:43.645401+0000","last_peered":"2023-10-18T12:25:43.645401+0000","last_clean":"2023-10-18T12:25:43.645401+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:25:43.645401+0000","last_undegraded":"2023-10-18T12:25:43.645401+0000","last_fullsized":"2023-10-18T12:25:43.645401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215355+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:25:14.215355+0000","last_peered":"2023-10-18T12:25:14.215355+0000","last_clean":"2023-10-18T12:25:14.215355+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:25:14.215355+0000","last_undegraded":"2023-10-18T12:25:14.215355+0000","last_fullsized":"2023-10-18T12:25:14.215355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207597+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:25:14.207597+0000","last_peered":"2023-10-18T12:25:14.207597+0000","last_clean":"2023-10-18T12:25:14.207597+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:25:14.207597+0000","last_undegraded":"2023-10-18T12:25:14.207597+0000","last_fullsized":"2023-10-18T12:25:14.207597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653839+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:25:43.653839+0000","last_peered":"2023-10-18T12:25:43.653839+0000","last_clean":"2023-10-18T12:25:43.653839+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:25:43.653839+0000","last_undegraded":"2023-10-18T12:25:43.653839+0000","last_fullsized":"2023-10-18T12:25:43.653839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217590+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:25:15.217590+0000","last_peered":"2023-10-18T12:25:15.217590+0000","last_clean":"2023-10-18T12:25:15.217590+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:25:15.217590+0000","last_undegraded":"2023-10-18T12:25:15.217590+0000","last_fullsized":"2023-10-18T12:25:15.217590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210651+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:25:14.210651+0000","last_peered":"2023-10-18T12:25:14.210651+0000","last_clean":"2023-10-18T12:25:14.210651+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:25:14.210651+0000","last_undegraded":"2023-10-18T12:25:14.210651+0000","last_fullsized":"2023-10-18T12:25:14.210651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639762+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:25:43.639762+0000","last_peered":"2023-10-18T12:25:43.639762+0000","last_clean":"2023-10-18T12:25:43.639762+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:25:43.639762+0000","last_undegraded":"2023-10-18T12:25:43.639762+0000","last_fullsized":"2023-10-18T12:25:43.639762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642014+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:25:43.642014+0000","last_peered":"2023-10-18T12:25:43.642014+0000","last_clean":"2023-10-18T12:25:43.642014+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:25:43.642014+0000","last_undegraded":"2023-10-18T12:25:43.642014+0000","last_fullsized":"2023-10-18T12:25:43.642014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208294+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:25:15.208294+0000","last_peered":"2023-10-18T12:25:15.208294+0000","last_clean":"2023-10-18T12:25:15.208294+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:25:15.208294+0000","last_undegraded":"2023-10-18T12:25:15.208294+0000","last_fullsized":"2023-10-18T12:25:15.208294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211053+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:14.211053+0000","last_peered":"2023-10-18T12:25:14.211053+0000","last_clean":"2023-10-18T12:25:14.211053+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:14.211053+0000","last_undegraded":"2023-10-18T12:25:14.211053+0000","last_fullsized":"2023-10-18T12:25:14.211053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652650+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:25:43.652650+0000","last_peered":"2023-10-18T12:25:43.652650+0000","last_clean":"2023-10-18T12:25:43.652650+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:25:43.652650+0000","last_undegraded":"2023-10-18T12:25:43.652650+0000","last_fullsized":"2023-10-18T12:25:43.652650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211621+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:14.211621+0000","last_peered":"2023-10-18T12:25:14.211621+0000","last_clean":"2023-10-18T12:25:14.211621+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:14.211621+0000","last_undegraded":"2023-10-18T12:25:14.211621+0000","last_fullsized":"2023-10-18T12:25:14.211621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211404+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:25:14.211404+0000","last_peered":"2023-10-18T12:25:14.211404+0000","last_clean":"2023-10-18T12:25:14.211404+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:25:14.211404+0000","last_undegraded":"2023-10-18T12:25:14.211404+0000","last_fullsized":"2023-10-18T12:25:14.211404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653417+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:25:43.653417+0000","last_peered":"2023-10-18T12:25:43.653417+0000","last_clean":"2023-10-18T12:25:43.653417+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:25:43.653417+0000","last_undegraded":"2023-10-18T12:25:43.653417+0000","last_fullsized":"2023-10-18T12:25:43.653417+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208356+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:25:15.208356+0000","last_peered":"2023-10-18T12:25:15.208356+0000","last_clean":"2023-10-18T12:25:15.208356+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:25:15.208356+0000","last_undegraded":"2023-10-18T12:25:15.208356+0000","last_fullsized":"2023-10-18T12:25:15.208356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:11.324634+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:26:11.324634+0000","last_peered":"2023-10-18T12:26:11.324634+0000","last_clean":"2023-10-18T12:26:11.324634+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:26:11.324634+0000","last_undegraded":"2023-10-18T12:26:11.324634+0000","last_fullsized":"2023-10-18T12:26:11.324634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61450,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":206,"num_read_kb":163,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636586+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:25:43.636586+0000","last_peered":"2023-10-18T12:25:43.636586+0000","last_clean":"2023-10-18T12:25:43.636586+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:25:43.636586+0000","last_undegraded":"2023-10-18T12:25:43.636586+0000","last_fullsized":"2023-10-18T12:25:43.636586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211801+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:25:14.211801+0000","last_peered":"2023-10-18T12:25:14.211801+0000","last_clean":"2023-10-18T12:25:14.211801+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:25:14.211801+0000","last_undegraded":"2023-10-18T12:25:14.211801+0000","last_fullsized":"2023-10-18T12:25:14.211801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211560+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:25:14.211560+0000","last_peered":"2023-10-18T12:25:14.211560+0000","last_clean":"2023-10-18T12:25:14.211560+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:25:14.211560+0000","last_undegraded":"2023-10-18T12:25:14.211560+0000","last_fullsized":"2023-10-18T12:25:14.211560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649605+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:25:43.649605+0000","last_peered":"2023-10-18T12:25:43.649605+0000","last_clean":"2023-10-18T12:25:43.649605+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:25:43.649605+0000","last_undegraded":"2023-10-18T12:25:43.649605+0000","last_fullsized":"2023-10-18T12:25:43.649605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214581+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:25:14.214581+0000","last_peered":"2023-10-18T12:25:14.214581+0000","last_clean":"2023-10-18T12:25:14.214581+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:25:14.214581+0000","last_undegraded":"2023-10-18T12:25:14.214581+0000","last_fullsized":"2023-10-18T12:25:14.214581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211113+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:25:14.211113+0000","last_peered":"2023-10-18T12:25:14.211113+0000","last_clean":"2023-10-18T12:25:14.211113+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:25:14.211113+0000","last_undegraded":"2023-10-18T12:25:14.211113+0000","last_fullsized":"2023-10-18T12:25:14.211113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.654349+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:43.654349+0000","last_peered":"2023-10-18T12:25:43.654349+0000","last_clean":"2023-10-18T12:25:43.654349+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:43.654349+0000","last_undegraded":"2023-10-18T12:25:43.654349+0000","last_fullsized":"2023-10-18T12:25:43.654349+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208931+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:25:15.208931+0000","last_peered":"2023-10-18T12:25:15.208931+0000","last_clean":"2023-10-18T12:25:15.208931+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:25:15.208931+0000","last_undegraded":"2023-10-18T12:25:15.208931+0000","last_fullsized":"2023-10-18T12:25:15.208931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210996+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:14.210996+0000","last_peered":"2023-10-18T12:25:14.210996+0000","last_clean":"2023-10-18T12:25:14.210996+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:14.210996+0000","last_undegraded":"2023-10-18T12:25:14.210996+0000","last_fullsized":"2023-10-18T12:25:14.210996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642268+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:43.642268+0000","last_peered":"2023-10-18T12:25:43.642268+0000","last_clean":"2023-10-18T12:25:43.642268+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:43.642268+0000","last_undegraded":"2023-10-18T12:25:43.642268+0000","last_fullsized":"2023-10-18T12:25:43.642268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208699+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:25:14.208699+0000","last_peered":"2023-10-18T12:25:14.208699+0000","last_clean":"2023-10-18T12:25:14.208699+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:25:14.208699+0000","last_undegraded":"2023-10-18T12:25:14.208699+0000","last_fullsized":"2023-10-18T12:25:14.208699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225793+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:25:15.225793+0000","last_peered":"2023-10-18T12:25:15.225793+0000","last_clean":"2023-10-18T12:25:15.225793+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:25:15.225793+0000","last_undegraded":"2023-10-18T12:25:15.225793+0000","last_fullsized":"2023-10-18T12:25:15.225793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643090+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:25:43.643090+0000","last_peered":"2023-10-18T12:25:43.643090+0000","last_clean":"2023-10-18T12:25:43.643090+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:25:43.643090+0000","last_undegraded":"2023-10-18T12:25:43.643090+0000","last_fullsized":"2023-10-18T12:25:43.643090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211886+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:25:14.211886+0000","last_peered":"2023-10-18T12:25:14.211886+0000","last_clean":"2023-10-18T12:25:14.211886+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:25:14.211886+0000","last_undegraded":"2023-10-18T12:25:14.211886+0000","last_fullsized":"2023-10-18T12:25:14.211886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.224004+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:25:15.224004+0000","last_peered":"2023-10-18T12:25:15.224004+0000","last_clean":"2023-10-18T12:25:15.224004+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:25:15.224004+0000","last_undegraded":"2023-10-18T12:25:15.224004+0000","last_fullsized":"2023-10-18T12:25:15.224004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644328+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:25:43.644328+0000","last_peered":"2023-10-18T12:25:43.644328+0000","last_clean":"2023-10-18T12:25:43.644328+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:25:43.644328+0000","last_undegraded":"2023-10-18T12:25:43.644328+0000","last_fullsized":"2023-10-18T12:25:43.644328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215695+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:25:14.215695+0000","last_peered":"2023-10-18T12:25:14.215695+0000","last_clean":"2023-10-18T12:25:14.215695+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:25:14.215695+0000","last_undegraded":"2023-10-18T12:25:14.215695+0000","last_fullsized":"2023-10-18T12:25:14.215695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213449+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:26:15.213449+0000","last_peered":"2023-10-18T12:26:15.213449+0000","last_clean":"2023-10-18T12:26:15.213449+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:26:15.213449+0000","last_undegraded":"2023-10-18T12:26:15.213449+0000","last_fullsized":"2023-10-18T12:26:15.213449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208378+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:25:14.208378+0000","last_peered":"2023-10-18T12:25:14.208378+0000","last_clean":"2023-10-18T12:25:14.208378+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:25:14.208378+0000","last_undegraded":"2023-10-18T12:25:14.208378+0000","last_fullsized":"2023-10-18T12:25:14.208378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213184+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:25:15.213184+0000","last_peered":"2023-10-18T12:25:15.213184+0000","last_clean":"2023-10-18T12:25:15.213184+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:25:15.213184+0000","last_undegraded":"2023-10-18T12:25:15.213184+0000","last_fullsized":"2023-10-18T12:25:15.213184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649880+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:25:43.649880+0000","last_peered":"2023-10-18T12:25:43.649880+0000","last_clean":"2023-10-18T12:25:43.649880+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:25:43.649880+0000","last_undegraded":"2023-10-18T12:25:43.649880+0000","last_fullsized":"2023-10-18T12:25:43.649880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209020+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:25:14.209020+0000","last_peered":"2023-10-18T12:25:14.209020+0000","last_clean":"2023-10-18T12:25:14.209020+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:25:14.209020+0000","last_undegraded":"2023-10-18T12:25:14.209020+0000","last_fullsized":"2023-10-18T12:25:14.209020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644627+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:25:43.644627+0000","last_peered":"2023-10-18T12:25:43.644627+0000","last_clean":"2023-10-18T12:25:43.644627+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:25:43.644627+0000","last_undegraded":"2023-10-18T12:25:43.644627+0000","last_fullsized":"2023-10-18T12:25:43.644627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210906+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:25:15.210906+0000","last_peered":"2023-10-18T12:25:15.210906+0000","last_clean":"2023-10-18T12:25:15.210906+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:25:15.210906+0000","last_undegraded":"2023-10-18T12:25:15.210906+0000","last_fullsized":"2023-10-18T12:25:15.210906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210590+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:25:14.210590+0000","last_peered":"2023-10-18T12:25:14.210590+0000","last_clean":"2023-10-18T12:25:14.210590+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:25:14.210590+0000","last_undegraded":"2023-10-18T12:25:14.210590+0000","last_fullsized":"2023-10-18T12:25:14.210590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-10-18T12:24:09.961298+0000","last_change":"2023-10-18T12:26:15.661239+0000","last_active":"2023-10-18T12:24:08.948983+0000","last_peered":"2023-10-18T12:24:08.948983+0000","last_clean":"2023-10-18T12:24:08.948983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:24:09.961298+0000","last_undegraded":"2023-10-18T12:24:09.961298+0000","last_fullsized":"2023-10-18T12:24:09.961298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211540+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:25:14.211540+0000","last_peered":"2023-10-18T12:25:14.211540+0000","last_clean":"2023-10-18T12:25:14.211540+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:25:14.211540+0000","last_undegraded":"2023-10-18T12:25:14.211540+0000","last_fullsized":"2023-10-18T12:25:14.211540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.212426+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:15.212426+0000","last_peered":"2023-10-18T12:25:15.212426+0000","last_clean":"2023-10-18T12:25:15.212426+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:15.212426+0000","last_undegraded":"2023-10-18T12:25:15.212426+0000","last_fullsized":"2023-10-18T12:25:15.212426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223137+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:25:15.223137+0000","last_peered":"2023-10-18T12:25:15.223137+0000","last_clean":"2023-10-18T12:25:15.223137+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:25:15.223137+0000","last_undegraded":"2023-10-18T12:25:15.223137+0000","last_fullsized":"2023-10-18T12:25:15.223137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649534+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:25:43.649534+0000","last_peered":"2023-10-18T12:25:43.649534+0000","last_clean":"2023-10-18T12:25:43.649534+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:25:43.649534+0000","last_undegraded":"2023-10-18T12:25:43.649534+0000","last_fullsized":"2023-10-18T12:25:43.649534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217523+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:25:15.217523+0000","last_peered":"2023-10-18T12:25:15.217523+0000","last_clean":"2023-10-18T12:25:15.217523+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:25:15.217523+0000","last_undegraded":"2023-10-18T12:25:15.217523+0000","last_fullsized":"2023-10-18T12:25:15.217523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214480+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:25:14.214480+0000","last_peered":"2023-10-18T12:25:14.214480+0000","last_clean":"2023-10-18T12:25:14.214480+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:25:14.214480+0000","last_undegraded":"2023-10-18T12:25:14.214480+0000","last_fullsized":"2023-10-18T12:25:14.214480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213970+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:25:15.213970+0000","last_peered":"2023-10-18T12:25:15.213970+0000","last_clean":"2023-10-18T12:25:15.213970+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:25:15.213970+0000","last_undegraded":"2023-10-18T12:25:15.213970+0000","last_fullsized":"2023-10-18T12:25:15.213970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649689+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:25:43.649689+0000","last_peered":"2023-10-18T12:25:43.649689+0000","last_clean":"2023-10-18T12:25:43.649689+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:25:43.649689+0000","last_undegraded":"2023-10-18T12:25:43.649689+0000","last_fullsized":"2023-10-18T12:25:43.649689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.214742+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:25:15.214742+0000","last_peered":"2023-10-18T12:25:15.214742+0000","last_clean":"2023-10-18T12:25:15.214742+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:25:15.214742+0000","last_undegraded":"2023-10-18T12:25:15.214742+0000","last_fullsized":"2023-10-18T12:25:15.214742+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:13.149946+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:26:13.149946+0000","last_peered":"2023-10-18T12:26:13.149946+0000","last_clean":"2023-10-18T12:26:13.149946+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:26:13.149946+0000","last_undegraded":"2023-10-18T12:26:13.149946+0000","last_fullsized":"2023-10-18T12:26:13.149946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643505+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:25:43.643505+0000","last_peered":"2023-10-18T12:25:43.643505+0000","last_clean":"2023-10-18T12:25:43.643505+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:25:43.643505+0000","last_undegraded":"2023-10-18T12:25:43.643505+0000","last_fullsized":"2023-10-18T12:25:43.643505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212653+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:25:14.212653+0000","last_peered":"2023-10-18T12:25:14.212653+0000","last_clean":"2023-10-18T12:25:14.212653+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:25:14.212653+0000","last_undegraded":"2023-10-18T12:25:14.212653+0000","last_fullsized":"2023-10-18T12:25:14.212653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.213075+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:14.213075+0000","last_peered":"2023-10-18T12:25:14.213075+0000","last_clean":"2023-10-18T12:25:14.213075+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:14.213075+0000","last_undegraded":"2023-10-18T12:25:14.213075+0000","last_fullsized":"2023-10-18T12:25:14.213075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649714+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:25:43.649714+0000","last_peered":"2023-10-18T12:25:43.649714+0000","last_clean":"2023-10-18T12:25:43.649714+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:25:43.649714+0000","last_undegraded":"2023-10-18T12:25:43.649714+0000","last_fullsized":"2023-10-18T12:25:43.649714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223250+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223250+0000","last_peered":"2023-10-18T12:25:15.223250+0000","last_clean":"2023-10-18T12:25:15.223250+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223250+0000","last_undegraded":"2023-10-18T12:25:15.223250+0000","last_fullsized":"2023-10-18T12:25:15.223250+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214524+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:25:14.214524+0000","last_peered":"2023-10-18T12:25:14.214524+0000","last_clean":"2023-10-18T12:25:14.214524+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:25:14.214524+0000","last_undegraded":"2023-10-18T12:25:14.214524+0000","last_fullsized":"2023-10-18T12:25:14.214524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.604171+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:25:43.604171+0000","last_peered":"2023-10-18T12:25:43.604171+0000","last_clean":"2023-10-18T12:25:43.604171+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:25:43.604171+0000","last_undegraded":"2023-10-18T12:25:43.604171+0000","last_fullsized":"2023-10-18T12:25:43.604171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.220890+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:25:15.220890+0000","last_peered":"2023-10-18T12:25:15.220890+0000","last_clean":"2023-10-18T12:25:15.220890+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:25:15.220890+0000","last_undegraded":"2023-10-18T12:25:15.220890+0000","last_fullsized":"2023-10-18T12:25:15.220890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210834+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:25:14.210834+0000","last_peered":"2023-10-18T12:25:14.210834+0000","last_clean":"2023-10-18T12:25:14.210834+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:25:14.210834+0000","last_undegraded":"2023-10-18T12:25:14.210834+0000","last_fullsized":"2023-10-18T12:25:14.210834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.651462+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:25:43.651462+0000","last_peered":"2023-10-18T12:25:43.651462+0000","last_clean":"2023-10-18T12:25:43.651462+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:25:43.651462+0000","last_undegraded":"2023-10-18T12:25:43.651462+0000","last_fullsized":"2023-10-18T12:25:43.651462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211350+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:14.211350+0000","last_peered":"2023-10-18T12:25:14.211350+0000","last_clean":"2023-10-18T12:25:14.211350+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:14.211350+0000","last_undegraded":"2023-10-18T12:25:14.211350+0000","last_fullsized":"2023-10-18T12:25:14.211350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210884+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:25:14.210884+0000","last_peered":"2023-10-18T12:25:14.210884+0000","last_clean":"2023-10-18T12:25:14.210884+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:25:14.210884+0000","last_undegraded":"2023-10-18T12:25:14.210884+0000","last_fullsized":"2023-10-18T12:25:14.210884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648198+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:25:43.648198+0000","last_peered":"2023-10-18T12:25:43.648198+0000","last_clean":"2023-10-18T12:25:43.648198+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:25:43.648198+0000","last_undegraded":"2023-10-18T12:25:43.648198+0000","last_fullsized":"2023-10-18T12:25:43.648198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212507+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:14.212507+0000","last_peered":"2023-10-18T12:25:14.212507+0000","last_clean":"2023-10-18T12:25:14.212507+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:14.212507+0000","last_undegraded":"2023-10-18T12:25:14.212507+0000","last_fullsized":"2023-10-18T12:25:14.212507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216327+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:25:14.216327+0000","last_peered":"2023-10-18T12:25:14.216327+0000","last_clean":"2023-10-18T12:25:14.216327+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:25:14.216327+0000","last_undegraded":"2023-10-18T12:25:14.216327+0000","last_fullsized":"2023-10-18T12:25:14.216327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652574+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:25:43.652574+0000","last_peered":"2023-10-18T12:25:43.652574+0000","last_clean":"2023-10-18T12:25:43.652574+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:25:43.652574+0000","last_undegraded":"2023-10-18T12:25:43.652574+0000","last_fullsized":"2023-10-18T12:25:43.652574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214357+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:25:14.214357+0000","last_peered":"2023-10-18T12:25:14.214357+0000","last_clean":"2023-10-18T12:25:14.214357+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:25:14.214357+0000","last_undegraded":"2023-10-18T12:25:14.214357+0000","last_fullsized":"2023-10-18T12:25:14.214357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211315+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:25:14.211315+0000","last_peered":"2023-10-18T12:25:14.211315+0000","last_clean":"2023-10-18T12:25:14.211315+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:25:14.211315+0000","last_undegraded":"2023-10-18T12:25:14.211315+0000","last_fullsized":"2023-10-18T12:25:14.211315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644409+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:43.644409+0000","last_peered":"2023-10-18T12:25:43.644409+0000","last_clean":"2023-10-18T12:25:43.644409+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:43.644409+0000","last_undegraded":"2023-10-18T12:25:43.644409+0000","last_fullsized":"2023-10-18T12:25:43.644409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218308+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:25:14.218308+0000","last_peered":"2023-10-18T12:25:14.218308+0000","last_clean":"2023-10-18T12:25:14.218308+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:25:14.218308+0000","last_undegraded":"2023-10-18T12:25:14.218308+0000","last_fullsized":"2023-10-18T12:25:14.218308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211578+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:25:14.211578+0000","last_peered":"2023-10-18T12:25:14.211578+0000","last_clean":"2023-10-18T12:25:14.211578+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:25:14.211578+0000","last_undegraded":"2023-10-18T12:25:14.211578+0000","last_fullsized":"2023-10-18T12:25:14.211578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639071+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:43.639071+0000","last_peered":"2023-10-18T12:25:43.639071+0000","last_clean":"2023-10-18T12:25:43.639071+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:43.639071+0000","last_undegraded":"2023-10-18T12:25:43.639071+0000","last_fullsized":"2023-10-18T12:25:43.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210493+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:25:14.210493+0000","last_peered":"2023-10-18T12:25:14.210493+0000","last_clean":"2023-10-18T12:25:14.210493+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:25:14.210493+0000","last_undegraded":"2023-10-18T12:25:14.210493+0000","last_fullsized":"2023-10-18T12:25:14.210493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225441+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:25:15.225441+0000","last_peered":"2023-10-18T12:25:15.225441+0000","last_clean":"2023-10-18T12:25:15.225441+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:25:15.225441+0000","last_undegraded":"2023-10-18T12:25:15.225441+0000","last_fullsized":"2023-10-18T12:25:15.225441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639284+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:43.639284+0000","last_peered":"2023-10-18T12:25:43.639284+0000","last_clean":"2023-10-18T12:25:43.639284+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:43.639284+0000","last_undegraded":"2023-10-18T12:25:43.639284+0000","last_fullsized":"2023-10-18T12:25:43.639284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.638348+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:25:43.638348+0000","last_peered":"2023-10-18T12:25:43.638348+0000","last_clean":"2023-10-18T12:25:43.638348+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:25:43.638348+0000","last_undegraded":"2023-10-18T12:25:43.638348+0000","last_fullsized":"2023-10-18T12:25:43.638348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643095+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:25:43.643095+0000","last_peered":"2023-10-18T12:25:43.643095+0000","last_clean":"2023-10-18T12:25:43.643095+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:25:43.643095+0000","last_undegraded":"2023-10-18T12:25:43.643095+0000","last_fullsized":"2023-10-18T12:25:43.643095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646655+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:25:43.646655+0000","last_peered":"2023-10-18T12:25:43.646655+0000","last_clean":"2023-10-18T12:25:43.646655+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:25:43.646655+0000","last_undegraded":"2023-10-18T12:25:43.646655+0000","last_fullsized":"2023-10-18T12:25:43.646655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656634+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:25:43.656634+0000","last_peered":"2023-10-18T12:25:43.656634+0000","last_clean":"2023-10-18T12:25:43.656634+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:25:43.656634+0000","last_undegraded":"2023-10-18T12:25:43.656634+0000","last_fullsized":"2023-10-18T12:25:43.656634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.655509+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:25:43.655509+0000","last_peered":"2023-10-18T12:25:43.655509+0000","last_clean":"2023-10-18T12:25:43.655509+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:25:43.655509+0000","last_undegraded":"2023-10-18T12:25:43.655509+0000","last_fullsized":"2023-10-18T12:25:43.655509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639525+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:25:43.639525+0000","last_peered":"2023-10-18T12:25:43.639525+0000","last_clean":"2023-10-18T12:25:43.639525+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:25:43.639525+0000","last_undegraded":"2023-10-18T12:25:43.639525+0000","last_fullsized":"2023-10-18T12:25:43.639525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644102+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:25:43.644102+0000","last_peered":"2023-10-18T12:25:43.644102+0000","last_clean":"2023-10-18T12:25:43.644102+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:25:43.644102+0000","last_undegraded":"2023-10-18T12:25:43.644102+0000","last_fullsized":"2023-10-18T12:25:43.644102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644112+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:25:43.644112+0000","last_peered":"2023-10-18T12:25:43.644112+0000","last_clean":"2023-10-18T12:25:43.644112+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:25:43.644112+0000","last_undegraded":"2023-10-18T12:25:43.644112+0000","last_fullsized":"2023-10-18T12:25:43.644112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636676+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:25:43.636676+0000","last_peered":"2023-10-18T12:25:43.636676+0000","last_clean":"2023-10-18T12:25:43.636676+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:25:43.636676+0000","last_undegraded":"2023-10-18T12:25:43.636676+0000","last_fullsized":"2023-10-18T12:25:43.636676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636709+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:25:43.636709+0000","last_peered":"2023-10-18T12:25:43.636709+0000","last_clean":"2023-10-18T12:25:43.636709+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:25:43.636709+0000","last_undegraded":"2023-10-18T12:25:43.636709+0000","last_fullsized":"2023-10-18T12:25:43.636709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643394+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:25:43.643394+0000","last_peered":"2023-10-18T12:25:43.643394+0000","last_clean":"2023-10-18T12:25:43.643394+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:25:43.643394+0000","last_undegraded":"2023-10-18T12:25:43.643394+0000","last_fullsized":"2023-10-18T12:25:43.643394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.657911+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:25:43.657911+0000","last_peered":"2023-10-18T12:25:43.657911+0000","last_clean":"2023-10-18T12:25:43.657911+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:25:43.657911+0000","last_undegraded":"2023-10-18T12:25:43.657911+0000","last_fullsized":"2023-10-18T12:25:43.657911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643480+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:25:43.643480+0000","last_peered":"2023-10-18T12:25:43.643480+0000","last_clean":"2023-10-18T12:25:43.643480+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:25:43.643480+0000","last_undegraded":"2023-10-18T12:25:43.643480+0000","last_fullsized":"2023-10-18T12:25:43.643480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62248,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":211,"num_read_kb":168,"num_write":81,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":197502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":1.147,"15min":1.147},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.745}]},{"osd":2,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88400000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":1.238,"15min":1.238},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75600000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":1.661,"15min":1.661},"last":0.84599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90400000000000003,"5min":1.361,"15min":1.361},"last":0.872},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":1,"15min":1},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.91100000000000003,"5min":1.726,"15min":1.726},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.012,"5min":1.401,"15min":1.401},"last":0.95899999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89000000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.92400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1161040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.54500000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.435,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.499},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63300000000000001,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.999,"15min":0.999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72499999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.432}]},{"osd":5,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84599999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.876,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.77600000000000002,"5min":1.099,"15min":1.099},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82799999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90400000000000003,"5min":1.071,"15min":1.071},"last":0.62},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.044,"15min":1.044},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63700000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.67500000000000004}]}]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":527643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0089999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.92700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.753},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.86199999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1739999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1519999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.91500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.871},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.94599999999999995,"15min":0.95199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":6.2220000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.69699999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.89100000000000001},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2430000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78000000000000003}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.432},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.501}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0149999999999999,"5min":1.044,"15min":1.044},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.72399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0640000000000001,"5min":1.26,"15min":1.26},"last":0.754}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.68300000000000005},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76800000000000002,"15min":0.72799999999999998},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.107,"15min":1.107},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66700000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1108170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.433,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.501},{"interface":"front","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60099999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35799999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.999,"15min":0.999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.137,"15min":1.137},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81299999999999994,"5min":1.071,"15min":1.071},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.66500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.98199999999999998,"5min":1.02,"15min":1.02},"last":0.72799999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95399999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69199999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70699999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":509028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.53000000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.622},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.753}]},{"osd":3,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.68799999999999994,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.9199999999999999,"5min":6.9199999999999999,"15min":6.9199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.026,"5min":0.81899999999999995,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":7.2270000000000003,"5min":7.2270000000000003,"15min":7.2270000000000003},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.016,"5min":1.234,"15min":1.234},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74299999999999999,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.28,"5min":1.849,"15min":1.849},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84999999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.88,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":7.7400000000000002,"5min":7.7400000000000002,"15min":7.7400000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.69699999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.90800000000000003,"15min":0.88900000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.4969999999999999,"5min":7.4969999999999999,"15min":7.4969999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95499999999999996,"15min":0.90300000000000002},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3240000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":570478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.161,"15min":1.161},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94699999999999995,"5min":1.454,"15min":1.454},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.57299999999999995},"min":{"1min":0.52000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88500000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71899999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.64800000000000002,"15min":0.59099999999999997},"min":{"1min":0.63200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":1.298,"15min":1.298},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.52700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89400000000000002,"5min":1.016,"15min":1.016},"last":0.88200000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.002}]}]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":509118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.65200000000000002,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.30299999999999999},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65600000000000003,"15min":0.55200000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.32400000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.60699999999999998},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.84099999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1479999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.024,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.65800000000000003,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.69699999999999995,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.498}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.66200000000000003},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.65700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.755}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69899999999999995,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.751,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":0.92100000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86799999999999999,"15min":0.84599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.61199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.89300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.876},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.89300000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.03,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93899999999999995,"15min":0.92400000000000004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":0.90000000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":575618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.26100000000000001},"max":{"1min":0.92100000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.54600000000000004},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.626},{"interface":"front","average":{"1min":0.745,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.86899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.34000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73799999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.42299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.58399999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.753},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85399999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76500000000000001,"15min":0.74099999999999999},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.254,"15min":1.254},"last":0.995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.74299999999999999,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91600000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":509132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.51400000000000001,"15min":0.436},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7880000000000003,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.52700000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.73599999999999999,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.622},{"interface":"front","average":{"1min":1.026,"5min":0.68999999999999995,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":5.0579999999999998,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.73699999999999999,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.085,"5min":0.77800000000000002,"15min":0.70599999999999996},"min":{"1min":0.60999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.78400000000000003,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.60099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.752,"15min":0.69499999999999995},"min":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.83299999999999996,"15min":0.753},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.984,"5min":4.984,"15min":4.984},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3789999999999996,"5min":5.3789999999999996,"15min":5.3789999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.86199999999999999},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93799999999999994,"15min":0.90500000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.83099999999999996}]},{"osd":11,"last update":"Wed Oct 18 12:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.016,"15min":1.052},"min":{"1min":0.61899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2030000000000001,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.111,"15min":1.155},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2170000000000001,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54800000000000004,"5min":0.621,"15min":0.621},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57099999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.876,"5min":1.04,"15min":1.04},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.52200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":1.22,"15min":1.22},"last":0.75}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61099999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80200000000000005,"15min":0.81100000000000005},"min":{"1min":0.65900000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.02,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.55700000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65000000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.501}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.88}]}]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1103343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":0.999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88100000000000001}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94099999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.752},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.01},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":4.548,"15min":4.548},"last":0.99399999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.745,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73999999999999999,"15min":0.70799999999999996},"min":{"1min":0.60699999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:26:17.335 INFO:tasks.fwd_scrub.ceph_manager:PG 3.2 is not active+clean 2023-10-18T12:26:17.335 INFO:tasks.fwd_scrub.ceph_manager:{'pgid': '3.2', 'version': "91'8", 'reported_seq': 2, 'reported_epoch': 92, 'state': 'peering', 'last_fresh': '2023-10-18T12:24:09.961298+0000', 'last_change': '2023-10-18T12:26:15.661239+0000', 'last_active': '2023-10-18T12:24:08.948983+0000', 'last_peered': '2023-10-18T12:24:08.948983+0000', 'last_clean': '2023-10-18T12:24:08.948983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:24:09.961298+0000', 'last_undegraded': '2023-10-18T12:24:09.961298+0000', 'last_fullsized': '2023-10-18T12:24:09.961298+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 84, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:24:08.948983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:24:08.948983+0000', 'last_clean_scrub_stamp': '2023-10-18T12:24:08.948983+0000', 'objects_scrubbed': 0, 'log_size': 8, 'log_dups_size': 0, 'ondisk_log_size': 8, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 5, 1], 'acting': [9, 5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-10-18T12:26:17.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T12:26:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:17 smithi186 ceph-mon[135253]: pgmap v321: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-18T12:26:17.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:17 smithi186 ceph-mon[135253]: osdmap e93: 12 total, 12 up, 12 in 2023-10-18T12:26:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 -- 172.21.15.70:0/3213154244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1304101060 msgr2=0x7f130410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 --2- 172.21.15.70:0/3213154244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1304101060 0x7f130410d570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00b0a0 tx=0x7f12fc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 -- 172.21.15.70:0/3213154244 shutdown_connections 2023-10-18T12:26:17.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 --2- 172.21.15.70:0/3213154244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1304101060 0x7f130410d570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 --2- 172.21.15.70:0/3213154244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 0x7f1304106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 --2- 172.21.15.70:0/3213154244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13041006a0 0x7f1304100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 -- 172.21.15.70:0/3213154244 >> 172.21.15.70:0/3213154244 conn(0x7f13040fc170 msgr2=0x7f13040fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:17.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 -- 172.21.15.70:0/3213154244 shutdown_connections 2023-10-18T12:26:17.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.560+0000 7f13095ff700 1 -- 172.21.15.70:0/3213154244 wait complete. 2023-10-18T12:26:17.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 Processor -- start 2023-10-18T12:26:17.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 -- start start 2023-10-18T12:26:17.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 0x7f1304072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1304101060 0x7f1304073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 0x7f1304073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041a4eb0 con 0x7f13041006a0 2023-10-18T12:26:17.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041a5020 con 0x7f13041066b0 2023-10-18T12:26:17.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.561+0000 7f13095ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041a51e0 con 0x7f1304101060 2023-10-18T12:26:17.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 0x7f1304072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 0x7f1304072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40134/0 (socket says 172.21.15.70:40134) 2023-10-18T12:26:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 -- 172.21.15.70:0/1628788372 learned_addr learned my addr 172.21.15.70:0/1628788372 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f13037fe700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 0x7f1304073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f12fbfff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1304101060 0x7f1304073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 -- 172.21.15.70:0/1628788372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1304101060 msgr2=0x7f1304073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1304101060 0x7f1304073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 -- 172.21.15.70:0/1628788372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 msgr2=0x7f1304073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 0x7f1304073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 -- 172.21.15.70:0/1628788372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc009d00 con 0x7f13041006a0 2023-10-18T12:26:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f13037fe700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 0x7f1304073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1302ffd700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 0x7f1304072af0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e9a0 tx=0x7f12f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4009790 con 0x7f13041006a0 2023-10-18T12:26:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f400cbd0 con 0x7f13041006a0 2023-10-18T12:26:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.562+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13041a54c0 con 0x7f13041006a0 2023-10-18T12:26:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.563+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4018680 con 0x7f13041006a0 2023-10-18T12:26:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.563+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13041a59d0 con 0x7f13041006a0 2023-10-18T12:26:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.564+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304108ca0 con 0x7f13041006a0 2023-10-18T12:26:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.565+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f12f400cd40 con 0x7f13041006a0 2023-10-18T12:26:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.567+0000 7f1300ff9700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12f00a14d0 0x7f12f00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.567+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f12f4014070 con 0x7f13041006a0 2023-10-18T12:26:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.567+0000 7f12fbfff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12f00a14d0 0x7f12f00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.568+0000 7f12fbfff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12f00a14d0 0x7f12f00a3990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1304074220 tx=0x7f12ec009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.570+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f12f40966a0 con 0x7f13041006a0 2023-10-18T12:26:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:17 smithi070 ceph-mon[98639]: pgmap v321: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-18T12:26:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:17 smithi070 ceph-mon[98639]: osdmap e93: 12 total, 12 up, 12 in 2023-10-18T12:26:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:17 smithi152 ceph-mon[134767]: pgmap v321: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-18T12:26:17.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:17 smithi152 ceph-mon[134767]: osdmap e93: 12 total, 12 up, 12 in 2023-10-18T12:26:17.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.858+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1304101de0 con 0x7f13041006a0 2023-10-18T12:26:17.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.858+0000 7f1300ff9700 1 -- 172.21.15.70:0/1628788372 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1203 (secure 0 0 0) 0x7f12f4099cc0 con 0x7f13041006a0 2023-10-18T12:26:17.861 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:17.862 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":498,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":93,"num_osds":12,"num_up_osds":12,"osd_up_since":1697631802,"num_in_osds":12,"osd_in_since":1697631784,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":26,"data_bytes":652862,"bytes_used":346296320,"bytes_avail":1151594131456,"bytes_total":1151940427776,"read_bytes_sec":1126,"write_bytes_sec":204,"read_op_per_sec":2,"write_op_per_sec":0},"fsmap":{"epoch":20,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"g","status":"up:standby-replay","gid":24719},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24743}],"up:standby":10},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"services":{"dashboard":"https://172.21.15.70:8443/"}},"servicemap":{"epoch":29,"modified":"2023-10-18T12:25:39.614754+0000","services":{}},"progress_events":{}} 2023-10-18T12:26:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12f00a14d0 msgr2=0x7f12f00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12f00a14d0 0x7f12f00a3990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1304074220 tx=0x7f12ec009260 comp rx=0 tx=0).stop 2023-10-18T12:26:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 msgr2=0x7f1304072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 0x7f1304072af0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12f400e9a0 tx=0x7f12f400ed60 comp rx=0 tx=0).stop 2023-10-18T12:26:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 shutdown_connections 2023-10-18T12:26:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f12f00a14d0 0x7f12f00a3990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13041066b0 0x7f1304073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13041006a0 0x7f1304072af0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 --2- 172.21.15.70:0/1628788372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1304101060 0x7f1304073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 >> 172.21.15.70:0/1628788372 conn(0x7f13040fc170 msgr2=0x7f13040fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 shutdown_connections 2023-10-18T12:26:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:17.861+0000 7f13095ff700 1 -- 172.21.15.70:0/1628788372 wait complete. 2023-10-18T12:26:17.883 INFO:tasks.fwd_scrub.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T12:26:17.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:26:18.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 -- 172.21.15.70:0/939726220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 msgr2=0x7f0ba8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/939726220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8106a90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009990 tx=0x7f0b98009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 -- 172.21.15.70:0/939726220 shutdown_connections 2023-10-18T12:26:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/939726220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba8101060 0x7f0ba810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/939726220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8106a90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/939726220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 0x7f0ba8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 -- 172.21.15.70:0/939726220 >> 172.21.15.70:0/939726220 conn(0x7f0ba80fc1d0 msgr2=0x7f0ba80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 -- 172.21.15.70:0/939726220 shutdown_connections 2023-10-18T12:26:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.083+0000 7f0bae3ff700 1 -- 172.21.15.70:0/939726220 wait complete. 2023-10-18T12:26:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 Processor -- start 2023-10-18T12:26:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 -- start start 2023-10-18T12:26:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 0x7f0ba8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:18.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba8101060 0x7f0ba8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:18.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:18.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba81a4eb0 con 0x7f0ba81066b0 2023-10-18T12:26:18.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba81a5020 con 0x7f0ba81006a0 2023-10-18T12:26:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.084+0000 7f0bae3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba81a51e0 con 0x7f0ba8101060 2023-10-18T12:26:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 0x7f0ba8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 0x7f0ba8072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:42584/0 (socket says 172.21.15.70:42584) 2023-10-18T12:26:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55386/0 (socket says 172.21.15.70:55386) 2023-10-18T12:26:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0ba7fff700 1 -- 172.21.15.70:0/3564277765 learned_addr learned my addr 172.21.15.70:0/3564277765 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0b9ffff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba8101060 0x7f0ba8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 -- 172.21.15.70:0/3564277765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba8101060 msgr2=0x7f0ba8073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba8101060 0x7f0ba8073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 -- 172.21.15.70:0/3564277765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 msgr2=0x7f0ba8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 0x7f0ba8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 -- 172.21.15.70:0/3564277765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980095f0 con 0x7f0ba81066b0 2023-10-18T12:26:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bac99c700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8073570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0007f10 tx=0x7f0ba000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0004db0 con 0x7f0ba81066b0 2023-10-18T12:26:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ba000f040 con 0x7f0ba81066b0 2023-10-18T12:26:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0005990 con 0x7f0ba81066b0 2023-10-18T12:26:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.085+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba81a54c0 con 0x7f0ba81066b0 2023-10-18T12:26:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.086+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba81a5980 con 0x7f0ba81066b0 2023-10-18T12:26:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.087+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba8108ca0 con 0x7f0ba81066b0 2023-10-18T12:26:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.088+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0ba000db20 con 0x7f0ba81066b0 2023-10-18T12:26:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.091+0000 7f0ba5ffb700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b940a1550 0x7f0b940a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.091+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ba00cc910 con 0x7f0ba81066b0 2023-10-18T12:26:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.091+0000 7f0ba7fff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b940a1550 0x7f0b940a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.091+0000 7f0ba7fff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b940a1550 0x7f0b940a3a10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b98006010 tx=0x7f0b9800b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.095+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0ba004f100 con 0x7f0ba81066b0 2023-10-18T12:26:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.298+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ba8110620 con 0x7f0b940a1550 2023-10-18T12:26:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.311+0000 7f0ba5ffb700 1 -- 172.21.15.70:0/3564277765 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427428 (secure 0 0 0) 0x7f0ba8110620 con 0x7f0b940a1550 2023-10-18T12:26:18.319 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b940a1550 msgr2=0x7f0b940a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b940a1550 0x7f0b940a3a10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b98006010 tx=0x7f0b9800b540 comp rx=0 tx=0).stop 2023-10-18T12:26:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 msgr2=0x7f0ba8073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8073570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0007f10 tx=0x7f0ba000d3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 shutdown_connections 2023-10-18T12:26:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ba81066b0 0x7f0ba8073570 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0b940a1550 0x7f0b940a3a10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba81006a0 0x7f0ba8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 --2- 172.21.15.70:0/3564277765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ba8101060 0x7f0ba8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 >> 172.21.15.70:0/3564277765 conn(0x7f0ba80fc1d0 msgr2=0x7f0ba80fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 shutdown_connections 2023-10-18T12:26:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:18.316+0000 7f0bae3ff700 1 -- 172.21.15.70:0/3564277765 wait complete. 2023-10-18T12:26:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:26:18.338 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-10-18T12:26:17.624970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658241,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":358,"num_read_kb":298,"num_write":265,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":711,"ondisk_log_size":711,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338268,"kb_used_data":14060,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603556,"statfs":{"total":1151940427776,"available":1151594041344,"internally_reserved":0,"allocated":14397440,"data_stored":8148486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002073"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650976+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:25:43.650976+0000","last_peered":"2023-10-18T12:25:43.650976+0000","last_clean":"2023-10-18T12:25:43.650976+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:25:43.650976+0000","last_undegraded":"2023-10-18T12:25:43.650976+0000","last_fullsized":"2023-10-18T12:25:43.650976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649821+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:25:43.649821+0000","last_peered":"2023-10-18T12:25:43.649821+0000","last_clean":"2023-10-18T12:25:43.649821+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:25:43.649821+0000","last_undegraded":"2023-10-18T12:25:43.649821+0000","last_fullsized":"2023-10-18T12:25:43.649821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.650939+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:25:43.650939+0000","last_peered":"2023-10-18T12:25:43.650939+0000","last_clean":"2023-10-18T12:25:43.650939+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:25:43.650939+0000","last_undegraded":"2023-10-18T12:25:43.650939+0000","last_fullsized":"2023-10-18T12:25:43.650939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:11.635030+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:26:11.635030+0000","last_peered":"2023-10-18T12:26:11.635030+0000","last_clean":"2023-10-18T12:26:11.635030+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:26:11.635030+0000","last_undegraded":"2023-10-18T12:26:11.635030+0000","last_fullsized":"2023-10-18T12:26:11.635030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.631268+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:25:43.631268+0000","last_peered":"2023-10-18T12:25:43.631268+0000","last_clean":"2023-10-18T12:25:43.631268+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:25:43.631268+0000","last_undegraded":"2023-10-18T12:25:43.631268+0000","last_fullsized":"2023-10-18T12:25:43.631268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210846+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:26:15.210846+0000","last_peered":"2023-10-18T12:26:15.210846+0000","last_clean":"2023-10-18T12:26:15.210846+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:26:15.210846+0000","last_undegraded":"2023-10-18T12:26:15.210846+0000","last_fullsized":"2023-10-18T12:26:15.210846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639863+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:43.639863+0000","last_peered":"2023-10-18T12:25:43.639863+0000","last_clean":"2023-10-18T12:25:43.639863+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:43.639863+0000","last_undegraded":"2023-10-18T12:25:43.639863+0000","last_fullsized":"2023-10-18T12:25:43.639863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639818+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:25:43.639818+0000","last_peered":"2023-10-18T12:25:43.639818+0000","last_clean":"2023-10-18T12:25:43.639818+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:25:43.639818+0000","last_undegraded":"2023-10-18T12:25:43.639818+0000","last_fullsized":"2023-10-18T12:25:43.639818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642671+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:25:43.642671+0000","last_peered":"2023-10-18T12:25:43.642671+0000","last_clean":"2023-10-18T12:25:43.642671+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:25:43.642671+0000","last_undegraded":"2023-10-18T12:25:43.642671+0000","last_fullsized":"2023-10-18T12:25:43.642671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210940+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:26:15.210940+0000","last_peered":"2023-10-18T12:26:15.210940+0000","last_clean":"2023-10-18T12:26:15.210940+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:26:15.210940+0000","last_undegraded":"2023-10-18T12:26:15.210940+0000","last_fullsized":"2023-10-18T12:26:15.210940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644261+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:43.644261+0000","last_peered":"2023-10-18T12:25:43.644261+0000","last_clean":"2023-10-18T12:25:43.644261+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:43.644261+0000","last_undegraded":"2023-10-18T12:25:43.644261+0000","last_fullsized":"2023-10-18T12:25:43.644261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633660+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:43.633660+0000","last_peered":"2023-10-18T12:25:43.633660+0000","last_clean":"2023-10-18T12:25:43.633660+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:43.633660+0000","last_undegraded":"2023-10-18T12:25:43.633660+0000","last_fullsized":"2023-10-18T12:25:43.633660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.658873+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:25:43.658873+0000","last_peered":"2023-10-18T12:25:43.658873+0000","last_clean":"2023-10-18T12:25:43.658873+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:25:43.658873+0000","last_undegraded":"2023-10-18T12:25:43.658873+0000","last_fullsized":"2023-10-18T12:25:43.658873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646839+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:25:43.646839+0000","last_peered":"2023-10-18T12:25:43.646839+0000","last_clean":"2023-10-18T12:25:43.646839+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:25:43.646839+0000","last_undegraded":"2023-10-18T12:25:43.646839+0000","last_fullsized":"2023-10-18T12:25:43.646839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649694+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:25:43.649694+0000","last_peered":"2023-10-18T12:25:43.649694+0000","last_clean":"2023-10-18T12:25:43.649694+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:25:43.649694+0000","last_undegraded":"2023-10-18T12:25:43.649694+0000","last_fullsized":"2023-10-18T12:25:43.649694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648146+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:25:43.648146+0000","last_peered":"2023-10-18T12:25:43.648146+0000","last_clean":"2023-10-18T12:25:43.648146+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:25:43.648146+0000","last_undegraded":"2023-10-18T12:25:43.648146+0000","last_fullsized":"2023-10-18T12:25:43.648146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652031+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:25:43.652031+0000","last_peered":"2023-10-18T12:25:43.652031+0000","last_clean":"2023-10-18T12:25:43.652031+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:25:43.652031+0000","last_undegraded":"2023-10-18T12:25:43.652031+0000","last_fullsized":"2023-10-18T12:25:43.652031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639903+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:25:43.639903+0000","last_peered":"2023-10-18T12:25:43.639903+0000","last_clean":"2023-10-18T12:25:43.639903+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:25:43.639903+0000","last_undegraded":"2023-10-18T12:25:43.639903+0000","last_fullsized":"2023-10-18T12:25:43.639903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643796+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:25:43.643796+0000","last_peered":"2023-10-18T12:25:43.643796+0000","last_clean":"2023-10-18T12:25:43.643796+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:25:43.643796+0000","last_undegraded":"2023-10-18T12:25:43.643796+0000","last_fullsized":"2023-10-18T12:25:43.643796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212928+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:25:14.212928+0000","last_peered":"2023-10-18T12:25:14.212928+0000","last_clean":"2023-10-18T12:25:14.212928+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:25:14.212928+0000","last_undegraded":"2023-10-18T12:25:14.212928+0000","last_fullsized":"2023-10-18T12:25:14.212928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213691+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:26:15.213691+0000","last_peered":"2023-10-18T12:26:15.213691+0000","last_clean":"2023-10-18T12:26:15.213691+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:26:15.213691+0000","last_undegraded":"2023-10-18T12:26:15.213691+0000","last_fullsized":"2023-10-18T12:26:15.213691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639283+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:25:43.639283+0000","last_peered":"2023-10-18T12:25:43.639283+0000","last_clean":"2023-10-18T12:25:43.639283+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:25:43.639283+0000","last_undegraded":"2023-10-18T12:25:43.639283+0000","last_fullsized":"2023-10-18T12:25:43.639283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210351+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:15.210351+0000","last_peered":"2023-10-18T12:25:15.210351+0000","last_clean":"2023-10-18T12:25:15.210351+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:15.210351+0000","last_undegraded":"2023-10-18T12:25:15.210351+0000","last_fullsized":"2023-10-18T12:25:15.210351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.216358+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:25:15.216358+0000","last_peered":"2023-10-18T12:25:15.216358+0000","last_clean":"2023-10-18T12:25:15.216358+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:25:15.216358+0000","last_undegraded":"2023-10-18T12:25:15.216358+0000","last_fullsized":"2023-10-18T12:25:15.216358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656048+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:25:43.656048+0000","last_peered":"2023-10-18T12:25:43.656048+0000","last_clean":"2023-10-18T12:25:43.656048+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:25:43.656048+0000","last_undegraded":"2023-10-18T12:25:43.656048+0000","last_fullsized":"2023-10-18T12:25:43.656048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210533+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:15.210533+0000","last_peered":"2023-10-18T12:25:15.210533+0000","last_clean":"2023-10-18T12:25:15.210533+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:15.210533+0000","last_undegraded":"2023-10-18T12:25:15.210533+0000","last_fullsized":"2023-10-18T12:25:15.210533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215742+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:25:14.215742+0000","last_peered":"2023-10-18T12:25:14.215742+0000","last_clean":"2023-10-18T12:25:14.215742+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:25:14.215742+0000","last_undegraded":"2023-10-18T12:25:14.215742+0000","last_fullsized":"2023-10-18T12:25:14.215742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652536+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:25:43.652536+0000","last_peered":"2023-10-18T12:25:43.652536+0000","last_clean":"2023-10-18T12:25:43.652536+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:25:43.652536+0000","last_undegraded":"2023-10-18T12:25:43.652536+0000","last_fullsized":"2023-10-18T12:25:43.652536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210967+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:15.210967+0000","last_peered":"2023-10-18T12:25:15.210967+0000","last_clean":"2023-10-18T12:25:15.210967+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:15.210967+0000","last_undegraded":"2023-10-18T12:25:15.210967+0000","last_fullsized":"2023-10-18T12:25:15.210967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216726+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:25:14.216726+0000","last_peered":"2023-10-18T12:25:14.216726+0000","last_clean":"2023-10-18T12:25:14.216726+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:25:14.216726+0000","last_undegraded":"2023-10-18T12:25:14.216726+0000","last_fullsized":"2023-10-18T12:25:14.216726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214868+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:26:15.214868+0000","last_peered":"2023-10-18T12:26:15.214868+0000","last_clean":"2023-10-18T12:26:15.214868+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:26:15.214868+0000","last_undegraded":"2023-10-18T12:26:15.214868+0000","last_fullsized":"2023-10-18T12:26:15.214868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217322+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:25:15.217322+0000","last_peered":"2023-10-18T12:25:15.217322+0000","last_clean":"2023-10-18T12:25:15.217322+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:25:15.217322+0000","last_undegraded":"2023-10-18T12:25:15.217322+0000","last_fullsized":"2023-10-18T12:25:15.217322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214751+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:26:15.214751+0000","last_peered":"2023-10-18T12:26:15.214751+0000","last_clean":"2023-10-18T12:26:15.214751+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:26:15.214751+0000","last_undegraded":"2023-10-18T12:26:15.214751+0000","last_fullsized":"2023-10-18T12:26:15.214751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633052+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:25:43.633052+0000","last_peered":"2023-10-18T12:25:43.633052+0000","last_clean":"2023-10-18T12:25:43.633052+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:25:43.633052+0000","last_undegraded":"2023-10-18T12:25:43.633052+0000","last_fullsized":"2023-10-18T12:25:43.633052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211137+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:25:14.211137+0000","last_peered":"2023-10-18T12:25:14.211137+0000","last_clean":"2023-10-18T12:25:14.211137+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:25:14.211137+0000","last_undegraded":"2023-10-18T12:25:14.211137+0000","last_fullsized":"2023-10-18T12:25:14.211137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210513+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:26:15.210513+0000","last_peered":"2023-10-18T12:26:15.210513+0000","last_clean":"2023-10-18T12:26:15.210513+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:26:15.210513+0000","last_undegraded":"2023-10-18T12:26:15.210513+0000","last_fullsized":"2023-10-18T12:26:15.210513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215014+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:26:15.215014+0000","last_peered":"2023-10-18T12:26:15.215014+0000","last_clean":"2023-10-18T12:26:15.215014+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:26:15.215014+0000","last_undegraded":"2023-10-18T12:26:15.215014+0000","last_fullsized":"2023-10-18T12:26:15.215014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215355+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:25:14.215355+0000","last_peered":"2023-10-18T12:25:14.215355+0000","last_clean":"2023-10-18T12:25:14.215355+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:25:14.215355+0000","last_undegraded":"2023-10-18T12:25:14.215355+0000","last_fullsized":"2023-10-18T12:25:14.215355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.207597+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:25:14.207597+0000","last_peered":"2023-10-18T12:25:14.207597+0000","last_clean":"2023-10-18T12:25:14.207597+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:25:14.207597+0000","last_undegraded":"2023-10-18T12:25:14.207597+0000","last_fullsized":"2023-10-18T12:25:14.207597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653839+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:25:43.653839+0000","last_peered":"2023-10-18T12:25:43.653839+0000","last_clean":"2023-10-18T12:25:43.653839+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:25:43.653839+0000","last_undegraded":"2023-10-18T12:25:43.653839+0000","last_fullsized":"2023-10-18T12:25:43.653839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217590+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:25:15.217590+0000","last_peered":"2023-10-18T12:25:15.217590+0000","last_clean":"2023-10-18T12:25:15.217590+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:25:15.217590+0000","last_undegraded":"2023-10-18T12:25:15.217590+0000","last_fullsized":"2023-10-18T12:25:15.217590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214658+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:26:15.214658+0000","last_peered":"2023-10-18T12:26:15.214658+0000","last_clean":"2023-10-18T12:26:15.214658+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:26:15.214658+0000","last_undegraded":"2023-10-18T12:26:15.214658+0000","last_fullsized":"2023-10-18T12:26:15.214658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639762+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:25:43.639762+0000","last_peered":"2023-10-18T12:25:43.639762+0000","last_clean":"2023-10-18T12:25:43.639762+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:25:43.639762+0000","last_undegraded":"2023-10-18T12:25:43.639762+0000","last_fullsized":"2023-10-18T12:25:43.639762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211000+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:25:14.211000+0000","last_peered":"2023-10-18T12:25:14.211000+0000","last_clean":"2023-10-18T12:25:14.211000+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:25:14.211000+0000","last_undegraded":"2023-10-18T12:25:14.211000+0000","last_fullsized":"2023-10-18T12:25:14.211000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210810+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:26:15.210810+0000","last_peered":"2023-10-18T12:26:15.210810+0000","last_clean":"2023-10-18T12:26:15.210810+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:26:15.210810+0000","last_undegraded":"2023-10-18T12:26:15.210810+0000","last_fullsized":"2023-10-18T12:26:15.210810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642014+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:25:43.642014+0000","last_peered":"2023-10-18T12:25:43.642014+0000","last_clean":"2023-10-18T12:25:43.642014+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:25:43.642014+0000","last_undegraded":"2023-10-18T12:25:43.642014+0000","last_fullsized":"2023-10-18T12:25:43.642014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208294+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:25:15.208294+0000","last_peered":"2023-10-18T12:25:15.208294+0000","last_clean":"2023-10-18T12:25:15.208294+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:25:15.208294+0000","last_undegraded":"2023-10-18T12:25:15.208294+0000","last_fullsized":"2023-10-18T12:25:15.208294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211053+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:14.211053+0000","last_peered":"2023-10-18T12:25:14.211053+0000","last_clean":"2023-10-18T12:25:14.211053+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:14.211053+0000","last_undegraded":"2023-10-18T12:25:14.211053+0000","last_fullsized":"2023-10-18T12:25:14.211053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652650+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:25:43.652650+0000","last_peered":"2023-10-18T12:25:43.652650+0000","last_clean":"2023-10-18T12:25:43.652650+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:25:43.652650+0000","last_undegraded":"2023-10-18T12:25:43.652650+0000","last_fullsized":"2023-10-18T12:25:43.652650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210694+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:15.210694+0000","last_peered":"2023-10-18T12:25:15.210694+0000","last_clean":"2023-10-18T12:25:15.210694+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:15.210694+0000","last_undegraded":"2023-10-18T12:25:15.210694+0000","last_fullsized":"2023-10-18T12:25:15.210694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214711+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:26:15.214711+0000","last_peered":"2023-10-18T12:26:15.214711+0000","last_clean":"2023-10-18T12:26:15.214711+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:26:15.214711+0000","last_undegraded":"2023-10-18T12:26:15.214711+0000","last_fullsized":"2023-10-18T12:26:15.214711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.653417+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:25:43.653417+0000","last_peered":"2023-10-18T12:25:43.653417+0000","last_clean":"2023-10-18T12:25:43.653417+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:25:43.653417+0000","last_undegraded":"2023-10-18T12:25:43.653417+0000","last_fullsized":"2023-10-18T12:25:43.653417+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208356+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:25:15.208356+0000","last_peered":"2023-10-18T12:25:15.208356+0000","last_clean":"2023-10-18T12:25:15.208356+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:25:15.208356+0000","last_undegraded":"2023-10-18T12:25:15.208356+0000","last_fullsized":"2023-10-18T12:25:15.208356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":272,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.336063+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:26:16.336063+0000","last_peered":"2023-10-18T12:26:16.336063+0000","last_clean":"2023-10-18T12:26:16.336063+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:26:16.336063+0000","last_undegraded":"2023-10-18T12:26:16.336063+0000","last_fullsized":"2023-10-18T12:26:16.336063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66829,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":217,"num_read_kb":174,"num_write":35,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636586+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:25:43.636586+0000","last_peered":"2023-10-18T12:25:43.636586+0000","last_clean":"2023-10-18T12:25:43.636586+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:25:43.636586+0000","last_undegraded":"2023-10-18T12:25:43.636586+0000","last_fullsized":"2023-10-18T12:25:43.636586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215157+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:26:15.215157+0000","last_peered":"2023-10-18T12:26:15.215157+0000","last_clean":"2023-10-18T12:26:15.215157+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:26:15.215157+0000","last_undegraded":"2023-10-18T12:26:15.215157+0000","last_fullsized":"2023-10-18T12:26:15.215157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211560+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:25:14.211560+0000","last_peered":"2023-10-18T12:25:14.211560+0000","last_clean":"2023-10-18T12:25:14.211560+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:25:14.211560+0000","last_undegraded":"2023-10-18T12:25:14.211560+0000","last_fullsized":"2023-10-18T12:25:14.211560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649605+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:25:43.649605+0000","last_peered":"2023-10-18T12:25:43.649605+0000","last_clean":"2023-10-18T12:25:43.649605+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:25:43.649605+0000","last_undegraded":"2023-10-18T12:25:43.649605+0000","last_fullsized":"2023-10-18T12:25:43.649605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214581+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:25:14.214581+0000","last_peered":"2023-10-18T12:25:14.214581+0000","last_clean":"2023-10-18T12:25:14.214581+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:25:14.214581+0000","last_undegraded":"2023-10-18T12:25:14.214581+0000","last_fullsized":"2023-10-18T12:25:14.214581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215428+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:26:15.215428+0000","last_peered":"2023-10-18T12:26:15.215428+0000","last_clean":"2023-10-18T12:26:15.215428+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:26:15.215428+0000","last_undegraded":"2023-10-18T12:26:15.215428+0000","last_fullsized":"2023-10-18T12:26:15.215428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.654349+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:43.654349+0000","last_peered":"2023-10-18T12:25:43.654349+0000","last_clean":"2023-10-18T12:25:43.654349+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:43.654349+0000","last_undegraded":"2023-10-18T12:25:43.654349+0000","last_fullsized":"2023-10-18T12:25:43.654349+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.208931+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:25:15.208931+0000","last_peered":"2023-10-18T12:25:15.208931+0000","last_clean":"2023-10-18T12:25:15.208931+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:25:15.208931+0000","last_undegraded":"2023-10-18T12:25:15.208931+0000","last_fullsized":"2023-10-18T12:25:15.208931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210457+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:15.210457+0000","last_peered":"2023-10-18T12:25:15.210457+0000","last_clean":"2023-10-18T12:25:15.210457+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:15.210457+0000","last_undegraded":"2023-10-18T12:25:15.210457+0000","last_fullsized":"2023-10-18T12:25:15.210457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642268+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:43.642268+0000","last_peered":"2023-10-18T12:25:43.642268+0000","last_clean":"2023-10-18T12:25:43.642268+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:43.642268+0000","last_undegraded":"2023-10-18T12:25:43.642268+0000","last_fullsized":"2023-10-18T12:25:43.642268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208699+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:25:14.208699+0000","last_peered":"2023-10-18T12:25:14.208699+0000","last_clean":"2023-10-18T12:25:14.208699+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:25:14.208699+0000","last_undegraded":"2023-10-18T12:25:14.208699+0000","last_fullsized":"2023-10-18T12:25:14.208699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225793+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:25:15.225793+0000","last_peered":"2023-10-18T12:25:15.225793+0000","last_clean":"2023-10-18T12:25:15.225793+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:25:15.225793+0000","last_undegraded":"2023-10-18T12:25:15.225793+0000","last_fullsized":"2023-10-18T12:25:15.225793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214502+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:26:15.214502+0000","last_peered":"2023-10-18T12:26:15.214502+0000","last_clean":"2023-10-18T12:26:15.214502+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:26:15.214502+0000","last_undegraded":"2023-10-18T12:26:15.214502+0000","last_fullsized":"2023-10-18T12:26:15.214502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214483+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:26:15.214483+0000","last_peered":"2023-10-18T12:26:15.214483+0000","last_clean":"2023-10-18T12:26:15.214483+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:26:15.214483+0000","last_undegraded":"2023-10-18T12:26:15.214483+0000","last_fullsized":"2023-10-18T12:26:15.214483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.224004+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:25:15.224004+0000","last_peered":"2023-10-18T12:25:15.224004+0000","last_clean":"2023-10-18T12:25:15.224004+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:25:15.224004+0000","last_undegraded":"2023-10-18T12:25:15.224004+0000","last_fullsized":"2023-10-18T12:25:15.224004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644328+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:25:43.644328+0000","last_peered":"2023-10-18T12:25:43.644328+0000","last_clean":"2023-10-18T12:25:43.644328+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:25:43.644328+0000","last_undegraded":"2023-10-18T12:25:43.644328+0000","last_fullsized":"2023-10-18T12:25:43.644328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.215695+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:25:14.215695+0000","last_peered":"2023-10-18T12:25:14.215695+0000","last_clean":"2023-10-18T12:25:14.215695+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:25:14.215695+0000","last_undegraded":"2023-10-18T12:25:14.215695+0000","last_fullsized":"2023-10-18T12:25:14.215695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213449+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:26:15.213449+0000","last_peered":"2023-10-18T12:26:15.213449+0000","last_clean":"2023-10-18T12:26:15.213449+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:26:15.213449+0000","last_undegraded":"2023-10-18T12:26:15.213449+0000","last_fullsized":"2023-10-18T12:26:15.213449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.336032+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:26:16.336032+0000","last_peered":"2023-10-18T12:26:16.336032+0000","last_clean":"2023-10-18T12:26:16.336032+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:26:16.336032+0000","last_undegraded":"2023-10-18T12:26:16.336032+0000","last_fullsized":"2023-10-18T12:26:16.336032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.208378+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:25:14.208378+0000","last_peered":"2023-10-18T12:25:14.208378+0000","last_clean":"2023-10-18T12:25:14.208378+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:25:14.208378+0000","last_undegraded":"2023-10-18T12:25:14.208378+0000","last_fullsized":"2023-10-18T12:25:14.208378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213184+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:25:15.213184+0000","last_peered":"2023-10-18T12:25:15.213184+0000","last_clean":"2023-10-18T12:25:15.213184+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:25:15.213184+0000","last_undegraded":"2023-10-18T12:25:15.213184+0000","last_fullsized":"2023-10-18T12:25:15.213184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.335890+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:26:16.335890+0000","last_peered":"2023-10-18T12:26:16.335890+0000","last_clean":"2023-10-18T12:26:16.335890+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:26:16.335890+0000","last_undegraded":"2023-10-18T12:26:16.335890+0000","last_fullsized":"2023-10-18T12:26:16.335890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.209020+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:25:14.209020+0000","last_peered":"2023-10-18T12:25:14.209020+0000","last_clean":"2023-10-18T12:25:14.209020+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:25:14.209020+0000","last_undegraded":"2023-10-18T12:25:14.209020+0000","last_fullsized":"2023-10-18T12:25:14.209020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214953+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:26:15.214953+0000","last_peered":"2023-10-18T12:26:15.214953+0000","last_clean":"2023-10-18T12:26:15.214953+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:26:15.214953+0000","last_undegraded":"2023-10-18T12:26:15.214953+0000","last_fullsized":"2023-10-18T12:26:15.214953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210906+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:25:15.210906+0000","last_peered":"2023-10-18T12:25:15.210906+0000","last_clean":"2023-10-18T12:25:15.210906+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:25:15.210906+0000","last_undegraded":"2023-10-18T12:25:15.210906+0000","last_fullsized":"2023-10-18T12:25:15.210906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210590+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:25:14.210590+0000","last_peered":"2023-10-18T12:25:14.210590+0000","last_clean":"2023-10-18T12:25:14.210590+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:25:14.210590+0000","last_undegraded":"2023-10-18T12:25:14.210590+0000","last_fullsized":"2023-10-18T12:25:14.210590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-10-18T12:24:09.961298+0000","last_change":"2023-10-18T12:26:15.661239+0000","last_active":"2023-10-18T12:24:08.948983+0000","last_peered":"2023-10-18T12:24:08.948983+0000","last_clean":"2023-10-18T12:24:08.948983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:24:09.961298+0000","last_undegraded":"2023-10-18T12:24:09.961298+0000","last_fullsized":"2023-10-18T12:24:09.961298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.216871+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:26:16.216871+0000","last_peered":"2023-10-18T12:26:16.216871+0000","last_clean":"2023-10-18T12:26:16.216871+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:26:16.216871+0000","last_undegraded":"2023-10-18T12:26:16.216871+0000","last_fullsized":"2023-10-18T12:26:16.216871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.212426+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:15.212426+0000","last_peered":"2023-10-18T12:25:15.212426+0000","last_clean":"2023-10-18T12:25:15.212426+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:15.212426+0000","last_undegraded":"2023-10-18T12:25:15.212426+0000","last_fullsized":"2023-10-18T12:25:15.212426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215078+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:26:15.215078+0000","last_peered":"2023-10-18T12:26:15.215078+0000","last_clean":"2023-10-18T12:26:15.215078+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:26:15.215078+0000","last_undegraded":"2023-10-18T12:26:15.215078+0000","last_fullsized":"2023-10-18T12:26:15.215078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649534+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:25:43.649534+0000","last_peered":"2023-10-18T12:25:43.649534+0000","last_clean":"2023-10-18T12:25:43.649534+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:25:43.649534+0000","last_undegraded":"2023-10-18T12:25:43.649534+0000","last_fullsized":"2023-10-18T12:25:43.649534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.217523+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:25:15.217523+0000","last_peered":"2023-10-18T12:25:15.217523+0000","last_clean":"2023-10-18T12:25:15.217523+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:25:15.217523+0000","last_undegraded":"2023-10-18T12:25:15.217523+0000","last_fullsized":"2023-10-18T12:25:15.217523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214480+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:25:14.214480+0000","last_peered":"2023-10-18T12:25:14.214480+0000","last_clean":"2023-10-18T12:25:14.214480+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:25:14.214480+0000","last_undegraded":"2023-10-18T12:25:14.214480+0000","last_fullsized":"2023-10-18T12:25:14.214480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.213970+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:25:15.213970+0000","last_peered":"2023-10-18T12:25:15.213970+0000","last_clean":"2023-10-18T12:25:15.213970+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:25:15.213970+0000","last_undegraded":"2023-10-18T12:25:15.213970+0000","last_fullsized":"2023-10-18T12:25:15.213970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649689+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:25:43.649689+0000","last_peered":"2023-10-18T12:25:43.649689+0000","last_clean":"2023-10-18T12:25:43.649689+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:25:43.649689+0000","last_undegraded":"2023-10-18T12:25:43.649689+0000","last_fullsized":"2023-10-18T12:25:43.649689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.214742+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:25:15.214742+0000","last_peered":"2023-10-18T12:25:15.214742+0000","last_clean":"2023-10-18T12:25:15.214742+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:25:15.214742+0000","last_undegraded":"2023-10-18T12:25:15.214742+0000","last_fullsized":"2023-10-18T12:25:15.214742+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:13.149946+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:26:13.149946+0000","last_peered":"2023-10-18T12:26:13.149946+0000","last_clean":"2023-10-18T12:26:13.149946+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:26:13.149946+0000","last_undegraded":"2023-10-18T12:26:13.149946+0000","last_fullsized":"2023-10-18T12:26:13.149946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643505+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:25:43.643505+0000","last_peered":"2023-10-18T12:25:43.643505+0000","last_clean":"2023-10-18T12:25:43.643505+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:25:43.643505+0000","last_undegraded":"2023-10-18T12:25:43.643505+0000","last_fullsized":"2023-10-18T12:25:43.643505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.212653+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:25:14.212653+0000","last_peered":"2023-10-18T12:25:14.212653+0000","last_clean":"2023-10-18T12:25:14.212653+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:25:14.212653+0000","last_undegraded":"2023-10-18T12:25:14.212653+0000","last_fullsized":"2023-10-18T12:25:14.212653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.211145+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:15.211145+0000","last_peered":"2023-10-18T12:25:15.211145+0000","last_clean":"2023-10-18T12:25:15.211145+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:15.211145+0000","last_undegraded":"2023-10-18T12:25:15.211145+0000","last_fullsized":"2023-10-18T12:25:15.211145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.649714+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:25:43.649714+0000","last_peered":"2023-10-18T12:25:43.649714+0000","last_clean":"2023-10-18T12:25:43.649714+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:25:43.649714+0000","last_undegraded":"2023-10-18T12:25:43.649714+0000","last_fullsized":"2023-10-18T12:25:43.649714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223680+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223680+0000","last_peered":"2023-10-18T12:25:15.223680+0000","last_clean":"2023-10-18T12:25:15.223680+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223680+0000","last_undegraded":"2023-10-18T12:25:15.223680+0000","last_fullsized":"2023-10-18T12:25:15.223680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214524+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:25:14.214524+0000","last_peered":"2023-10-18T12:25:14.214524+0000","last_clean":"2023-10-18T12:25:14.214524+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:25:14.214524+0000","last_undegraded":"2023-10-18T12:25:14.214524+0000","last_fullsized":"2023-10-18T12:25:14.214524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.604171+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:25:43.604171+0000","last_peered":"2023-10-18T12:25:43.604171+0000","last_clean":"2023-10-18T12:25:43.604171+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:25:43.604171+0000","last_undegraded":"2023-10-18T12:25:43.604171+0000","last_fullsized":"2023-10-18T12:25:43.604171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.220890+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:25:15.220890+0000","last_peered":"2023-10-18T12:25:15.220890+0000","last_clean":"2023-10-18T12:25:15.220890+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:25:15.220890+0000","last_undegraded":"2023-10-18T12:25:15.220890+0000","last_fullsized":"2023-10-18T12:25:15.220890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210834+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:25:14.210834+0000","last_peered":"2023-10-18T12:25:14.210834+0000","last_clean":"2023-10-18T12:25:14.210834+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:25:14.210834+0000","last_undegraded":"2023-10-18T12:25:14.210834+0000","last_fullsized":"2023-10-18T12:25:14.210834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.651462+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:25:43.651462+0000","last_peered":"2023-10-18T12:25:43.651462+0000","last_clean":"2023-10-18T12:25:43.651462+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:25:43.651462+0000","last_undegraded":"2023-10-18T12:25:43.651462+0000","last_fullsized":"2023-10-18T12:25:43.651462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210778+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:15.210778+0000","last_peered":"2023-10-18T12:25:15.210778+0000","last_clean":"2023-10-18T12:25:15.210778+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:15.210778+0000","last_undegraded":"2023-10-18T12:25:15.210778+0000","last_fullsized":"2023-10-18T12:25:15.210778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.210884+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:25:14.210884+0000","last_peered":"2023-10-18T12:25:14.210884+0000","last_clean":"2023-10-18T12:25:14.210884+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:25:14.210884+0000","last_undegraded":"2023-10-18T12:25:14.210884+0000","last_fullsized":"2023-10-18T12:25:14.210884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.648198+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:25:43.648198+0000","last_peered":"2023-10-18T12:25:43.648198+0000","last_clean":"2023-10-18T12:25:43.648198+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:25:43.648198+0000","last_undegraded":"2023-10-18T12:25:43.648198+0000","last_fullsized":"2023-10-18T12:25:43.648198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210651+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:15.210651+0000","last_peered":"2023-10-18T12:25:15.210651+0000","last_clean":"2023-10-18T12:25:15.210651+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:15.210651+0000","last_undegraded":"2023-10-18T12:25:15.210651+0000","last_fullsized":"2023-10-18T12:25:15.210651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.216327+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:25:14.216327+0000","last_peered":"2023-10-18T12:25:14.216327+0000","last_clean":"2023-10-18T12:25:14.216327+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:25:14.216327+0000","last_undegraded":"2023-10-18T12:25:14.216327+0000","last_fullsized":"2023-10-18T12:25:14.216327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.652574+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:25:43.652574+0000","last_peered":"2023-10-18T12:25:43.652574+0000","last_clean":"2023-10-18T12:25:43.652574+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:25:43.652574+0000","last_undegraded":"2023-10-18T12:25:43.652574+0000","last_fullsized":"2023-10-18T12:25:43.652574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.214357+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:25:14.214357+0000","last_peered":"2023-10-18T12:25:14.214357+0000","last_clean":"2023-10-18T12:25:14.214357+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:25:14.214357+0000","last_undegraded":"2023-10-18T12:25:14.214357+0000","last_fullsized":"2023-10-18T12:25:14.214357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.211315+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:25:14.211315+0000","last_peered":"2023-10-18T12:25:14.211315+0000","last_clean":"2023-10-18T12:25:14.211315+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:25:14.211315+0000","last_undegraded":"2023-10-18T12:25:14.211315+0000","last_fullsized":"2023-10-18T12:25:14.211315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644409+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:43.644409+0000","last_peered":"2023-10-18T12:25:43.644409+0000","last_clean":"2023-10-18T12:25:43.644409+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:43.644409+0000","last_undegraded":"2023-10-18T12:25:43.644409+0000","last_fullsized":"2023-10-18T12:25:43.644409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T12:25:14.218308+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:25:14.218308+0000","last_peered":"2023-10-18T12:25:14.218308+0000","last_clean":"2023-10-18T12:25:14.218308+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:25:14.218308+0000","last_undegraded":"2023-10-18T12:25:14.218308+0000","last_fullsized":"2023-10-18T12:25:14.218308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214930+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:26:15.214930+0000","last_peered":"2023-10-18T12:26:15.214930+0000","last_clean":"2023-10-18T12:26:15.214930+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:26:15.214930+0000","last_undegraded":"2023-10-18T12:26:15.214930+0000","last_fullsized":"2023-10-18T12:26:15.214930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639071+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:43.639071+0000","last_peered":"2023-10-18T12:25:43.639071+0000","last_clean":"2023-10-18T12:25:43.639071+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:43.639071+0000","last_undegraded":"2023-10-18T12:25:43.639071+0000","last_fullsized":"2023-10-18T12:25:43.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214280+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:26:15.214280+0000","last_peered":"2023-10-18T12:26:15.214280+0000","last_clean":"2023-10-18T12:26:15.214280+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:26:15.214280+0000","last_undegraded":"2023-10-18T12:26:15.214280+0000","last_fullsized":"2023-10-18T12:26:15.214280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.225441+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:25:15.225441+0000","last_peered":"2023-10-18T12:25:15.225441+0000","last_clean":"2023-10-18T12:25:15.225441+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:25:15.225441+0000","last_undegraded":"2023-10-18T12:25:15.225441+0000","last_fullsized":"2023-10-18T12:25:15.225441+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639284+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:43.639284+0000","last_peered":"2023-10-18T12:25:43.639284+0000","last_clean":"2023-10-18T12:25:43.639284+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:43.639284+0000","last_undegraded":"2023-10-18T12:25:43.639284+0000","last_fullsized":"2023-10-18T12:25:43.639284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.638348+0000","last_change":"2023-10-18T12:24:09.972756+0000","last_active":"2023-10-18T12:25:43.638348+0000","last_peered":"2023-10-18T12:25:43.638348+0000","last_clean":"2023-10-18T12:25:43.638348+0000","last_became_active":"2023-10-18T12:24:09.972359+0000","last_became_peered":"2023-10-18T12:24:09.972359+0000","last_unstale":"2023-10-18T12:25:43.638348+0000","last_undegraded":"2023-10-18T12:25:43.638348+0000","last_fullsized":"2023-10-18T12:25:43.638348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:12.219348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643095+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:25:43.643095+0000","last_peered":"2023-10-18T12:25:43.643095+0000","last_clean":"2023-10-18T12:25:43.643095+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:25:43.643095+0000","last_undegraded":"2023-10-18T12:25:43.643095+0000","last_fullsized":"2023-10-18T12:25:43.643095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.646655+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:25:43.646655+0000","last_peered":"2023-10-18T12:25:43.646655+0000","last_clean":"2023-10-18T12:25:43.646655+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:25:43.646655+0000","last_undegraded":"2023-10-18T12:25:43.646655+0000","last_fullsized":"2023-10-18T12:25:43.646655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.656634+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:25:43.656634+0000","last_peered":"2023-10-18T12:25:43.656634+0000","last_clean":"2023-10-18T12:25:43.656634+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:25:43.656634+0000","last_undegraded":"2023-10-18T12:25:43.656634+0000","last_fullsized":"2023-10-18T12:25:43.656634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.655509+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:25:43.655509+0000","last_peered":"2023-10-18T12:25:43.655509+0000","last_clean":"2023-10-18T12:25:43.655509+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:25:43.655509+0000","last_undegraded":"2023-10-18T12:25:43.655509+0000","last_fullsized":"2023-10-18T12:25:43.655509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639525+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:25:43.639525+0000","last_peered":"2023-10-18T12:25:43.639525+0000","last_clean":"2023-10-18T12:25:43.639525+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:25:43.639525+0000","last_undegraded":"2023-10-18T12:25:43.639525+0000","last_fullsized":"2023-10-18T12:25:43.639525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644102+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:25:43.644102+0000","last_peered":"2023-10-18T12:25:43.644102+0000","last_clean":"2023-10-18T12:25:43.644102+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:25:43.644102+0000","last_undegraded":"2023-10-18T12:25:43.644102+0000","last_fullsized":"2023-10-18T12:25:43.644102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644112+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:25:43.644112+0000","last_peered":"2023-10-18T12:25:43.644112+0000","last_clean":"2023-10-18T12:25:43.644112+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:25:43.644112+0000","last_undegraded":"2023-10-18T12:25:43.644112+0000","last_fullsized":"2023-10-18T12:25:43.644112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214790+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:26:15.214790+0000","last_peered":"2023-10-18T12:26:15.214790+0000","last_clean":"2023-10-18T12:26:15.214790+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:26:15.214790+0000","last_undegraded":"2023-10-18T12:26:15.214790+0000","last_fullsized":"2023-10-18T12:26:15.214790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.636709+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:25:43.636709+0000","last_peered":"2023-10-18T12:25:43.636709+0000","last_clean":"2023-10-18T12:25:43.636709+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:25:43.636709+0000","last_undegraded":"2023-10-18T12:25:43.636709+0000","last_fullsized":"2023-10-18T12:25:43.636709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643394+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:25:43.643394+0000","last_peered":"2023-10-18T12:25:43.643394+0000","last_clean":"2023-10-18T12:25:43.643394+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:25:43.643394+0000","last_undegraded":"2023-10-18T12:25:43.643394+0000","last_fullsized":"2023-10-18T12:25:43.643394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.657911+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:25:43.657911+0000","last_peered":"2023-10-18T12:25:43.657911+0000","last_clean":"2023-10-18T12:25:43.657911+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:25:43.657911+0000","last_undegraded":"2023-10-18T12:25:43.657911+0000","last_fullsized":"2023-10-18T12:25:43.657911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.643480+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:25:43.643480+0000","last_peered":"2023-10-18T12:25:43.643480+0000","last_clean":"2023-10-18T12:25:43.643480+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:25:43.643480+0000","last_undegraded":"2023-10-18T12:25:43.643480+0000","last_fullsized":"2023-10-18T12:25:43.643480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67627,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":222,"num_read_kb":179,"num_write":82,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":202881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":1.147,"15min":1.147},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.745}]},{"osd":2,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88400000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":1.238,"15min":1.238},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75600000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":1.661,"15min":1.661},"last":0.84599999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90400000000000003,"5min":1.361,"15min":1.361},"last":0.872},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":1,"15min":1},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.91100000000000003,"5min":1.726,"15min":1.726},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.012,"5min":1.401,"15min":1.401},"last":0.95899999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89000000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.92400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1720320,"data_stored":1176260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.54500000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.435,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63300000000000001,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.999,"15min":0.999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.85199999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72499999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.76600000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84599999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.876,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.92700000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.77600000000000002,"5min":1.099,"15min":1.099},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82799999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.014}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90400000000000003,"5min":1.071,"15min":1.071},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.044,"15min":1.044},"last":1.0289999999999999}]},{"osd":9,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.999}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.048},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":1.0589999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":527643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0089999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.92700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.753},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.86199999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1739999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1519999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.91500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.871},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.94599999999999995,"15min":0.95199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":6.2220000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.69699999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.89100000000000001},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2430000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78000000000000003}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.432},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.501}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Wed Oct 18 12:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0149999999999999,"5min":1.044,"15min":1.044},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.72399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0640000000000001,"5min":1.26,"15min":1.26},"last":0.754}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.68300000000000005},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76800000000000002,"15min":0.72799999999999998},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.107,"15min":1.107},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66700000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1108170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.433,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.501},{"interface":"front","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60099999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35799999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.999,"15min":0.999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.137,"15min":1.137},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81299999999999994,"5min":1.071,"15min":1.071},"last":0.57299999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.66500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.98199999999999998,"5min":1.02,"15min":1.02},"last":0.72799999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95399999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69199999999999995}]},{"osd":11,"last update":"Wed Oct 18 12:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70699999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":509028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.53000000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.622},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.753}]},{"osd":3,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.68799999999999994,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.9199999999999999,"5min":6.9199999999999999,"15min":6.9199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.026,"5min":0.81899999999999995,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":7.2270000000000003,"5min":7.2270000000000003,"15min":7.2270000000000003},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.016,"5min":1.234,"15min":1.234},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74299999999999999,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.28,"5min":1.849,"15min":1.849},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84999999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.88,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":7.7400000000000002,"5min":7.7400000000000002,"15min":7.7400000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.69699999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.90800000000000003,"15min":0.88900000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.4969999999999999,"5min":7.4969999999999999,"15min":7.4969999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95499999999999996,"15min":0.90300000000000002},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3240000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.83699999999999997}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":570478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.161,"15min":1.161},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94699999999999995,"5min":1.454,"15min":1.454},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.57299999999999995},"min":{"1min":0.52000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88500000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71899999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.76300000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.64800000000000002,"15min":0.59099999999999997},"min":{"1min":0.63200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":1.298,"15min":1.298},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.92500000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.52700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89400000000000002,"5min":1.016,"15min":1.016},"last":0.88200000000000001}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.002}]}]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":509118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.65200000000000002,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.30299999999999999},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65600000000000003,"15min":0.55200000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.32400000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.60699999999999998},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.84099999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1479999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.024,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.65800000000000003,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.69699999999999995,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.498}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.66200000000000003},"min":{"1min":0.51800000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.65700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.755}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69899999999999995,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.751,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0489999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":0.92100000000000004}]},{"osd":9,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86799999999999999,"15min":0.84599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.61199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.89300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.876},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.89300000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.03,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93899999999999995,"15min":0.92400000000000004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":0.90000000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":575618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.26100000000000001},"max":{"1min":0.92100000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.54600000000000004},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.626},{"interface":"front","average":{"1min":0.745,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.86899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.34000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73799999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.42299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.58399999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.753},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85399999999999998}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76500000000000001,"15min":0.74099999999999999},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.254,"15min":1.254},"last":0.995}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.74299999999999999,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91600000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":509132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.51400000000000001,"15min":0.436},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7880000000000003,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.52700000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.73599999999999999,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.622},{"interface":"front","average":{"1min":1.026,"5min":0.68999999999999995,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":5.0579999999999998,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.73699999999999999,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.085,"5min":0.77800000000000002,"15min":0.70599999999999996},"min":{"1min":0.60999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:25:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.78400000000000003,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.7439999999999998,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.753,"15min":0.70699999999999996},"min":{"1min":0.60099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.806,"5min":3.806,"15min":3.806},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.752,"15min":0.69499999999999995},"min":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.83299999999999996,"15min":0.753},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.984,"5min":4.984,"15min":4.984},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3789999999999996,"5min":5.3789999999999996,"15min":5.3789999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.86199999999999999},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93799999999999994,"15min":0.90500000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.83099999999999996}]},{"osd":11,"last update":"Wed Oct 18 12:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.016,"15min":1.052},"min":{"1min":0.61899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2030000000000001,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.111,"15min":1.155},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2170000000000001,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.92900000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54800000000000004,"5min":0.621,"15min":0.621},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57099999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.876,"5min":1.04,"15min":1.04},"last":0.64700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.52200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":1.22,"15min":1.22},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52800000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80200000000000005,"15min":0.81100000000000005},"min":{"1min":0.65900000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.02,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.55700000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.996,"5min":0.997,"15min":0.997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85599999999999998,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1123041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":0.999},"last":0.754},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63500000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.752},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":4.548,"15min":4.548},"last":0.57399999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.745,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73999999999999999,"15min":0.70799999999999996},"min":{"1min":0.60699999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:26:18.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:18 smithi070 ceph-mon[98639]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:18.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:18 smithi070 ceph-mon[98639]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:18.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:18 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1628788372' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-18T12:26:18.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:18 smithi152 ceph-mon[134767]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:18.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:18 smithi152 ceph-mon[134767]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:18.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:18 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1628788372' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-18T12:26:18.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:18 smithi186 ceph-mon[135253]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:18.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:18 smithi186 ceph-mon[135253]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:18.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:18 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1628788372' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-18T12:26:19.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:19 smithi070 ceph-mon[98639]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:19.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:19 smithi070 ceph-mon[98639]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2023-10-18T12:26:19.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:19 smithi070 ceph-mon[98639]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-18T12:26:19.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:19 smithi070 ceph-mon[98639]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:19 smithi152 ceph-mon[134767]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:19.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:19 smithi152 ceph-mon[134767]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2023-10-18T12:26:19.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:19 smithi152 ceph-mon[134767]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-18T12:26:19.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:19 smithi152 ceph-mon[134767]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:19 smithi186 ceph-mon[135253]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:19.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:19 smithi186 ceph-mon[135253]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2023-10-18T12:26:19.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:19 smithi186 ceph-mon[135253]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-18T12:26:19.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:19 smithi186 ceph-mon[135253]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:20.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:20 smithi070 ceph-mon[98639]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 895 B/s wr, 1 op/s 2023-10-18T12:26:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:20 smithi152 ceph-mon[134767]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 895 B/s wr, 1 op/s 2023-10-18T12:26:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:20 smithi186 ceph-mon[135253]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 895 B/s wr, 1 op/s 2023-10-18T12:26:21.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:26:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 -- 172.21.15.70:0/3255630770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 msgr2=0x7f05a8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 --2- 172.21.15.70:0/3255630770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 0x7f05a8106a90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009990 tx=0x7f05a4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 -- 172.21.15.70:0/3255630770 shutdown_connections 2023-10-18T12:26:21.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 --2- 172.21.15.70:0/3255630770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a8101060 0x7f05a810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 --2- 172.21.15.70:0/3255630770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 0x7f05a8106a90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 --2- 172.21.15.70:0/3255630770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a81006a0 0x7f05a8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 -- 172.21.15.70:0/3255630770 >> 172.21.15.70:0/3255630770 conn(0x7f05a80fc1d0 msgr2=0x7f05a80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:21.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 -- 172.21.15.70:0/3255630770 shutdown_connections 2023-10-18T12:26:21.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.572+0000 7f05af3ff700 1 -- 172.21.15.70:0/3255630770 wait complete. 2023-10-18T12:26:21.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.573+0000 7f05af3ff700 1 Processor -- start 2023-10-18T12:26:21.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.573+0000 7f05af3ff700 1 -- start start 2023-10-18T12:26:21.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05af3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 0x7f05a8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05af3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a8101060 0x7f05a8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05af3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 0x7f05a819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05af3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a819cb60 con 0x7f05a81006a0 2023-10-18T12:26:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05af3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a819ccd0 con 0x7f05a8101060 2023-10-18T12:26:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05af3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a819ce90 con 0x7f05a81066b0 2023-10-18T12:26:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 0x7f05a8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 0x7f05a8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55394/0 (socket says 172.21.15.70:55394) 2023-10-18T12:26:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 -- 172.21.15.70:0/2513307197 learned_addr learned my addr 172.21.15.70:0/2513307197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad99c700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 0x7f05a819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ac99a700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a8101060 0x7f05a8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 -- 172.21.15.70:0/2513307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 msgr2=0x7f05a819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 0x7f05a819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 -- 172.21.15.70:0/2513307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a8101060 msgr2=0x7f05a8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a8101060 0x7f05a8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.574+0000 7f05ad19b700 1 -- 172.21.15.70:0/2513307197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009520 con 0x7f05a81006a0 2023-10-18T12:26:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.575+0000 7f05ad19b700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 0x7f05a8198d30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005910 tx=0x7f05a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.575+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a401d070 con 0x7f05a81006a0 2023-10-18T12:26:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.575+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a4022470 con 0x7f05a81006a0 2023-10-18T12:26:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.575+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a0009c40 con 0x7f05a81006a0 2023-10-18T12:26:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.575+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4017650 con 0x7f05a81006a0 2023-10-18T12:26:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.575+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a40095f0 con 0x7f05a81006a0 2023-10-18T12:26:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.576+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a8101de0 con 0x7f05a81006a0 2023-10-18T12:26:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.578+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f05a40225e0 con 0x7f05a81006a0 2023-10-18T12:26:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.579+0000 7f059e7fc700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05940a1550 0x7f05940a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.579+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f05a40ce270 con 0x7f05a81006a0 2023-10-18T12:26:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.580+0000 7f05ac99a700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05940a1550 0x7f05940a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.580+0000 7f05ac99a700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05940a1550 0x7f05940a3a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f05a819a240 tx=0x7f0598009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:21.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.584+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f05a405d100 con 0x7f05a81006a0 2023-10-18T12:26:21.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.788+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05a8110620 con 0x7f05940a1550 2023-10-18T12:26:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.801+0000 7f059e7fc700 1 -- 172.21.15.70:0/2513307197 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427303 (secure 0 0 0) 0x7f05a8110620 con 0x7f05940a1550 2023-10-18T12:26:21.809 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:21.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05940a1550 msgr2=0x7f05940a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:21.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05940a1550 0x7f05940a3a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f05a819a240 tx=0x7f0598009260 comp rx=0 tx=0).stop 2023-10-18T12:26:21.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 msgr2=0x7f05a8198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:21.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 0x7f05a8198d30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005910 tx=0x7f05a4004730 comp rx=0 tx=0).stop 2023-10-18T12:26:21.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 shutdown_connections 2023-10-18T12:26:21.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a81066b0 0x7f05a819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a81006a0 0x7f05a8198d30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f05940a1550 0x7f05940a3a10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 --2- 172.21.15.70:0/2513307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a8101060 0x7f05a8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 >> 172.21.15.70:0/2513307197 conn(0x7f05a80fc1d0 msgr2=0x7f05a80fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 shutdown_connections 2023-10-18T12:26:21.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:21.806+0000 7f05af3ff700 1 -- 172.21.15.70:0/2513307197 wait complete. 2023-10-18T12:26:21.818 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:26:21.829 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2023-10-18T12:26:21.626233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658241,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":368,"num_read_kb":303,"num_write":265,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":711,"ondisk_log_size":711,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338628,"kb_used_data":14356,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603196,"statfs":{"total":1151940427776,"available":1151593672704,"internally_reserved":0,"allocated":14700544,"data_stored":8331163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5379,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":16,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002265"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.220660+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:26:16.220660+0000","last_peered":"2023-10-18T12:26:16.220660+0000","last_clean":"2023-10-18T12:26:16.220660+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:26:16.220660+0000","last_undegraded":"2023-10-18T12:26:16.220660+0000","last_fullsized":"2023-10-18T12:26:16.220660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926320+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:26:15.926320+0000","last_peered":"2023-10-18T12:26:15.926320+0000","last_clean":"2023-10-18T12:26:15.926320+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:26:15.926320+0000","last_undegraded":"2023-10-18T12:26:15.926320+0000","last_fullsized":"2023-10-18T12:26:15.926320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926264+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:26:15.926264+0000","last_peered":"2023-10-18T12:26:15.926264+0000","last_clean":"2023-10-18T12:26:15.926264+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:26:15.926264+0000","last_undegraded":"2023-10-18T12:26:15.926264+0000","last_fullsized":"2023-10-18T12:26:15.926264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:11.635030+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:26:11.635030+0000","last_peered":"2023-10-18T12:26:11.635030+0000","last_clean":"2023-10-18T12:26:11.635030+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:26:11.635030+0000","last_undegraded":"2023-10-18T12:26:11.635030+0000","last_fullsized":"2023-10-18T12:26:11.635030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804954+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:26:15.804954+0000","last_peered":"2023-10-18T12:26:15.804954+0000","last_clean":"2023-10-18T12:26:15.804954+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:26:15.804954+0000","last_undegraded":"2023-10-18T12:26:15.804954+0000","last_fullsized":"2023-10-18T12:26:15.804954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210846+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:26:15.210846+0000","last_peered":"2023-10-18T12:26:15.210846+0000","last_clean":"2023-10-18T12:26:15.210846+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:26:15.210846+0000","last_undegraded":"2023-10-18T12:26:15.210846+0000","last_fullsized":"2023-10-18T12:26:15.210846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639977+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:43.639977+0000","last_peered":"2023-10-18T12:25:43.639977+0000","last_clean":"2023-10-18T12:25:43.639977+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:43.639977+0000","last_undegraded":"2023-10-18T12:25:43.639977+0000","last_fullsized":"2023-10-18T12:25:43.639977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210117+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:26:15.210117+0000","last_peered":"2023-10-18T12:26:15.210117+0000","last_clean":"2023-10-18T12:26:15.210117+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:26:15.210117+0000","last_undegraded":"2023-10-18T12:26:15.210117+0000","last_fullsized":"2023-10-18T12:26:15.210117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214861+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:26:15.214861+0000","last_peered":"2023-10-18T12:26:15.214861+0000","last_clean":"2023-10-18T12:26:15.214861+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:26:15.214861+0000","last_undegraded":"2023-10-18T12:26:15.214861+0000","last_fullsized":"2023-10-18T12:26:15.214861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210940+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:26:15.210940+0000","last_peered":"2023-10-18T12:26:15.210940+0000","last_clean":"2023-10-18T12:26:15.210940+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:26:15.210940+0000","last_undegraded":"2023-10-18T12:26:15.210940+0000","last_fullsized":"2023-10-18T12:26:15.210940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644437+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:43.644437+0000","last_peered":"2023-10-18T12:25:43.644437+0000","last_clean":"2023-10-18T12:25:43.644437+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:43.644437+0000","last_undegraded":"2023-10-18T12:25:43.644437+0000","last_fullsized":"2023-10-18T12:25:43.644437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633660+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:43.633660+0000","last_peered":"2023-10-18T12:25:43.633660+0000","last_clean":"2023-10-18T12:25:43.633660+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:43.633660+0000","last_undegraded":"2023-10-18T12:25:43.633660+0000","last_fullsized":"2023-10-18T12:25:43.633660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422485+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:26:15.422485+0000","last_peered":"2023-10-18T12:26:15.422485+0000","last_clean":"2023-10-18T12:26:15.422485+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:26:15.422485+0000","last_undegraded":"2023-10-18T12:26:15.422485+0000","last_fullsized":"2023-10-18T12:26:15.422485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.925939+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:26:15.925939+0000","last_peered":"2023-10-18T12:26:15.925939+0000","last_clean":"2023-10-18T12:26:15.925939+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:26:15.925939+0000","last_undegraded":"2023-10-18T12:26:15.925939+0000","last_fullsized":"2023-10-18T12:26:15.925939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.925933+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:26:15.925933+0000","last_peered":"2023-10-18T12:26:15.925933+0000","last_clean":"2023-10-18T12:26:15.925933+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:26:15.925933+0000","last_undegraded":"2023-10-18T12:26:15.925933+0000","last_fullsized":"2023-10-18T12:26:15.925933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214139+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:26:15.214139+0000","last_peered":"2023-10-18T12:26:15.214139+0000","last_clean":"2023-10-18T12:26:15.214139+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:26:15.214139+0000","last_undegraded":"2023-10-18T12:26:15.214139+0000","last_fullsized":"2023-10-18T12:26:15.214139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213393+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:26:15.213393+0000","last_peered":"2023-10-18T12:26:15.213393+0000","last_clean":"2023-10-18T12:26:15.213393+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:26:15.213393+0000","last_undegraded":"2023-10-18T12:26:15.213393+0000","last_fullsized":"2023-10-18T12:26:15.213393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213495+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:26:15.213495+0000","last_peered":"2023-10-18T12:26:15.213495+0000","last_clean":"2023-10-18T12:26:15.213495+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:26:15.213495+0000","last_undegraded":"2023-10-18T12:26:15.213495+0000","last_fullsized":"2023-10-18T12:26:15.213495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211068+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:26:15.211068+0000","last_peered":"2023-10-18T12:26:15.211068+0000","last_clean":"2023-10-18T12:26:15.211068+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:26:15.211068+0000","last_undegraded":"2023-10-18T12:26:15.211068+0000","last_fullsized":"2023-10-18T12:26:15.211068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210779+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:26:15.210779+0000","last_peered":"2023-10-18T12:26:15.210779+0000","last_clean":"2023-10-18T12:26:15.210779+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:26:15.210779+0000","last_undegraded":"2023-10-18T12:26:15.210779+0000","last_fullsized":"2023-10-18T12:26:15.210779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.661330+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:26:15.661330+0000","last_peered":"2023-10-18T12:26:15.661330+0000","last_clean":"2023-10-18T12:26:15.661330+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:26:15.661330+0000","last_undegraded":"2023-10-18T12:26:15.661330+0000","last_fullsized":"2023-10-18T12:26:15.661330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210706+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:26:15.210706+0000","last_peered":"2023-10-18T12:26:15.210706+0000","last_clean":"2023-10-18T12:26:15.210706+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:26:15.210706+0000","last_undegraded":"2023-10-18T12:26:15.210706+0000","last_fullsized":"2023-10-18T12:26:15.210706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210351+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:15.210351+0000","last_peered":"2023-10-18T12:25:15.210351+0000","last_clean":"2023-10-18T12:25:15.210351+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:15.210351+0000","last_undegraded":"2023-10-18T12:25:15.210351+0000","last_fullsized":"2023-10-18T12:25:15.210351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210836+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:26:15.210836+0000","last_peered":"2023-10-18T12:26:15.210836+0000","last_clean":"2023-10-18T12:26:15.210836+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:26:15.210836+0000","last_undegraded":"2023-10-18T12:26:15.210836+0000","last_fullsized":"2023-10-18T12:26:15.210836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926330+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:26:15.926330+0000","last_peered":"2023-10-18T12:26:15.926330+0000","last_clean":"2023-10-18T12:26:15.926330+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:26:15.926330+0000","last_undegraded":"2023-10-18T12:26:15.926330+0000","last_fullsized":"2023-10-18T12:26:15.926330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210533+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:15.210533+0000","last_peered":"2023-10-18T12:25:15.210533+0000","last_clean":"2023-10-18T12:25:15.210533+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:15.210533+0000","last_undegraded":"2023-10-18T12:25:15.210533+0000","last_fullsized":"2023-10-18T12:25:15.210533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805155+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:26:15.805155+0000","last_peered":"2023-10-18T12:26:15.805155+0000","last_clean":"2023-10-18T12:26:15.805155+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:26:15.805155+0000","last_undegraded":"2023-10-18T12:26:15.805155+0000","last_fullsized":"2023-10-18T12:26:15.805155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214360+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:26:15.214360+0000","last_peered":"2023-10-18T12:26:15.214360+0000","last_clean":"2023-10-18T12:26:15.214360+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:26:15.214360+0000","last_undegraded":"2023-10-18T12:26:15.214360+0000","last_fullsized":"2023-10-18T12:26:15.214360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210967+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:15.210967+0000","last_peered":"2023-10-18T12:25:15.210967+0000","last_clean":"2023-10-18T12:25:15.210967+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:15.210967+0000","last_undegraded":"2023-10-18T12:25:15.210967+0000","last_fullsized":"2023-10-18T12:25:15.210967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805304+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:26:15.805304+0000","last_peered":"2023-10-18T12:26:15.805304+0000","last_clean":"2023-10-18T12:26:15.805304+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:26:15.805304+0000","last_undegraded":"2023-10-18T12:26:15.805304+0000","last_fullsized":"2023-10-18T12:26:15.805304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214868+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:26:15.214868+0000","last_peered":"2023-10-18T12:26:15.214868+0000","last_clean":"2023-10-18T12:26:15.214868+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:26:15.214868+0000","last_undegraded":"2023-10-18T12:26:15.214868+0000","last_fullsized":"2023-10-18T12:26:15.214868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210854+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:26:15.210854+0000","last_peered":"2023-10-18T12:26:15.210854+0000","last_clean":"2023-10-18T12:26:15.210854+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:26:15.210854+0000","last_undegraded":"2023-10-18T12:26:15.210854+0000","last_fullsized":"2023-10-18T12:26:15.210854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214751+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:26:15.214751+0000","last_peered":"2023-10-18T12:26:15.214751+0000","last_clean":"2023-10-18T12:26:15.214751+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:26:15.214751+0000","last_undegraded":"2023-10-18T12:26:15.214751+0000","last_fullsized":"2023-10-18T12:26:15.214751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804971+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:26:15.804971+0000","last_peered":"2023-10-18T12:26:15.804971+0000","last_clean":"2023-10-18T12:26:15.804971+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:26:15.804971+0000","last_undegraded":"2023-10-18T12:26:15.804971+0000","last_fullsized":"2023-10-18T12:26:15.804971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211287+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:26:15.211287+0000","last_peered":"2023-10-18T12:26:15.211287+0000","last_clean":"2023-10-18T12:26:15.211287+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:26:15.211287+0000","last_undegraded":"2023-10-18T12:26:15.211287+0000","last_fullsized":"2023-10-18T12:26:15.211287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210513+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:26:15.210513+0000","last_peered":"2023-10-18T12:26:15.210513+0000","last_clean":"2023-10-18T12:26:15.210513+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:26:15.210513+0000","last_undegraded":"2023-10-18T12:26:15.210513+0000","last_fullsized":"2023-10-18T12:26:15.210513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215014+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:26:15.215014+0000","last_peered":"2023-10-18T12:26:15.215014+0000","last_clean":"2023-10-18T12:26:15.215014+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:26:15.215014+0000","last_undegraded":"2023-10-18T12:26:15.215014+0000","last_fullsized":"2023-10-18T12:26:15.215014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213129+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:26:15.213129+0000","last_peered":"2023-10-18T12:26:15.213129+0000","last_clean":"2023-10-18T12:26:15.213129+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:26:15.213129+0000","last_undegraded":"2023-10-18T12:26:15.213129+0000","last_fullsized":"2023-10-18T12:26:15.213129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422291+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:26:15.422291+0000","last_peered":"2023-10-18T12:26:15.422291+0000","last_clean":"2023-10-18T12:26:15.422291+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:26:15.422291+0000","last_undegraded":"2023-10-18T12:26:15.422291+0000","last_fullsized":"2023-10-18T12:26:15.422291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210045+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:26:15.210045+0000","last_peered":"2023-10-18T12:26:15.210045+0000","last_clean":"2023-10-18T12:26:15.210045+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:26:15.210045+0000","last_undegraded":"2023-10-18T12:26:15.210045+0000","last_fullsized":"2023-10-18T12:26:15.210045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926312+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:26:15.926312+0000","last_peered":"2023-10-18T12:26:15.926312+0000","last_clean":"2023-10-18T12:26:15.926312+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:26:15.926312+0000","last_undegraded":"2023-10-18T12:26:15.926312+0000","last_fullsized":"2023-10-18T12:26:15.926312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214658+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:26:15.214658+0000","last_peered":"2023-10-18T12:26:15.214658+0000","last_clean":"2023-10-18T12:26:15.214658+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:26:15.214658+0000","last_undegraded":"2023-10-18T12:26:15.214658+0000","last_fullsized":"2023-10-18T12:26:15.214658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213122+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:26:15.213122+0000","last_peered":"2023-10-18T12:26:15.213122+0000","last_clean":"2023-10-18T12:26:15.213122+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:26:15.213122+0000","last_undegraded":"2023-10-18T12:26:15.213122+0000","last_fullsized":"2023-10-18T12:26:15.213122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210318+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:26:15.210318+0000","last_peered":"2023-10-18T12:26:15.210318+0000","last_clean":"2023-10-18T12:26:15.210318+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:26:15.210318+0000","last_undegraded":"2023-10-18T12:26:15.210318+0000","last_fullsized":"2023-10-18T12:26:15.210318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210810+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:26:15.210810+0000","last_peered":"2023-10-18T12:26:15.210810+0000","last_clean":"2023-10-18T12:26:15.210810+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:26:15.210810+0000","last_undegraded":"2023-10-18T12:26:15.210810+0000","last_fullsized":"2023-10-18T12:26:15.210810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214669+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:26:15.214669+0000","last_peered":"2023-10-18T12:26:15.214669+0000","last_clean":"2023-10-18T12:26:15.214669+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:26:15.214669+0000","last_undegraded":"2023-10-18T12:26:15.214669+0000","last_fullsized":"2023-10-18T12:26:15.214669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212867+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:26:15.212867+0000","last_peered":"2023-10-18T12:26:15.212867+0000","last_clean":"2023-10-18T12:26:15.212867+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:26:15.212867+0000","last_undegraded":"2023-10-18T12:26:15.212867+0000","last_fullsized":"2023-10-18T12:26:15.212867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.212937+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:15.212937+0000","last_peered":"2023-10-18T12:25:15.212937+0000","last_clean":"2023-10-18T12:25:15.212937+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:15.212937+0000","last_undegraded":"2023-10-18T12:25:15.212937+0000","last_fullsized":"2023-10-18T12:25:15.212937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926421+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:26:15.926421+0000","last_peered":"2023-10-18T12:26:15.926421+0000","last_clean":"2023-10-18T12:26:15.926421+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:26:15.926421+0000","last_undegraded":"2023-10-18T12:26:15.926421+0000","last_fullsized":"2023-10-18T12:26:15.926421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210694+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:15.210694+0000","last_peered":"2023-10-18T12:25:15.210694+0000","last_clean":"2023-10-18T12:25:15.210694+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:15.210694+0000","last_undegraded":"2023-10-18T12:25:15.210694+0000","last_fullsized":"2023-10-18T12:25:15.210694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214711+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:26:15.214711+0000","last_peered":"2023-10-18T12:26:15.214711+0000","last_clean":"2023-10-18T12:26:15.214711+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:26:15.214711+0000","last_undegraded":"2023-10-18T12:26:15.214711+0000","last_fullsized":"2023-10-18T12:26:15.214711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214469+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:26:15.214469+0000","last_peered":"2023-10-18T12:26:15.214469+0000","last_clean":"2023-10-18T12:26:15.214469+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:26:15.214469+0000","last_undegraded":"2023-10-18T12:26:15.214469+0000","last_fullsized":"2023-10-18T12:26:15.214469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213164+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:26:15.213164+0000","last_peered":"2023-10-18T12:26:15.213164+0000","last_clean":"2023-10-18T12:26:15.213164+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:26:15.213164+0000","last_undegraded":"2023-10-18T12:26:15.213164+0000","last_fullsized":"2023-10-18T12:26:15.213164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":282,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:21.345751+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:26:21.345751+0000","last_peered":"2023-10-18T12:26:21.345751+0000","last_clean":"2023-10-18T12:26:21.345751+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:26:21.345751+0000","last_undegraded":"2023-10-18T12:26:21.345751+0000","last_fullsized":"2023-10-18T12:26:21.345751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66829,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":227,"num_read_kb":179,"num_write":35,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805288+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:26:15.805288+0000","last_peered":"2023-10-18T12:26:15.805288+0000","last_clean":"2023-10-18T12:26:15.805288+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:26:15.805288+0000","last_undegraded":"2023-10-18T12:26:15.805288+0000","last_fullsized":"2023-10-18T12:26:15.805288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215157+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:26:15.215157+0000","last_peered":"2023-10-18T12:26:15.215157+0000","last_clean":"2023-10-18T12:26:15.215157+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:26:15.215157+0000","last_undegraded":"2023-10-18T12:26:15.215157+0000","last_fullsized":"2023-10-18T12:26:15.215157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210818+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:26:15.210818+0000","last_peered":"2023-10-18T12:26:15.210818+0000","last_clean":"2023-10-18T12:26:15.210818+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:26:15.210818+0000","last_undegraded":"2023-10-18T12:26:15.210818+0000","last_fullsized":"2023-10-18T12:26:15.210818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422498+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:26:15.422498+0000","last_peered":"2023-10-18T12:26:15.422498+0000","last_clean":"2023-10-18T12:26:15.422498+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:26:15.422498+0000","last_undegraded":"2023-10-18T12:26:15.422498+0000","last_fullsized":"2023-10-18T12:26:15.422498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212659+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:26:15.212659+0000","last_peered":"2023-10-18T12:26:15.212659+0000","last_clean":"2023-10-18T12:26:15.212659+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:26:15.212659+0000","last_undegraded":"2023-10-18T12:26:15.212659+0000","last_fullsized":"2023-10-18T12:26:15.212659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215428+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:26:15.215428+0000","last_peered":"2023-10-18T12:26:15.215428+0000","last_clean":"2023-10-18T12:26:15.215428+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:26:15.215428+0000","last_undegraded":"2023-10-18T12:26:15.215428+0000","last_fullsized":"2023-10-18T12:26:15.215428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.654415+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:43.654415+0000","last_peered":"2023-10-18T12:25:43.654415+0000","last_clean":"2023-10-18T12:25:43.654415+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:43.654415+0000","last_undegraded":"2023-10-18T12:25:43.654415+0000","last_fullsized":"2023-10-18T12:25:43.654415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213505+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:26:15.213505+0000","last_peered":"2023-10-18T12:26:15.213505+0000","last_clean":"2023-10-18T12:26:15.213505+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:26:15.213505+0000","last_undegraded":"2023-10-18T12:26:15.213505+0000","last_fullsized":"2023-10-18T12:26:15.213505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210457+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:15.210457+0000","last_peered":"2023-10-18T12:25:15.210457+0000","last_clean":"2023-10-18T12:25:15.210457+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:15.210457+0000","last_undegraded":"2023-10-18T12:25:15.210457+0000","last_fullsized":"2023-10-18T12:25:15.210457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642483+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:43.642483+0000","last_peered":"2023-10-18T12:25:43.642483+0000","last_clean":"2023-10-18T12:25:43.642483+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:43.642483+0000","last_undegraded":"2023-10-18T12:25:43.642483+0000","last_fullsized":"2023-10-18T12:25:43.642483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422704+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:26:15.422704+0000","last_peered":"2023-10-18T12:26:15.422704+0000","last_clean":"2023-10-18T12:26:15.422704+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:26:15.422704+0000","last_undegraded":"2023-10-18T12:26:15.422704+0000","last_fullsized":"2023-10-18T12:26:15.422704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804857+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:26:15.804857+0000","last_peered":"2023-10-18T12:26:15.804857+0000","last_clean":"2023-10-18T12:26:15.804857+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:26:15.804857+0000","last_undegraded":"2023-10-18T12:26:15.804857+0000","last_fullsized":"2023-10-18T12:26:15.804857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214502+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:26:15.214502+0000","last_peered":"2023-10-18T12:26:15.214502+0000","last_clean":"2023-10-18T12:26:15.214502+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:26:15.214502+0000","last_undegraded":"2023-10-18T12:26:15.214502+0000","last_fullsized":"2023-10-18T12:26:15.214502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214483+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:26:15.214483+0000","last_peered":"2023-10-18T12:26:15.214483+0000","last_clean":"2023-10-18T12:26:15.214483+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:26:15.214483+0000","last_undegraded":"2023-10-18T12:26:15.214483+0000","last_fullsized":"2023-10-18T12:26:15.214483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213136+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:26:15.213136+0000","last_peered":"2023-10-18T12:26:15.213136+0000","last_clean":"2023-10-18T12:26:15.213136+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:26:15.213136+0000","last_undegraded":"2023-10-18T12:26:15.213136+0000","last_fullsized":"2023-10-18T12:26:15.213136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213093+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:26:15.213093+0000","last_peered":"2023-10-18T12:26:15.213093+0000","last_clean":"2023-10-18T12:26:15.213093+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:26:15.213093+0000","last_undegraded":"2023-10-18T12:26:15.213093+0000","last_fullsized":"2023-10-18T12:26:15.213093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.215978+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:26:16.215978+0000","last_peered":"2023-10-18T12:26:16.215978+0000","last_clean":"2023-10-18T12:26:16.215978+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:26:16.215978+0000","last_undegraded":"2023-10-18T12:26:16.215978+0000","last_fullsized":"2023-10-18T12:26:16.215978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.661271+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:26:15.661271+0000","last_peered":"2023-10-18T12:26:15.661271+0000","last_clean":"2023-10-18T12:26:15.661271+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:26:15.661271+0000","last_undegraded":"2023-10-18T12:26:15.661271+0000","last_fullsized":"2023-10-18T12:26:15.661271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.336032+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:26:16.336032+0000","last_peered":"2023-10-18T12:26:16.336032+0000","last_clean":"2023-10-18T12:26:16.336032+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:26:16.336032+0000","last_undegraded":"2023-10-18T12:26:16.336032+0000","last_fullsized":"2023-10-18T12:26:16.336032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422601+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:26:15.422601+0000","last_peered":"2023-10-18T12:26:15.422601+0000","last_clean":"2023-10-18T12:26:15.422601+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:26:15.422601+0000","last_undegraded":"2023-10-18T12:26:15.422601+0000","last_fullsized":"2023-10-18T12:26:15.422601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:14.772089+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:26:14.772089+0000","last_peered":"2023-10-18T12:26:14.772089+0000","last_clean":"2023-10-18T12:26:14.772089+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:26:14.772089+0000","last_undegraded":"2023-10-18T12:26:14.772089+0000","last_fullsized":"2023-10-18T12:26:14.772089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.335890+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:26:16.335890+0000","last_peered":"2023-10-18T12:26:16.335890+0000","last_clean":"2023-10-18T12:26:16.335890+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:26:16.335890+0000","last_undegraded":"2023-10-18T12:26:16.335890+0000","last_fullsized":"2023-10-18T12:26:16.335890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422708+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:26:15.422708+0000","last_peered":"2023-10-18T12:26:15.422708+0000","last_clean":"2023-10-18T12:26:15.422708+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:26:15.422708+0000","last_undegraded":"2023-10-18T12:26:15.422708+0000","last_fullsized":"2023-10-18T12:26:15.422708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214953+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:26:15.214953+0000","last_peered":"2023-10-18T12:26:15.214953+0000","last_clean":"2023-10-18T12:26:15.214953+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:26:15.214953+0000","last_undegraded":"2023-10-18T12:26:15.214953+0000","last_fullsized":"2023-10-18T12:26:15.214953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422171+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:26:15.422171+0000","last_peered":"2023-10-18T12:26:15.422171+0000","last_clean":"2023-10-18T12:26:15.422171+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:26:15.422171+0000","last_undegraded":"2023-10-18T12:26:15.422171+0000","last_fullsized":"2023-10-18T12:26:15.422171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.696371+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:26:16.696371+0000","last_peered":"2023-10-18T12:26:16.696371+0000","last_clean":"2023-10-18T12:26:16.696371+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:26:16.696371+0000","last_undegraded":"2023-10-18T12:26:16.696371+0000","last_fullsized":"2023-10-18T12:26:16.696371+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.219876+0000","last_change":"2023-10-18T12:26:16.219876+0000","last_active":"2023-10-18T12:26:16.219876+0000","last_peered":"2023-10-18T12:26:16.219876+0000","last_clean":"2023-10-18T12:26:16.219876+0000","last_became_active":"2023-10-18T12:26:16.219515+0000","last_became_peered":"2023-10-18T12:26:16.219515+0000","last_unstale":"2023-10-18T12:26:16.219876+0000","last_undegraded":"2023-10-18T12:26:16.219876+0000","last_fullsized":"2023-10-18T12:26:16.219876+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:55.404152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.216871+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:26:16.216871+0000","last_peered":"2023-10-18T12:26:16.216871+0000","last_clean":"2023-10-18T12:26:16.216871+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:26:16.216871+0000","last_undegraded":"2023-10-18T12:26:16.216871+0000","last_fullsized":"2023-10-18T12:26:16.216871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":328,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:59.770287+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:59.770287+0000","last_peered":"2023-10-18T12:25:59.770287+0000","last_clean":"2023-10-18T12:25:59.770287+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:59.770287+0000","last_undegraded":"2023-10-18T12:25:59.770287+0000","last_fullsized":"2023-10-18T12:25:59.770287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215078+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:26:15.215078+0000","last_peered":"2023-10-18T12:26:15.215078+0000","last_clean":"2023-10-18T12:26:15.215078+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:26:15.215078+0000","last_undegraded":"2023-10-18T12:26:15.215078+0000","last_fullsized":"2023-10-18T12:26:15.215078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:17.196681+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:26:17.196681+0000","last_peered":"2023-10-18T12:26:17.196681+0000","last_clean":"2023-10-18T12:26:17.196681+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:26:17.196681+0000","last_undegraded":"2023-10-18T12:26:17.196681+0000","last_fullsized":"2023-10-18T12:26:17.196681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212471+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:26:15.212471+0000","last_peered":"2023-10-18T12:26:15.212471+0000","last_clean":"2023-10-18T12:26:15.212471+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:26:15.212471+0000","last_undegraded":"2023-10-18T12:26:15.212471+0000","last_fullsized":"2023-10-18T12:26:15.212471+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212735+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:26:15.212735+0000","last_peered":"2023-10-18T12:26:15.212735+0000","last_clean":"2023-10-18T12:26:15.212735+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:26:15.212735+0000","last_undegraded":"2023-10-18T12:26:15.212735+0000","last_fullsized":"2023-10-18T12:26:15.212735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805277+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:26:15.805277+0000","last_peered":"2023-10-18T12:26:15.805277+0000","last_clean":"2023-10-18T12:26:15.805277+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:26:15.805277+0000","last_undegraded":"2023-10-18T12:26:15.805277+0000","last_fullsized":"2023-10-18T12:26:15.805277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:17.196955+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:26:17.196955+0000","last_peered":"2023-10-18T12:26:17.196955+0000","last_clean":"2023-10-18T12:26:17.196955+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:26:17.196955+0000","last_undegraded":"2023-10-18T12:26:17.196955+0000","last_fullsized":"2023-10-18T12:26:17.196955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210022+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:26:15.210022+0000","last_peered":"2023-10-18T12:26:15.210022+0000","last_clean":"2023-10-18T12:26:15.210022+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:26:15.210022+0000","last_undegraded":"2023-10-18T12:26:15.210022+0000","last_fullsized":"2023-10-18T12:26:15.210022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.216330+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:26:16.216330+0000","last_peered":"2023-10-18T12:26:16.216330+0000","last_clean":"2023-10-18T12:26:16.216330+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:26:16.216330+0000","last_undegraded":"2023-10-18T12:26:16.216330+0000","last_fullsized":"2023-10-18T12:26:16.216330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.635861+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:26:16.635861+0000","last_peered":"2023-10-18T12:26:16.635861+0000","last_clean":"2023-10-18T12:26:16.635861+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:26:16.635861+0000","last_undegraded":"2023-10-18T12:26:16.635861+0000","last_fullsized":"2023-10-18T12:26:16.635861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.221950+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:26:16.221950+0000","last_peered":"2023-10-18T12:26:16.221950+0000","last_clean":"2023-10-18T12:26:16.221950+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:26:16.221950+0000","last_undegraded":"2023-10-18T12:26:16.221950+0000","last_fullsized":"2023-10-18T12:26:16.221950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.211145+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:15.211145+0000","last_peered":"2023-10-18T12:25:15.211145+0000","last_clean":"2023-10-18T12:25:15.211145+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:15.211145+0000","last_undegraded":"2023-10-18T12:25:15.211145+0000","last_fullsized":"2023-10-18T12:25:15.211145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:17.326996+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:26:17.326996+0000","last_peered":"2023-10-18T12:26:17.326996+0000","last_clean":"2023-10-18T12:26:17.326996+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:26:17.326996+0000","last_undegraded":"2023-10-18T12:26:17.326996+0000","last_fullsized":"2023-10-18T12:26:17.326996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223680+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223680+0000","last_peered":"2023-10-18T12:25:15.223680+0000","last_clean":"2023-10-18T12:25:15.223680+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223680+0000","last_undegraded":"2023-10-18T12:25:15.223680+0000","last_fullsized":"2023-10-18T12:25:15.223680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.218389+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:26:16.218389+0000","last_peered":"2023-10-18T12:26:16.218389+0000","last_clean":"2023-10-18T12:26:16.218389+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:26:16.218389+0000","last_undegraded":"2023-10-18T12:26:16.218389+0000","last_fullsized":"2023-10-18T12:26:16.218389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.215824+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:26:16.215824+0000","last_peered":"2023-10-18T12:26:16.215824+0000","last_clean":"2023-10-18T12:26:16.215824+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:26:16.215824+0000","last_undegraded":"2023-10-18T12:26:16.215824+0000","last_fullsized":"2023-10-18T12:26:16.215824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805400+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:26:15.805400+0000","last_peered":"2023-10-18T12:26:15.805400+0000","last_clean":"2023-10-18T12:26:15.805400+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:26:15.805400+0000","last_undegraded":"2023-10-18T12:26:15.805400+0000","last_fullsized":"2023-10-18T12:26:15.805400+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.635113+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:26:16.635113+0000","last_peered":"2023-10-18T12:26:16.635113+0000","last_clean":"2023-10-18T12:26:16.635113+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:26:16.635113+0000","last_undegraded":"2023-10-18T12:26:16.635113+0000","last_fullsized":"2023-10-18T12:26:16.635113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212690+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:26:15.212690+0000","last_peered":"2023-10-18T12:26:15.212690+0000","last_clean":"2023-10-18T12:26:15.212690+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:26:15.212690+0000","last_undegraded":"2023-10-18T12:26:15.212690+0000","last_fullsized":"2023-10-18T12:26:15.212690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210778+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:15.210778+0000","last_peered":"2023-10-18T12:25:15.210778+0000","last_clean":"2023-10-18T12:25:15.210778+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:15.210778+0000","last_undegraded":"2023-10-18T12:25:15.210778+0000","last_fullsized":"2023-10-18T12:25:15.210778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.222322+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:26:16.222322+0000","last_peered":"2023-10-18T12:26:16.222322+0000","last_clean":"2023-10-18T12:26:16.222322+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:26:16.222322+0000","last_undegraded":"2023-10-18T12:26:16.222322+0000","last_fullsized":"2023-10-18T12:26:16.222322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210503+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:26:15.210503+0000","last_peered":"2023-10-18T12:26:15.210503+0000","last_clean":"2023-10-18T12:26:15.210503+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:26:15.210503+0000","last_undegraded":"2023-10-18T12:26:15.210503+0000","last_fullsized":"2023-10-18T12:26:15.210503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210651+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:15.210651+0000","last_peered":"2023-10-18T12:25:15.210651+0000","last_clean":"2023-10-18T12:25:15.210651+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:15.210651+0000","last_undegraded":"2023-10-18T12:25:15.210651+0000","last_fullsized":"2023-10-18T12:25:15.210651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804975+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:26:15.804975+0000","last_peered":"2023-10-18T12:26:15.804975+0000","last_clean":"2023-10-18T12:26:15.804975+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:26:15.804975+0000","last_undegraded":"2023-10-18T12:26:15.804975+0000","last_fullsized":"2023-10-18T12:26:15.804975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.696098+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:26:16.696098+0000","last_peered":"2023-10-18T12:26:16.696098+0000","last_clean":"2023-10-18T12:26:16.696098+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:26:16.696098+0000","last_undegraded":"2023-10-18T12:26:16.696098+0000","last_fullsized":"2023-10-18T12:26:16.696098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214199+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:26:15.214199+0000","last_peered":"2023-10-18T12:26:15.214199+0000","last_clean":"2023-10-18T12:26:15.214199+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:26:15.214199+0000","last_undegraded":"2023-10-18T12:26:15.214199+0000","last_fullsized":"2023-10-18T12:26:15.214199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211510+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:26:15.211510+0000","last_peered":"2023-10-18T12:26:15.211510+0000","last_clean":"2023-10-18T12:26:15.211510+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:26:15.211510+0000","last_undegraded":"2023-10-18T12:26:15.211510+0000","last_fullsized":"2023-10-18T12:26:15.211510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644615+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:43.644615+0000","last_peered":"2023-10-18T12:25:43.644615+0000","last_clean":"2023-10-18T12:25:43.644615+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:43.644615+0000","last_undegraded":"2023-10-18T12:25:43.644615+0000","last_fullsized":"2023-10-18T12:25:43.644615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926059+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:26:15.926059+0000","last_peered":"2023-10-18T12:26:15.926059+0000","last_clean":"2023-10-18T12:26:15.926059+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:26:15.926059+0000","last_undegraded":"2023-10-18T12:26:15.926059+0000","last_fullsized":"2023-10-18T12:26:15.926059+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214930+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:26:15.214930+0000","last_peered":"2023-10-18T12:26:15.214930+0000","last_clean":"2023-10-18T12:26:15.214930+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:26:15.214930+0000","last_undegraded":"2023-10-18T12:26:15.214930+0000","last_fullsized":"2023-10-18T12:26:15.214930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639182+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:43.639182+0000","last_peered":"2023-10-18T12:25:43.639182+0000","last_clean":"2023-10-18T12:25:43.639182+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:43.639182+0000","last_undegraded":"2023-10-18T12:25:43.639182+0000","last_fullsized":"2023-10-18T12:25:43.639182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214280+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:26:15.214280+0000","last_peered":"2023-10-18T12:26:15.214280+0000","last_clean":"2023-10-18T12:26:15.214280+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:26:15.214280+0000","last_undegraded":"2023-10-18T12:26:15.214280+0000","last_fullsized":"2023-10-18T12:26:15.214280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422728+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:26:15.422728+0000","last_peered":"2023-10-18T12:26:15.422728+0000","last_clean":"2023-10-18T12:26:15.422728+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:26:15.422728+0000","last_undegraded":"2023-10-18T12:26:15.422728+0000","last_fullsized":"2023-10-18T12:26:15.422728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.640317+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:43.640317+0000","last_peered":"2023-10-18T12:25:43.640317+0000","last_clean":"2023-10-18T12:25:43.640317+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:43.640317+0000","last_undegraded":"2023-10-18T12:25:43.640317+0000","last_fullsized":"2023-10-18T12:25:43.640317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.223970+0000","last_change":"2023-10-18T12:26:16.223538+0000","last_active":"2023-10-18T12:26:16.223970+0000","last_peered":"2023-10-18T12:26:16.223970+0000","last_clean":"2023-10-18T12:26:16.223970+0000","last_became_active":"2023-10-18T12:26:16.223094+0000","last_became_peered":"2023-10-18T12:26:16.223094+0000","last_unstale":"2023-10-18T12:26:16.223970+0000","last_undegraded":"2023-10-18T12:26:16.223970+0000","last_fullsized":"2023-10-18T12:26:16.223970+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:23:31.237530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.222109+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:26:16.222109+0000","last_peered":"2023-10-18T12:26:16.222109+0000","last_clean":"2023-10-18T12:26:16.222109+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:26:16.222109+0000","last_undegraded":"2023-10-18T12:26:16.222109+0000","last_fullsized":"2023-10-18T12:26:16.222109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.222132+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:26:16.222132+0000","last_peered":"2023-10-18T12:26:16.222132+0000","last_clean":"2023-10-18T12:26:16.222132+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:26:16.222132+0000","last_undegraded":"2023-10-18T12:26:16.222132+0000","last_fullsized":"2023-10-18T12:26:16.222132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926231+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:26:15.926231+0000","last_peered":"2023-10-18T12:26:15.926231+0000","last_clean":"2023-10-18T12:26:15.926231+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:26:15.926231+0000","last_undegraded":"2023-10-18T12:26:15.926231+0000","last_fullsized":"2023-10-18T12:26:15.926231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422501+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:26:15.422501+0000","last_peered":"2023-10-18T12:26:15.422501+0000","last_clean":"2023-10-18T12:26:15.422501+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:26:15.422501+0000","last_undegraded":"2023-10-18T12:26:15.422501+0000","last_fullsized":"2023-10-18T12:26:15.422501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212928+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:26:15.212928+0000","last_peered":"2023-10-18T12:26:15.212928+0000","last_clean":"2023-10-18T12:26:15.212928+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:26:15.212928+0000","last_undegraded":"2023-10-18T12:26:15.212928+0000","last_fullsized":"2023-10-18T12:26:15.212928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211073+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:26:15.211073+0000","last_peered":"2023-10-18T12:26:15.211073+0000","last_clean":"2023-10-18T12:26:15.211073+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:26:15.211073+0000","last_undegraded":"2023-10-18T12:26:15.211073+0000","last_fullsized":"2023-10-18T12:26:15.211073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.635968+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:26:16.635968+0000","last_peered":"2023-10-18T12:26:16.635968+0000","last_clean":"2023-10-18T12:26:16.635968+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:26:16.635968+0000","last_undegraded":"2023-10-18T12:26:16.635968+0000","last_fullsized":"2023-10-18T12:26:16.635968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214790+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:26:15.214790+0000","last_peered":"2023-10-18T12:26:15.214790+0000","last_clean":"2023-10-18T12:26:15.214790+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:26:15.214790+0000","last_undegraded":"2023-10-18T12:26:15.214790+0000","last_fullsized":"2023-10-18T12:26:15.214790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210102+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:26:15.210102+0000","last_peered":"2023-10-18T12:26:15.210102+0000","last_clean":"2023-10-18T12:26:15.210102+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:26:15.210102+0000","last_undegraded":"2023-10-18T12:26:15.210102+0000","last_fullsized":"2023-10-18T12:26:15.210102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214765+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:26:15.214765+0000","last_peered":"2023-10-18T12:26:15.214765+0000","last_clean":"2023-10-18T12:26:15.214765+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:26:15.214765+0000","last_undegraded":"2023-10-18T12:26:15.214765+0000","last_fullsized":"2023-10-18T12:26:15.214765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422543+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:26:15.422543+0000","last_peered":"2023-10-18T12:26:15.422543+0000","last_clean":"2023-10-18T12:26:15.422543+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:26:15.422543+0000","last_undegraded":"2023-10-18T12:26:15.422543+0000","last_fullsized":"2023-10-18T12:26:15.422543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212687+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:26:15.212687+0000","last_peered":"2023-10-18T12:26:15.212687+0000","last_clean":"2023-10-18T12:26:15.212687+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:26:15.212687+0000","last_undegraded":"2023-10-18T12:26:15.212687+0000","last_fullsized":"2023-10-18T12:26:15.212687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67627,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":232,"num_read_kb":184,"num_write":82,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":202881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":532844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":1.147,"15min":1.147},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88400000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":1.238,"15min":1.238},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75600000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":1.661,"15min":1.661},"last":0.73499999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90400000000000003,"5min":1.361,"15min":1.361},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":1,"15min":1},"last":0.64300000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.622},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.752}]},{"osd":9,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.91100000000000003,"5min":1.726,"15min":1.726},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.012,"5min":1.401,"15min":1.401},"last":0.69999999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89000000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.76700000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1186117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.54500000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.497}]},{"osd":2,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63300000000000001,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.999,"15min":0.999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72499999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84599999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.876,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.77600000000000002,"5min":1.099,"15min":1.099},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82799999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.97499999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90400000000000003,"5min":1.071,"15min":1.071},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.044,"15min":1.044},"last":0.76400000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.91700000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.81299999999999994}]}]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":537500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0089999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.753},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.622}]},{"osd":3,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.753},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.86199999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1739999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1519999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.91500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.77300000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.94599999999999995,"15min":0.95199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":6.2220000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.83499999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.89100000000000001},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.875},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2430000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92700000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.92200000000000004,"15min":0.95199999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.995,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.97099999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":533404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.878}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.74099999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.89500000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76600000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0149999999999999,"5min":1.044,"15min":1.044},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.72399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0640000000000001,"5min":1.26,"15min":1.26},"last":0.95199999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.68300000000000005},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76800000000000002,"15min":0.72799999999999998},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.107,"15min":1.107},"last":1.0089999999999999}]},{"osd":11,"last update":"Wed Oct 18 12:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0209999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.98299999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1127868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.433,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.432},{"interface":"front","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.999,"15min":0.999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68400000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.137,"15min":1.137},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81299999999999994,"5min":1.071,"15min":1.071},"last":0.69799999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.65600000000000003}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.98199999999999998,"5min":1.02,"15min":1.02},"last":0.83599999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95399999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79700000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83399999999999996},"min":{"1min":0.64500000000000002,"5min":0.63,"15min":0.63},"max":{"1min":0.97099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.80400000000000005},"min":{"1min":0.63600000000000001,"5min":0.62,"15min":0.62},"max":{"1min":0.94599999999999995,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.67100000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":528726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.53000000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.377},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.499}]},{"osd":2,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.68799999999999994,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.9199999999999999,"5min":6.9199999999999999,"15min":6.9199999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.026,"5min":0.81899999999999995,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":7.2270000000000003,"5min":7.2270000000000003,"15min":7.2270000000000003},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.016,"5min":1.234,"15min":1.234},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.996}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74299999999999999,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.28,"5min":1.849,"15min":1.849},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.68000000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.88,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":7.7400000000000002,"5min":7.7400000000000002,"15min":7.7400000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.753}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.90800000000000003,"15min":0.88900000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.4969999999999999,"5min":7.4969999999999999,"15min":7.4969999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.66300000000000003}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95499999999999996,"15min":0.90300000000000002},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3240000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.90300000000000002}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":0.98299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":595555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.161,"15min":1.161},"last":0.624},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94699999999999995,"5min":1.454,"15min":1.454},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.57299999999999995},"min":{"1min":0.52000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88500000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71899999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76600000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.755},{"interface":"front","average":{"1min":0.749,"5min":0.68500000000000005,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.999,"5min":1.298,"15min":1.298},"last":0.875}]},{"osd":6,"last update":"Wed Oct 18 12:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82599999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.94599999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.98799999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.52700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89400000000000002,"5min":1.016,"15min":1.016},"last":0.95899999999999996}]},{"osd":11,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76900000000000002,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.91800000000000004,"15min":0.92500000000000004},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":1.0029999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":528816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.65200000000000002,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.30299999999999999},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65600000000000003,"15min":0.55200000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.32400000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.60699999999999998},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1479999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.024,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.65800000000000003,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.69699999999999995,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.497,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93700000000000006,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.879},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.66200000000000003},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.82399999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69899999999999995,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.751,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.96599999999999997,"15min":0.998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2949999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.874,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.044,"5min":3.222,"15min":3.222},"last":0.77400000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":1.2410000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86799999999999999,"15min":0.84599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.61199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.89300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.89300000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.03,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.97599999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93899999999999995,"15min":0.92400000000000004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.008},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":0.95899999999999996}]}]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":595316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.26100000000000001},"max":{"1min":0.92100000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69099999999999995}]},{"osd":3,"last update":"Wed Oct 18 12:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.54600000000000004},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.748}]},{"osd":5,"last update":"Wed Oct 18 12:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.38,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.876,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.52100000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.88800000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.58399999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79800000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.96499999999999997}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":1.0349999999999999}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76500000000000001,"15min":0.74099999999999999},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.254,"15min":1.254},"last":0.872}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.74299999999999999,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":1.079},{"interface":"front","average":{"1min":0.878,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0169999999999999}]},{"osd":11,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.64600000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.002},{"interface":"front","average":{"1min":0.877,"5min":0.80400000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.1060000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":528830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.51400000000000001,"15min":0.436},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7880000000000003,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.52700000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.73599999999999999,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.68999999999999995,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":5.0579999999999998,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.73699999999999999,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.747}]},{"osd":4,"last update":"Wed Oct 18 12:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.085,"5min":0.77800000000000002,"15min":0.70599999999999996},"min":{"1min":0.60999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.78700000000000003,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91700000000000004,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.752,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.87,"5min":3.806,"15min":3.806},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.504}]},{"osd":7,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.752,"15min":0.69499999999999995},"min":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.83299999999999996,"15min":0.753},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.984,"5min":4.984,"15min":4.984},"last":0.93999999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3789999999999996,"5min":5.3789999999999996,"15min":5.3789999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.86199999999999999},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":0.89400000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93799999999999994,"15min":0.90500000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.92000000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.016,"15min":1.052},"min":{"1min":0.61899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2030000000000001,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.111,"15min":1.155},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2170000000000001,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.95999999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54800000000000004,"5min":0.621,"15min":0.621},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57099999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.876,"5min":1.04,"15min":1.04},"last":0.64700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.52200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":1.22,"15min":1.22},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52800000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80200000000000005,"15min":0.81100000000000005},"min":{"1min":0.65900000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.02,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.55700000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.996,"5min":0.997,"15min":0.997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85599999999999998,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1123041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":0.999},"last":0.754},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63500000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.752},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":4.548,"15min":4.548},"last":0.57399999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.745,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73999999999999999,"15min":0.70799999999999996},"min":{"1min":0.60699999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:26:21.839 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-18T12:26:21.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:26:22.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/2293051244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 msgr2=0x7fcc18106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/2293051244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc18106a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009990 tx=0x7fcc14009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/2293051244 shutdown_connections 2023-10-18T12:26:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/2293051244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc18101060 0x7fcc1810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/2293051244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc18106a90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/2293051244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/2293051244 >> 172.21.15.70:0/2293051244 conn(0x7fcc180fc170 msgr2=0x7fcc180fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/2293051244 shutdown_connections 2023-10-18T12:26:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.064+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/2293051244 wait complete. 2023-10-18T12:26:22.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.065+0000 7fcc1f9ff700 1 Processor -- start 2023-10-18T12:26:22.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.065+0000 7fcc1f9ff700 1 -- start start 2023-10-18T12:26:22.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1f9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc18101060 0x7fcc18199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1f9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc1819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1f9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819cb30 con 0x7fcc181066b0 2023-10-18T12:26:22.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1f9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819cca0 con 0x7fcc18101060 2023-10-18T12:26:22.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819ce60 con 0x7fcc181006a0 2023-10-18T12:26:22.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1d79b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1d79b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57884/0 (socket says 172.21.15.70:57884) 2023-10-18T12:26:22.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1d79b700 1 -- 172.21.15.70:0/3266604183 learned_addr learned my addr 172.21.15.70:0/3266604183 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:22.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1df9c700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc1819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1cf9a700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc18101060 0x7fcc18199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1df9c700 1 -- 172.21.15.70:0/3266604183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 msgr2=0x7fcc18198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1df9c700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1df9c700 1 -- 172.21.15.70:0/3266604183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc18101060 msgr2=0x7fcc18199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1df9c700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc18101060 0x7fcc18199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1df9c700 1 -- 172.21.15.70:0/3266604183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140095f0 con 0x7fcc181066b0 2023-10-18T12:26:22.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.066+0000 7fcc1d79b700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:22.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.067+0000 7fcc1df9c700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc1819c530 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000efc0 tx=0x7fcc1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:22.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.067+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10018070 con 0x7fcc181066b0 2023-10-18T12:26:22.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.067+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1000f040 con 0x7fcc181066b0 2023-10-18T12:26:22.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.067+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000cd40 con 0x7fcc181066b0 2023-10-18T12:26:22.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.067+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1819d0e0 con 0x7fcc181066b0 2023-10-18T12:26:22.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.067+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc181a5aa0 con 0x7fcc181066b0 2023-10-18T12:26:22.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.068+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18101de0 con 0x7fcc181066b0 2023-10-18T12:26:22.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.070+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcc10003a00 con 0x7fcc181066b0 2023-10-18T12:26:22.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.071+0000 7fcc0e7fc700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc040a1550 0x7fcc040a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.071+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcc100ccc30 con 0x7fcc181066b0 2023-10-18T12:26:22.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.072+0000 7fcc1d79b700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc040a1550 0x7fcc040a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.072+0000 7fcc1d79b700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc040a1550 0x7fcc040a3a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006010 tx=0x7fcc1400b280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:22.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.075+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc10095f00 con 0x7fcc181066b0 2023-10-18T12:26:22.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:21 smithi070 ceph-mon[98639]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-18T12:26:22.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:21 smithi070 ceph-mon[98639]: Cluster is now healthy 2023-10-18T12:26:22.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:21 smithi152 ceph-mon[134767]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-18T12:26:22.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:21 smithi152 ceph-mon[134767]: Cluster is now healthy 2023-10-18T12:26:22.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:21 smithi186 ceph-mon[135253]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-18T12:26:22.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:21 smithi186 ceph-mon[135253]: Cluster is now healthy 2023-10-18T12:26:22.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.279+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc18110620 con 0x7fcc040a1550 2023-10-18T12:26:22.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.292+0000 7fcc0e7fc700 1 -- 172.21.15.70:0/3266604183 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427303 (secure 0 0 0) 0x7fcc18110620 con 0x7fcc040a1550 2023-10-18T12:26:22.304 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc040a1550 msgr2=0x7fcc040a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc040a1550 0x7fcc040a3a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006010 tx=0x7fcc1400b280 comp rx=0 tx=0).stop 2023-10-18T12:26:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 msgr2=0x7fcc1819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc1819c530 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000efc0 tx=0x7fcc1000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 shutdown_connections 2023-10-18T12:26:22.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc181066b0 0x7fcc1819c530 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc181006a0 0x7fcc18198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc040a1550 0x7fcc040a3a10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.297+0000 7fcc1f9ff700 1 --2- 172.21.15.70:0/3266604183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc18101060 0x7fcc18199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.298+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 >> 172.21.15.70:0/3266604183 conn(0x7fcc180fc170 msgr2=0x7fcc1810bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:22.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.298+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 shutdown_connections 2023-10-18T12:26:22.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.298+0000 7fcc1f9ff700 1 -- 172.21.15.70:0/3266604183 wait complete. 2023-10-18T12:26:22.311 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-18T12:26:22.320 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2023-10-18T12:26:21.626233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658241,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":368,"num_read_kb":303,"num_write":265,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":711,"ondisk_log_size":711,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338628,"kb_used_data":14356,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603196,"statfs":{"total":1151940427776,"available":1151593672704,"internally_reserved":0,"allocated":14700544,"data_stored":8331163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5379,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":16,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002265"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.220660+0000","last_change":"2023-10-18T12:24:09.983263+0000","last_active":"2023-10-18T12:26:16.220660+0000","last_peered":"2023-10-18T12:26:16.220660+0000","last_clean":"2023-10-18T12:26:16.220660+0000","last_became_active":"2023-10-18T12:24:09.983050+0000","last_became_peered":"2023-10-18T12:24:09.983050+0000","last_unstale":"2023-10-18T12:26:16.220660+0000","last_undegraded":"2023-10-18T12:26:16.220660+0000","last_fullsized":"2023-10-18T12:26:16.220660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.691071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926320+0000","last_change":"2023-10-18T12:24:09.974705+0000","last_active":"2023-10-18T12:26:15.926320+0000","last_peered":"2023-10-18T12:26:15.926320+0000","last_clean":"2023-10-18T12:26:15.926320+0000","last_became_active":"2023-10-18T12:24:09.974447+0000","last_became_peered":"2023-10-18T12:24:09.974447+0000","last_unstale":"2023-10-18T12:26:15.926320+0000","last_undegraded":"2023-10-18T12:26:15.926320+0000","last_fullsized":"2023-10-18T12:26:15.926320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:51.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926264+0000","last_change":"2023-10-18T12:24:09.978814+0000","last_active":"2023-10-18T12:26:15.926264+0000","last_peered":"2023-10-18T12:26:15.926264+0000","last_clean":"2023-10-18T12:26:15.926264+0000","last_became_active":"2023-10-18T12:24:09.977349+0000","last_became_peered":"2023-10-18T12:24:09.977349+0000","last_unstale":"2023-10-18T12:26:15.926264+0000","last_undegraded":"2023-10-18T12:26:15.926264+0000","last_fullsized":"2023-10-18T12:26:15.926264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:08.167353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:11.635030+0000","last_change":"2023-10-18T12:24:09.970460+0000","last_active":"2023-10-18T12:26:11.635030+0000","last_peered":"2023-10-18T12:26:11.635030+0000","last_clean":"2023-10-18T12:26:11.635030+0000","last_became_active":"2023-10-18T12:24:09.970082+0000","last_became_peered":"2023-10-18T12:24:09.970082+0000","last_unstale":"2023-10-18T12:26:11.635030+0000","last_undegraded":"2023-10-18T12:26:11.635030+0000","last_fullsized":"2023-10-18T12:26:11.635030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:15.557147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804954+0000","last_change":"2023-10-18T12:24:09.975054+0000","last_active":"2023-10-18T12:26:15.804954+0000","last_peered":"2023-10-18T12:26:15.804954+0000","last_clean":"2023-10-18T12:26:15.804954+0000","last_became_active":"2023-10-18T12:24:09.974723+0000","last_became_peered":"2023-10-18T12:24:09.974723+0000","last_unstale":"2023-10-18T12:26:15.804954+0000","last_undegraded":"2023-10-18T12:26:15.804954+0000","last_fullsized":"2023-10-18T12:26:15.804954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210846+0000","last_change":"2023-10-18T12:24:09.974646+0000","last_active":"2023-10-18T12:26:15.210846+0000","last_peered":"2023-10-18T12:26:15.210846+0000","last_clean":"2023-10-18T12:26:15.210846+0000","last_became_active":"2023-10-18T12:24:09.974323+0000","last_became_peered":"2023-10-18T12:24:09.974323+0000","last_unstale":"2023-10-18T12:26:15.210846+0000","last_undegraded":"2023-10-18T12:26:15.210846+0000","last_fullsized":"2023-10-18T12:26:15.210846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:05.571993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639977+0000","last_change":"2023-10-18T12:24:09.980754+0000","last_active":"2023-10-18T12:25:43.639977+0000","last_peered":"2023-10-18T12:25:43.639977+0000","last_clean":"2023-10-18T12:25:43.639977+0000","last_became_active":"2023-10-18T12:24:09.980434+0000","last_became_peered":"2023-10-18T12:24:09.980434+0000","last_unstale":"2023-10-18T12:25:43.639977+0000","last_undegraded":"2023-10-18T12:25:43.639977+0000","last_fullsized":"2023-10-18T12:25:43.639977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:26.748151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210117+0000","last_change":"2023-10-18T12:24:09.984230+0000","last_active":"2023-10-18T12:26:15.210117+0000","last_peered":"2023-10-18T12:26:15.210117+0000","last_clean":"2023-10-18T12:26:15.210117+0000","last_became_active":"2023-10-18T12:24:09.976197+0000","last_became_peered":"2023-10-18T12:24:09.976197+0000","last_unstale":"2023-10-18T12:26:15.210117+0000","last_undegraded":"2023-10-18T12:26:15.210117+0000","last_fullsized":"2023-10-18T12:26:15.210117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:06.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214861+0000","last_change":"2023-10-18T12:24:09.967098+0000","last_active":"2023-10-18T12:26:15.214861+0000","last_peered":"2023-10-18T12:26:15.214861+0000","last_clean":"2023-10-18T12:26:15.214861+0000","last_became_active":"2023-10-18T12:24:09.966794+0000","last_became_peered":"2023-10-18T12:24:09.966794+0000","last_unstale":"2023-10-18T12:26:15.214861+0000","last_undegraded":"2023-10-18T12:26:15.214861+0000","last_fullsized":"2023-10-18T12:26:15.214861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:01.853393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210940+0000","last_change":"2023-10-18T12:24:09.977743+0000","last_active":"2023-10-18T12:26:15.210940+0000","last_peered":"2023-10-18T12:26:15.210940+0000","last_clean":"2023-10-18T12:26:15.210940+0000","last_became_active":"2023-10-18T12:24:09.977425+0000","last_became_peered":"2023-10-18T12:24:09.977425+0000","last_unstale":"2023-10-18T12:26:15.210940+0000","last_undegraded":"2023-10-18T12:26:15.210940+0000","last_fullsized":"2023-10-18T12:26:15.210940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:18.725543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644437+0000","last_change":"2023-10-18T12:24:09.967499+0000","last_active":"2023-10-18T12:25:43.644437+0000","last_peered":"2023-10-18T12:25:43.644437+0000","last_clean":"2023-10-18T12:25:43.644437+0000","last_became_active":"2023-10-18T12:24:09.967178+0000","last_became_peered":"2023-10-18T12:24:09.967178+0000","last_unstale":"2023-10-18T12:25:43.644437+0000","last_undegraded":"2023-10-18T12:25:43.644437+0000","last_fullsized":"2023-10-18T12:25:43.644437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.560208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.633660+0000","last_change":"2023-10-18T12:24:09.972966+0000","last_active":"2023-10-18T12:25:43.633660+0000","last_peered":"2023-10-18T12:25:43.633660+0000","last_clean":"2023-10-18T12:25:43.633660+0000","last_became_active":"2023-10-18T12:24:09.972383+0000","last_became_peered":"2023-10-18T12:24:09.972383+0000","last_unstale":"2023-10-18T12:25:43.633660+0000","last_undegraded":"2023-10-18T12:25:43.633660+0000","last_fullsized":"2023-10-18T12:25:43.633660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:37.377454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422485+0000","last_change":"2023-10-18T12:24:09.972685+0000","last_active":"2023-10-18T12:26:15.422485+0000","last_peered":"2023-10-18T12:26:15.422485+0000","last_clean":"2023-10-18T12:26:15.422485+0000","last_became_active":"2023-10-18T12:24:09.972102+0000","last_became_peered":"2023-10-18T12:24:09.972102+0000","last_unstale":"2023-10-18T12:26:15.422485+0000","last_undegraded":"2023-10-18T12:26:15.422485+0000","last_fullsized":"2023-10-18T12:26:15.422485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:36.794419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.925939+0000","last_change":"2023-10-18T12:24:09.981470+0000","last_active":"2023-10-18T12:26:15.925939+0000","last_peered":"2023-10-18T12:26:15.925939+0000","last_clean":"2023-10-18T12:26:15.925939+0000","last_became_active":"2023-10-18T12:24:09.974080+0000","last_became_peered":"2023-10-18T12:24:09.974080+0000","last_unstale":"2023-10-18T12:26:15.925939+0000","last_undegraded":"2023-10-18T12:26:15.925939+0000","last_fullsized":"2023-10-18T12:26:15.925939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:47.097371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.925933+0000","last_change":"2023-10-18T12:24:09.978404+0000","last_active":"2023-10-18T12:26:15.925933+0000","last_peered":"2023-10-18T12:26:15.925933+0000","last_clean":"2023-10-18T12:26:15.925933+0000","last_became_active":"2023-10-18T12:24:09.978001+0000","last_became_peered":"2023-10-18T12:24:09.978001+0000","last_unstale":"2023-10-18T12:26:15.925933+0000","last_undegraded":"2023-10-18T12:26:15.925933+0000","last_fullsized":"2023-10-18T12:26:15.925933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:05.200276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214139+0000","last_change":"2023-10-18T12:24:09.976254+0000","last_active":"2023-10-18T12:26:15.214139+0000","last_peered":"2023-10-18T12:26:15.214139+0000","last_clean":"2023-10-18T12:26:15.214139+0000","last_became_active":"2023-10-18T12:24:09.975942+0000","last_became_peered":"2023-10-18T12:24:09.975942+0000","last_unstale":"2023-10-18T12:26:15.214139+0000","last_undegraded":"2023-10-18T12:26:15.214139+0000","last_fullsized":"2023-10-18T12:26:15.214139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:16.468043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213393+0000","last_change":"2023-10-18T12:24:09.983287+0000","last_active":"2023-10-18T12:26:15.213393+0000","last_peered":"2023-10-18T12:26:15.213393+0000","last_clean":"2023-10-18T12:26:15.213393+0000","last_became_active":"2023-10-18T12:24:09.983048+0000","last_became_peered":"2023-10-18T12:24:09.983048+0000","last_unstale":"2023-10-18T12:26:15.213393+0000","last_undegraded":"2023-10-18T12:26:15.213393+0000","last_fullsized":"2023-10-18T12:26:15.213393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:18.095571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213495+0000","last_change":"2023-10-18T12:24:09.969283+0000","last_active":"2023-10-18T12:26:15.213495+0000","last_peered":"2023-10-18T12:26:15.213495+0000","last_clean":"2023-10-18T12:26:15.213495+0000","last_became_active":"2023-10-18T12:24:09.968885+0000","last_became_peered":"2023-10-18T12:24:09.968885+0000","last_unstale":"2023-10-18T12:26:15.213495+0000","last_undegraded":"2023-10-18T12:26:15.213495+0000","last_fullsized":"2023-10-18T12:26:15.213495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:00.958534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211068+0000","last_change":"2023-10-18T12:24:09.976018+0000","last_active":"2023-10-18T12:26:15.211068+0000","last_peered":"2023-10-18T12:26:15.211068+0000","last_clean":"2023-10-18T12:26:15.211068+0000","last_became_active":"2023-10-18T12:24:09.975696+0000","last_became_peered":"2023-10-18T12:24:09.975696+0000","last_unstale":"2023-10-18T12:26:15.211068+0000","last_undegraded":"2023-10-18T12:26:15.211068+0000","last_fullsized":"2023-10-18T12:26:15.211068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:31.073942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210779+0000","last_change":"2023-10-18T12:24:14.160808+0000","last_active":"2023-10-18T12:26:15.210779+0000","last_peered":"2023-10-18T12:26:15.210779+0000","last_clean":"2023-10-18T12:26:15.210779+0000","last_became_active":"2023-10-18T12:24:14.160405+0000","last_became_peered":"2023-10-18T12:24:14.160405+0000","last_unstale":"2023-10-18T12:26:15.210779+0000","last_undegraded":"2023-10-18T12:26:15.210779+0000","last_fullsized":"2023-10-18T12:26:15.210779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:30.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.661330+0000","last_change":"2023-10-18T12:24:08.969807+0000","last_active":"2023-10-18T12:26:15.661330+0000","last_peered":"2023-10-18T12:26:15.661330+0000","last_clean":"2023-10-18T12:26:15.661330+0000","last_became_active":"2023-10-18T12:24:08.969445+0000","last_became_peered":"2023-10-18T12:24:08.969445+0000","last_unstale":"2023-10-18T12:26:15.661330+0000","last_undegraded":"2023-10-18T12:26:15.661330+0000","last_fullsized":"2023-10-18T12:26:15.661330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:16:50.325929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210706+0000","last_change":"2023-10-18T12:24:09.984308+0000","last_active":"2023-10-18T12:26:15.210706+0000","last_peered":"2023-10-18T12:26:15.210706+0000","last_clean":"2023-10-18T12:26:15.210706+0000","last_became_active":"2023-10-18T12:24:09.973555+0000","last_became_peered":"2023-10-18T12:24:09.973555+0000","last_unstale":"2023-10-18T12:26:15.210706+0000","last_undegraded":"2023-10-18T12:26:15.210706+0000","last_fullsized":"2023-10-18T12:26:15.210706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:57.443931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210351+0000","last_change":"2023-10-18T12:24:14.158760+0000","last_active":"2023-10-18T12:25:15.210351+0000","last_peered":"2023-10-18T12:25:15.210351+0000","last_clean":"2023-10-18T12:25:15.210351+0000","last_became_active":"2023-10-18T12:24:14.158269+0000","last_became_peered":"2023-10-18T12:24:14.158269+0000","last_unstale":"2023-10-18T12:25:15.210351+0000","last_undegraded":"2023-10-18T12:25:15.210351+0000","last_fullsized":"2023-10-18T12:25:15.210351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:46.782293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210836+0000","last_change":"2023-10-18T12:24:08.975676+0000","last_active":"2023-10-18T12:26:15.210836+0000","last_peered":"2023-10-18T12:26:15.210836+0000","last_clean":"2023-10-18T12:26:15.210836+0000","last_became_active":"2023-10-18T12:24:08.975332+0000","last_became_peered":"2023-10-18T12:24:08.975332+0000","last_unstale":"2023-10-18T12:26:15.210836+0000","last_undegraded":"2023-10-18T12:26:15.210836+0000","last_fullsized":"2023-10-18T12:26:15.210836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:59.229697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926330+0000","last_change":"2023-10-18T12:24:09.976371+0000","last_active":"2023-10-18T12:26:15.926330+0000","last_peered":"2023-10-18T12:26:15.926330+0000","last_clean":"2023-10-18T12:26:15.926330+0000","last_became_active":"2023-10-18T12:24:09.976050+0000","last_became_peered":"2023-10-18T12:24:09.976050+0000","last_unstale":"2023-10-18T12:26:15.926330+0000","last_undegraded":"2023-10-18T12:26:15.926330+0000","last_fullsized":"2023-10-18T12:26:15.926330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:55.860696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210533+0000","last_change":"2023-10-18T12:24:14.158438+0000","last_active":"2023-10-18T12:25:15.210533+0000","last_peered":"2023-10-18T12:25:15.210533+0000","last_clean":"2023-10-18T12:25:15.210533+0000","last_became_active":"2023-10-18T12:24:14.157934+0000","last_became_peered":"2023-10-18T12:24:14.157934+0000","last_unstale":"2023-10-18T12:25:15.210533+0000","last_undegraded":"2023-10-18T12:25:15.210533+0000","last_fullsized":"2023-10-18T12:25:15.210533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:22:35.019457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805155+0000","last_change":"2023-10-18T12:24:08.968109+0000","last_active":"2023-10-18T12:26:15.805155+0000","last_peered":"2023-10-18T12:26:15.805155+0000","last_clean":"2023-10-18T12:26:15.805155+0000","last_became_active":"2023-10-18T12:24:08.967812+0000","last_became_peered":"2023-10-18T12:24:08.967812+0000","last_unstale":"2023-10-18T12:26:15.805155+0000","last_undegraded":"2023-10-18T12:26:15.805155+0000","last_fullsized":"2023-10-18T12:26:15.805155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:41.799709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214360+0000","last_change":"2023-10-18T12:24:09.969270+0000","last_active":"2023-10-18T12:26:15.214360+0000","last_peered":"2023-10-18T12:26:15.214360+0000","last_clean":"2023-10-18T12:26:15.214360+0000","last_became_active":"2023-10-18T12:24:09.968979+0000","last_became_peered":"2023-10-18T12:24:09.968979+0000","last_unstale":"2023-10-18T12:26:15.214360+0000","last_undegraded":"2023-10-18T12:26:15.214360+0000","last_fullsized":"2023-10-18T12:26:15.214360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:43.968911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210967+0000","last_change":"2023-10-18T12:24:14.154562+0000","last_active":"2023-10-18T12:25:15.210967+0000","last_peered":"2023-10-18T12:25:15.210967+0000","last_clean":"2023-10-18T12:25:15.210967+0000","last_became_active":"2023-10-18T12:24:14.154088+0000","last_became_peered":"2023-10-18T12:24:14.154088+0000","last_unstale":"2023-10-18T12:25:15.210967+0000","last_undegraded":"2023-10-18T12:25:15.210967+0000","last_fullsized":"2023-10-18T12:25:15.210967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:05:32.885391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805304+0000","last_change":"2023-10-18T12:24:08.966237+0000","last_active":"2023-10-18T12:26:15.805304+0000","last_peered":"2023-10-18T12:26:15.805304+0000","last_clean":"2023-10-18T12:26:15.805304+0000","last_became_active":"2023-10-18T12:24:08.965932+0000","last_became_peered":"2023-10-18T12:24:08.965932+0000","last_unstale":"2023-10-18T12:26:15.805304+0000","last_undegraded":"2023-10-18T12:26:15.805304+0000","last_fullsized":"2023-10-18T12:26:15.805304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:56:02.071789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214868+0000","last_change":"2023-10-18T12:24:09.974420+0000","last_active":"2023-10-18T12:26:15.214868+0000","last_peered":"2023-10-18T12:26:15.214868+0000","last_clean":"2023-10-18T12:26:15.214868+0000","last_became_active":"2023-10-18T12:24:09.974119+0000","last_became_peered":"2023-10-18T12:24:09.974119+0000","last_unstale":"2023-10-18T12:26:15.214868+0000","last_undegraded":"2023-10-18T12:26:15.214868+0000","last_fullsized":"2023-10-18T12:26:15.214868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:13.189509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210854+0000","last_change":"2023-10-18T12:25:15.217322+0000","last_active":"2023-10-18T12:26:15.210854+0000","last_peered":"2023-10-18T12:26:15.210854+0000","last_clean":"2023-10-18T12:26:15.210854+0000","last_became_active":"2023-10-18T12:25:15.216900+0000","last_became_peered":"2023-10-18T12:25:15.216900+0000","last_unstale":"2023-10-18T12:26:15.210854+0000","last_undegraded":"2023-10-18T12:26:15.210854+0000","last_fullsized":"2023-10-18T12:26:15.210854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:24.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214751+0000","last_change":"2023-10-18T12:24:08.963312+0000","last_active":"2023-10-18T12:26:15.214751+0000","last_peered":"2023-10-18T12:26:15.214751+0000","last_clean":"2023-10-18T12:26:15.214751+0000","last_became_active":"2023-10-18T12:24:08.963025+0000","last_became_peered":"2023-10-18T12:24:08.963025+0000","last_unstale":"2023-10-18T12:26:15.214751+0000","last_undegraded":"2023-10-18T12:26:15.214751+0000","last_fullsized":"2023-10-18T12:26:15.214751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:05:06.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804971+0000","last_change":"2023-10-18T12:24:09.972969+0000","last_active":"2023-10-18T12:26:15.804971+0000","last_peered":"2023-10-18T12:26:15.804971+0000","last_clean":"2023-10-18T12:26:15.804971+0000","last_became_active":"2023-10-18T12:24:09.972663+0000","last_became_peered":"2023-10-18T12:24:09.972663+0000","last_unstale":"2023-10-18T12:26:15.804971+0000","last_undegraded":"2023-10-18T12:26:15.804971+0000","last_fullsized":"2023-10-18T12:26:15.804971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:15.309944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211287+0000","last_change":"2023-10-18T12:24:14.167499+0000","last_active":"2023-10-18T12:26:15.211287+0000","last_peered":"2023-10-18T12:26:15.211287+0000","last_clean":"2023-10-18T12:26:15.211287+0000","last_became_active":"2023-10-18T12:24:14.167094+0000","last_became_peered":"2023-10-18T12:24:14.167094+0000","last_unstale":"2023-10-18T12:26:15.211287+0000","last_undegraded":"2023-10-18T12:26:15.211287+0000","last_fullsized":"2023-10-18T12:26:15.211287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:00.751317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210513+0000","last_change":"2023-10-18T12:24:08.972229+0000","last_active":"2023-10-18T12:26:15.210513+0000","last_peered":"2023-10-18T12:26:15.210513+0000","last_clean":"2023-10-18T12:26:15.210513+0000","last_became_active":"2023-10-18T12:24:08.971944+0000","last_became_peered":"2023-10-18T12:24:08.971944+0000","last_unstale":"2023-10-18T12:26:15.210513+0000","last_undegraded":"2023-10-18T12:26:15.210513+0000","last_fullsized":"2023-10-18T12:26:15.210513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:16.023754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215014+0000","last_change":"2023-10-18T12:24:09.975943+0000","last_active":"2023-10-18T12:26:15.215014+0000","last_peered":"2023-10-18T12:26:15.215014+0000","last_clean":"2023-10-18T12:26:15.215014+0000","last_became_active":"2023-10-18T12:24:09.975615+0000","last_became_peered":"2023-10-18T12:24:09.975615+0000","last_unstale":"2023-10-18T12:26:15.215014+0000","last_undegraded":"2023-10-18T12:26:15.215014+0000","last_fullsized":"2023-10-18T12:26:15.215014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:48.088950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213129+0000","last_change":"2023-10-18T12:24:14.165255+0000","last_active":"2023-10-18T12:26:15.213129+0000","last_peered":"2023-10-18T12:26:15.213129+0000","last_clean":"2023-10-18T12:26:15.213129+0000","last_became_active":"2023-10-18T12:24:14.164814+0000","last_became_peered":"2023-10-18T12:24:14.164814+0000","last_unstale":"2023-10-18T12:26:15.213129+0000","last_undegraded":"2023-10-18T12:26:15.213129+0000","last_fullsized":"2023-10-18T12:26:15.213129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:48.163543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422291+0000","last_change":"2023-10-18T12:24:08.967235+0000","last_active":"2023-10-18T12:26:15.422291+0000","last_peered":"2023-10-18T12:26:15.422291+0000","last_clean":"2023-10-18T12:26:15.422291+0000","last_became_active":"2023-10-18T12:24:08.966602+0000","last_became_peered":"2023-10-18T12:24:08.966602+0000","last_unstale":"2023-10-18T12:26:15.422291+0000","last_undegraded":"2023-10-18T12:26:15.422291+0000","last_fullsized":"2023-10-18T12:26:15.422291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:57:19.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210045+0000","last_change":"2023-10-18T12:24:09.976547+0000","last_active":"2023-10-18T12:26:15.210045+0000","last_peered":"2023-10-18T12:26:15.210045+0000","last_clean":"2023-10-18T12:26:15.210045+0000","last_became_active":"2023-10-18T12:24:09.975708+0000","last_became_peered":"2023-10-18T12:24:09.975708+0000","last_unstale":"2023-10-18T12:26:15.210045+0000","last_undegraded":"2023-10-18T12:26:15.210045+0000","last_fullsized":"2023-10-18T12:26:15.210045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:42.259397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926312+0000","last_change":"2023-10-18T12:24:14.155978+0000","last_active":"2023-10-18T12:26:15.926312+0000","last_peered":"2023-10-18T12:26:15.926312+0000","last_clean":"2023-10-18T12:26:15.926312+0000","last_became_active":"2023-10-18T12:24:14.154827+0000","last_became_peered":"2023-10-18T12:24:14.154827+0000","last_unstale":"2023-10-18T12:26:15.926312+0000","last_undegraded":"2023-10-18T12:26:15.926312+0000","last_fullsized":"2023-10-18T12:26:15.926312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:41.978561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214658+0000","last_change":"2023-10-18T12:24:08.966723+0000","last_active":"2023-10-18T12:26:15.214658+0000","last_peered":"2023-10-18T12:26:15.214658+0000","last_clean":"2023-10-18T12:26:15.214658+0000","last_became_active":"2023-10-18T12:24:08.966451+0000","last_became_peered":"2023-10-18T12:24:08.966451+0000","last_unstale":"2023-10-18T12:26:15.214658+0000","last_undegraded":"2023-10-18T12:26:15.214658+0000","last_fullsized":"2023-10-18T12:26:15.214658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:29.036693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213122+0000","last_change":"2023-10-18T12:24:09.975266+0000","last_active":"2023-10-18T12:26:15.213122+0000","last_peered":"2023-10-18T12:26:15.213122+0000","last_clean":"2023-10-18T12:26:15.213122+0000","last_became_active":"2023-10-18T12:24:09.974928+0000","last_became_peered":"2023-10-18T12:24:09.974928+0000","last_unstale":"2023-10-18T12:26:15.213122+0000","last_undegraded":"2023-10-18T12:26:15.213122+0000","last_fullsized":"2023-10-18T12:26:15.213122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:07.634107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210318+0000","last_change":"2023-10-18T12:24:14.155062+0000","last_active":"2023-10-18T12:26:15.210318+0000","last_peered":"2023-10-18T12:26:15.210318+0000","last_clean":"2023-10-18T12:26:15.210318+0000","last_became_active":"2023-10-18T12:24:14.153158+0000","last_became_peered":"2023-10-18T12:24:14.153158+0000","last_unstale":"2023-10-18T12:26:15.210318+0000","last_undegraded":"2023-10-18T12:26:15.210318+0000","last_fullsized":"2023-10-18T12:26:15.210318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:11.123486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210810+0000","last_change":"2023-10-18T12:24:08.972133+0000","last_active":"2023-10-18T12:26:15.210810+0000","last_peered":"2023-10-18T12:26:15.210810+0000","last_clean":"2023-10-18T12:26:15.210810+0000","last_became_active":"2023-10-18T12:24:08.971829+0000","last_became_peered":"2023-10-18T12:24:08.971829+0000","last_unstale":"2023-10-18T12:26:15.210810+0000","last_undegraded":"2023-10-18T12:26:15.210810+0000","last_fullsized":"2023-10-18T12:26:15.210810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:39.827915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214669+0000","last_change":"2023-10-18T12:24:09.975360+0000","last_active":"2023-10-18T12:26:15.214669+0000","last_peered":"2023-10-18T12:26:15.214669+0000","last_clean":"2023-10-18T12:26:15.214669+0000","last_became_active":"2023-10-18T12:24:09.975049+0000","last_became_peered":"2023-10-18T12:24:09.975049+0000","last_unstale":"2023-10-18T12:26:15.214669+0000","last_undegraded":"2023-10-18T12:26:15.214669+0000","last_fullsized":"2023-10-18T12:26:15.214669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:52:48.626914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212867+0000","last_change":"2023-10-18T12:24:14.158768+0000","last_active":"2023-10-18T12:26:15.212867+0000","last_peered":"2023-10-18T12:26:15.212867+0000","last_clean":"2023-10-18T12:26:15.212867+0000","last_became_active":"2023-10-18T12:24:14.158069+0000","last_became_peered":"2023-10-18T12:24:14.158069+0000","last_unstale":"2023-10-18T12:26:15.212867+0000","last_undegraded":"2023-10-18T12:26:15.212867+0000","last_fullsized":"2023-10-18T12:26:15.212867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:46.495607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.212937+0000","last_change":"2023-10-18T12:24:08.971825+0000","last_active":"2023-10-18T12:25:15.212937+0000","last_peered":"2023-10-18T12:25:15.212937+0000","last_clean":"2023-10-18T12:25:15.212937+0000","last_became_active":"2023-10-18T12:24:08.971534+0000","last_became_peered":"2023-10-18T12:24:08.971534+0000","last_unstale":"2023-10-18T12:25:15.212937+0000","last_undegraded":"2023-10-18T12:25:15.212937+0000","last_fullsized":"2023-10-18T12:25:15.212937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:21:37.029306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926421+0000","last_change":"2023-10-18T12:24:09.979014+0000","last_active":"2023-10-18T12:26:15.926421+0000","last_peered":"2023-10-18T12:26:15.926421+0000","last_clean":"2023-10-18T12:26:15.926421+0000","last_became_active":"2023-10-18T12:24:09.978516+0000","last_became_peered":"2023-10-18T12:24:09.978516+0000","last_unstale":"2023-10-18T12:26:15.926421+0000","last_undegraded":"2023-10-18T12:26:15.926421+0000","last_fullsized":"2023-10-18T12:26:15.926421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:38.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210694+0000","last_change":"2023-10-18T12:24:14.158311+0000","last_active":"2023-10-18T12:25:15.210694+0000","last_peered":"2023-10-18T12:25:15.210694+0000","last_clean":"2023-10-18T12:25:15.210694+0000","last_became_active":"2023-10-18T12:24:14.157866+0000","last_became_peered":"2023-10-18T12:24:14.157866+0000","last_unstale":"2023-10-18T12:25:15.210694+0000","last_undegraded":"2023-10-18T12:25:15.210694+0000","last_fullsized":"2023-10-18T12:25:15.210694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:00.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214711+0000","last_change":"2023-10-18T12:24:08.981247+0000","last_active":"2023-10-18T12:26:15.214711+0000","last_peered":"2023-10-18T12:26:15.214711+0000","last_clean":"2023-10-18T12:26:15.214711+0000","last_became_active":"2023-10-18T12:24:08.980858+0000","last_became_peered":"2023-10-18T12:24:08.980858+0000","last_unstale":"2023-10-18T12:26:15.214711+0000","last_undegraded":"2023-10-18T12:26:15.214711+0000","last_fullsized":"2023-10-18T12:26:15.214711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:45.650864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214469+0000","last_change":"2023-10-18T12:25:15.216324+0000","last_active":"2023-10-18T12:26:15.214469+0000","last_peered":"2023-10-18T12:26:15.214469+0000","last_clean":"2023-10-18T12:26:15.214469+0000","last_became_active":"2023-10-18T12:25:15.215976+0000","last_became_peered":"2023-10-18T12:25:15.215976+0000","last_unstale":"2023-10-18T12:26:15.214469+0000","last_undegraded":"2023-10-18T12:26:15.214469+0000","last_fullsized":"2023-10-18T12:26:15.214469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:17:00.083345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213164+0000","last_change":"2023-10-18T12:24:14.161142+0000","last_active":"2023-10-18T12:26:15.213164+0000","last_peered":"2023-10-18T12:26:15.213164+0000","last_clean":"2023-10-18T12:26:15.213164+0000","last_became_active":"2023-10-18T12:24:14.160728+0000","last_became_peered":"2023-10-18T12:24:14.160728+0000","last_unstale":"2023-10-18T12:26:15.213164+0000","last_undegraded":"2023-10-18T12:26:15.213164+0000","last_fullsized":"2023-10-18T12:26:15.213164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:13.896924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":282,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:21.345751+0000","last_change":"2023-10-18T12:24:08.968518+0000","last_active":"2023-10-18T12:26:21.345751+0000","last_peered":"2023-10-18T12:26:21.345751+0000","last_clean":"2023-10-18T12:26:21.345751+0000","last_became_active":"2023-10-18T12:24:08.968134+0000","last_became_peered":"2023-10-18T12:24:08.968134+0000","last_unstale":"2023-10-18T12:26:21.345751+0000","last_undegraded":"2023-10-18T12:26:21.345751+0000","last_fullsized":"2023-10-18T12:26:21.345751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:19.191460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66829,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":227,"num_read_kb":179,"num_write":35,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805288+0000","last_change":"2023-10-18T12:24:09.970998+0000","last_active":"2023-10-18T12:26:15.805288+0000","last_peered":"2023-10-18T12:26:15.805288+0000","last_clean":"2023-10-18T12:26:15.805288+0000","last_became_active":"2023-10-18T12:24:09.970661+0000","last_became_peered":"2023-10-18T12:24:09.970661+0000","last_unstale":"2023-10-18T12:26:15.805288+0000","last_undegraded":"2023-10-18T12:26:15.805288+0000","last_fullsized":"2023-10-18T12:26:15.805288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:35.317646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215157+0000","last_change":"2023-10-18T12:24:14.154271+0000","last_active":"2023-10-18T12:26:15.215157+0000","last_peered":"2023-10-18T12:26:15.215157+0000","last_clean":"2023-10-18T12:26:15.215157+0000","last_became_active":"2023-10-18T12:24:14.153871+0000","last_became_peered":"2023-10-18T12:24:14.153871+0000","last_unstale":"2023-10-18T12:26:15.215157+0000","last_undegraded":"2023-10-18T12:26:15.215157+0000","last_fullsized":"2023-10-18T12:26:15.215157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:29.637502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210818+0000","last_change":"2023-10-18T12:24:08.968812+0000","last_active":"2023-10-18T12:26:15.210818+0000","last_peered":"2023-10-18T12:26:15.210818+0000","last_clean":"2023-10-18T12:26:15.210818+0000","last_became_active":"2023-10-18T12:24:08.968521+0000","last_became_peered":"2023-10-18T12:24:08.968521+0000","last_unstale":"2023-10-18T12:26:15.210818+0000","last_undegraded":"2023-10-18T12:26:15.210818+0000","last_fullsized":"2023-10-18T12:26:15.210818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:20.882801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422498+0000","last_change":"2023-10-18T12:24:09.987334+0000","last_active":"2023-10-18T12:26:15.422498+0000","last_peered":"2023-10-18T12:26:15.422498+0000","last_clean":"2023-10-18T12:26:15.422498+0000","last_became_active":"2023-10-18T12:24:09.986363+0000","last_became_peered":"2023-10-18T12:24:09.986363+0000","last_unstale":"2023-10-18T12:26:15.422498+0000","last_undegraded":"2023-10-18T12:26:15.422498+0000","last_fullsized":"2023-10-18T12:26:15.422498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:00.443225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212659+0000","last_change":"2023-10-18T12:24:14.164998+0000","last_active":"2023-10-18T12:26:15.212659+0000","last_peered":"2023-10-18T12:26:15.212659+0000","last_clean":"2023-10-18T12:26:15.212659+0000","last_became_active":"2023-10-18T12:24:14.164553+0000","last_became_peered":"2023-10-18T12:24:14.164553+0000","last_unstale":"2023-10-18T12:26:15.212659+0000","last_undegraded":"2023-10-18T12:26:15.212659+0000","last_fullsized":"2023-10-18T12:26:15.212659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:26.418364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215428+0000","last_change":"2023-10-18T12:24:08.978425+0000","last_active":"2023-10-18T12:26:15.215428+0000","last_peered":"2023-10-18T12:26:15.215428+0000","last_clean":"2023-10-18T12:26:15.215428+0000","last_became_active":"2023-10-18T12:24:08.978056+0000","last_became_peered":"2023-10-18T12:24:08.978056+0000","last_unstale":"2023-10-18T12:26:15.215428+0000","last_undegraded":"2023-10-18T12:26:15.215428+0000","last_fullsized":"2023-10-18T12:26:15.215428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:15.750282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.654415+0000","last_change":"2023-10-18T12:25:15.219532+0000","last_active":"2023-10-18T12:25:43.654415+0000","last_peered":"2023-10-18T12:25:43.654415+0000","last_clean":"2023-10-18T12:25:43.654415+0000","last_became_active":"2023-10-18T12:25:15.219206+0000","last_became_peered":"2023-10-18T12:25:15.219206+0000","last_unstale":"2023-10-18T12:25:43.654415+0000","last_undegraded":"2023-10-18T12:25:43.654415+0000","last_fullsized":"2023-10-18T12:25:43.654415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.796302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213505+0000","last_change":"2023-10-18T12:24:14.166761+0000","last_active":"2023-10-18T12:26:15.213505+0000","last_peered":"2023-10-18T12:26:15.213505+0000","last_clean":"2023-10-18T12:26:15.213505+0000","last_became_active":"2023-10-18T12:24:14.164518+0000","last_became_peered":"2023-10-18T12:24:14.164518+0000","last_unstale":"2023-10-18T12:26:15.213505+0000","last_undegraded":"2023-10-18T12:26:15.213505+0000","last_fullsized":"2023-10-18T12:26:15.213505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:38.230304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210457+0000","last_change":"2023-10-18T12:24:08.967221+0000","last_active":"2023-10-18T12:25:15.210457+0000","last_peered":"2023-10-18T12:25:15.210457+0000","last_clean":"2023-10-18T12:25:15.210457+0000","last_became_active":"2023-10-18T12:24:08.966931+0000","last_became_peered":"2023-10-18T12:24:08.966931+0000","last_unstale":"2023-10-18T12:25:15.210457+0000","last_undegraded":"2023-10-18T12:25:15.210457+0000","last_fullsized":"2023-10-18T12:25:15.210457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:46.452229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.642483+0000","last_change":"2023-10-18T12:24:09.969776+0000","last_active":"2023-10-18T12:25:43.642483+0000","last_peered":"2023-10-18T12:25:43.642483+0000","last_clean":"2023-10-18T12:25:43.642483+0000","last_became_active":"2023-10-18T12:24:09.969490+0000","last_became_peered":"2023-10-18T12:24:09.969490+0000","last_unstale":"2023-10-18T12:25:43.642483+0000","last_undegraded":"2023-10-18T12:25:43.642483+0000","last_fullsized":"2023-10-18T12:25:43.642483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:43.707383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422704+0000","last_change":"2023-10-18T12:24:14.168725+0000","last_active":"2023-10-18T12:26:15.422704+0000","last_peered":"2023-10-18T12:26:15.422704+0000","last_clean":"2023-10-18T12:26:15.422704+0000","last_became_active":"2023-10-18T12:24:14.166519+0000","last_became_peered":"2023-10-18T12:24:14.166519+0000","last_unstale":"2023-10-18T12:26:15.422704+0000","last_undegraded":"2023-10-18T12:26:15.422704+0000","last_fullsized":"2023-10-18T12:26:15.422704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:06.123016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804857+0000","last_change":"2023-10-18T12:25:15.225793+0000","last_active":"2023-10-18T12:26:15.804857+0000","last_peered":"2023-10-18T12:26:15.804857+0000","last_clean":"2023-10-18T12:26:15.804857+0000","last_became_active":"2023-10-18T12:25:15.218301+0000","last_became_peered":"2023-10-18T12:25:15.218301+0000","last_unstale":"2023-10-18T12:26:15.804857+0000","last_undegraded":"2023-10-18T12:26:15.804857+0000","last_fullsized":"2023-10-18T12:26:15.804857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:23.469237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214502+0000","last_change":"2023-10-18T12:24:09.970012+0000","last_active":"2023-10-18T12:26:15.214502+0000","last_peered":"2023-10-18T12:26:15.214502+0000","last_clean":"2023-10-18T12:26:15.214502+0000","last_became_active":"2023-10-18T12:24:09.968963+0000","last_became_peered":"2023-10-18T12:24:09.968963+0000","last_unstale":"2023-10-18T12:26:15.214502+0000","last_undegraded":"2023-10-18T12:26:15.214502+0000","last_fullsized":"2023-10-18T12:26:15.214502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:20.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214483+0000","last_change":"2023-10-18T12:24:14.157093+0000","last_active":"2023-10-18T12:26:15.214483+0000","last_peered":"2023-10-18T12:26:15.214483+0000","last_clean":"2023-10-18T12:26:15.214483+0000","last_became_active":"2023-10-18T12:24:14.156675+0000","last_became_peered":"2023-10-18T12:24:14.156675+0000","last_unstale":"2023-10-18T12:26:15.214483+0000","last_undegraded":"2023-10-18T12:26:15.214483+0000","last_fullsized":"2023-10-18T12:26:15.214483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:05.621527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213136+0000","last_change":"2023-10-18T12:25:15.223142+0000","last_active":"2023-10-18T12:26:15.213136+0000","last_peered":"2023-10-18T12:26:15.213136+0000","last_clean":"2023-10-18T12:26:15.213136+0000","last_became_active":"2023-10-18T12:25:15.218709+0000","last_became_peered":"2023-10-18T12:25:15.218709+0000","last_unstale":"2023-10-18T12:26:15.213136+0000","last_undegraded":"2023-10-18T12:26:15.213136+0000","last_fullsized":"2023-10-18T12:26:15.213136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:47.290720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.213093+0000","last_change":"2023-10-18T12:24:09.969681+0000","last_active":"2023-10-18T12:26:15.213093+0000","last_peered":"2023-10-18T12:26:15.213093+0000","last_clean":"2023-10-18T12:26:15.213093+0000","last_became_active":"2023-10-18T12:24:09.969383+0000","last_became_peered":"2023-10-18T12:24:09.969383+0000","last_unstale":"2023-10-18T12:26:15.213093+0000","last_undegraded":"2023-10-18T12:26:15.213093+0000","last_fullsized":"2023-10-18T12:26:15.213093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:12.281351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.215978+0000","last_change":"2023-10-18T12:24:14.179435+0000","last_active":"2023-10-18T12:26:16.215978+0000","last_peered":"2023-10-18T12:26:16.215978+0000","last_clean":"2023-10-18T12:26:16.215978+0000","last_became_active":"2023-10-18T12:24:14.179057+0000","last_became_peered":"2023-10-18T12:24:14.179057+0000","last_unstale":"2023-10-18T12:26:16.215978+0000","last_undegraded":"2023-10-18T12:26:16.215978+0000","last_fullsized":"2023-10-18T12:26:16.215978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:24.900010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.661271+0000","last_change":"2023-10-18T12:24:08.977778+0000","last_active":"2023-10-18T12:26:15.661271+0000","last_peered":"2023-10-18T12:26:15.661271+0000","last_clean":"2023-10-18T12:26:15.661271+0000","last_became_active":"2023-10-18T12:24:08.977465+0000","last_became_peered":"2023-10-18T12:24:08.977465+0000","last_unstale":"2023-10-18T12:26:15.661271+0000","last_undegraded":"2023-10-18T12:26:15.661271+0000","last_fullsized":"2023-10-18T12:26:15.661271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:56.860327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.336032+0000","last_change":"2023-10-18T12:24:09.976803+0000","last_active":"2023-10-18T12:26:16.336032+0000","last_peered":"2023-10-18T12:26:16.336032+0000","last_clean":"2023-10-18T12:26:16.336032+0000","last_became_active":"2023-10-18T12:24:09.976380+0000","last_became_peered":"2023-10-18T12:24:09.976380+0000","last_unstale":"2023-10-18T12:26:16.336032+0000","last_undegraded":"2023-10-18T12:26:16.336032+0000","last_fullsized":"2023-10-18T12:26:16.336032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:30.989810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422601+0000","last_change":"2023-10-18T12:24:14.149540+0000","last_active":"2023-10-18T12:26:15.422601+0000","last_peered":"2023-10-18T12:26:15.422601+0000","last_clean":"2023-10-18T12:26:15.422601+0000","last_became_active":"2023-10-18T12:24:14.149159+0000","last_became_peered":"2023-10-18T12:24:14.149159+0000","last_unstale":"2023-10-18T12:26:15.422601+0000","last_undegraded":"2023-10-18T12:26:15.422601+0000","last_fullsized":"2023-10-18T12:26:15.422601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:10.730021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:26:14.772089+0000","last_change":"2023-10-18T12:24:08.971633+0000","last_active":"2023-10-18T12:26:14.772089+0000","last_peered":"2023-10-18T12:26:14.772089+0000","last_clean":"2023-10-18T12:26:14.772089+0000","last_became_active":"2023-10-18T12:24:08.971271+0000","last_became_peered":"2023-10-18T12:24:08.971271+0000","last_unstale":"2023-10-18T12:26:14.772089+0000","last_undegraded":"2023-10-18T12:26:14.772089+0000","last_fullsized":"2023-10-18T12:26:14.772089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:44.140927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.335890+0000","last_change":"2023-10-18T12:24:09.982905+0000","last_active":"2023-10-18T12:26:16.335890+0000","last_peered":"2023-10-18T12:26:16.335890+0000","last_clean":"2023-10-18T12:26:16.335890+0000","last_became_active":"2023-10-18T12:24:09.982624+0000","last_became_peered":"2023-10-18T12:24:09.982624+0000","last_unstale":"2023-10-18T12:26:16.335890+0000","last_undegraded":"2023-10-18T12:26:16.335890+0000","last_fullsized":"2023-10-18T12:26:16.335890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:47.830470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422708+0000","last_change":"2023-10-18T12:24:14.179802+0000","last_active":"2023-10-18T12:26:15.422708+0000","last_peered":"2023-10-18T12:26:15.422708+0000","last_clean":"2023-10-18T12:26:15.422708+0000","last_became_active":"2023-10-18T12:24:14.179316+0000","last_became_peered":"2023-10-18T12:24:14.179316+0000","last_unstale":"2023-10-18T12:26:15.422708+0000","last_undegraded":"2023-10-18T12:26:15.422708+0000","last_fullsized":"2023-10-18T12:26:15.422708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:30.214244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214953+0000","last_change":"2023-10-18T12:24:09.969337+0000","last_active":"2023-10-18T12:26:15.214953+0000","last_peered":"2023-10-18T12:26:15.214953+0000","last_clean":"2023-10-18T12:26:15.214953+0000","last_became_active":"2023-10-18T12:24:09.968800+0000","last_became_peered":"2023-10-18T12:24:09.968800+0000","last_unstale":"2023-10-18T12:26:15.214953+0000","last_undegraded":"2023-10-18T12:26:15.214953+0000","last_fullsized":"2023-10-18T12:26:15.214953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:49.437710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422171+0000","last_change":"2023-10-18T12:24:08.978118+0000","last_active":"2023-10-18T12:26:15.422171+0000","last_peered":"2023-10-18T12:26:15.422171+0000","last_clean":"2023-10-18T12:26:15.422171+0000","last_became_active":"2023-10-18T12:24:08.977785+0000","last_became_peered":"2023-10-18T12:24:08.977785+0000","last_unstale":"2023-10-18T12:26:15.422171+0000","last_undegraded":"2023-10-18T12:26:15.422171+0000","last_fullsized":"2023-10-18T12:26:15.422171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:31.256790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.696371+0000","last_change":"2023-10-18T12:24:14.164142+0000","last_active":"2023-10-18T12:26:16.696371+0000","last_peered":"2023-10-18T12:26:16.696371+0000","last_clean":"2023-10-18T12:26:16.696371+0000","last_became_active":"2023-10-18T12:24:14.163777+0000","last_became_peered":"2023-10-18T12:24:14.163777+0000","last_unstale":"2023-10-18T12:26:16.696371+0000","last_undegraded":"2023-10-18T12:26:16.696371+0000","last_fullsized":"2023-10-18T12:26:16.696371+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:47.778478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.219876+0000","last_change":"2023-10-18T12:26:16.219876+0000","last_active":"2023-10-18T12:26:16.219876+0000","last_peered":"2023-10-18T12:26:16.219876+0000","last_clean":"2023-10-18T12:26:16.219876+0000","last_became_active":"2023-10-18T12:26:16.219515+0000","last_became_peered":"2023-10-18T12:26:16.219515+0000","last_unstale":"2023-10-18T12:26:16.219876+0000","last_undegraded":"2023-10-18T12:26:16.219876+0000","last_fullsized":"2023-10-18T12:26:16.219876+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:55.404152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.216871+0000","last_change":"2023-10-18T12:24:08.965744+0000","last_active":"2023-10-18T12:26:16.216871+0000","last_peered":"2023-10-18T12:26:16.216871+0000","last_clean":"2023-10-18T12:26:16.216871+0000","last_became_active":"2023-10-18T12:24:08.965313+0000","last_became_peered":"2023-10-18T12:24:08.965313+0000","last_unstale":"2023-10-18T12:26:16.216871+0000","last_undegraded":"2023-10-18T12:26:16.216871+0000","last_fullsized":"2023-10-18T12:26:16.216871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:34.120898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":328,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:59.770287+0000","last_change":"2023-10-18T12:23:04.679558+0000","last_active":"2023-10-18T12:25:59.770287+0000","last_peered":"2023-10-18T12:25:59.770287+0000","last_clean":"2023-10-18T12:25:59.770287+0000","last_became_active":"2023-10-18T12:23:04.679023+0000","last_became_peered":"2023-10-18T12:23:04.679023+0000","last_unstale":"2023-10-18T12:25:59.770287+0000","last_undegraded":"2023-10-18T12:25:59.770287+0000","last_fullsized":"2023-10-18T12:25:59.770287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:19:58.322880+0000","last_clean_scrub_stamp":"2023-10-18T12:19:58.322880+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.215078+0000","last_change":"2023-10-18T12:25:15.223137+0000","last_active":"2023-10-18T12:26:15.215078+0000","last_peered":"2023-10-18T12:26:15.215078+0000","last_clean":"2023-10-18T12:26:15.215078+0000","last_became_active":"2023-10-18T12:25:15.222746+0000","last_became_peered":"2023-10-18T12:25:15.222746+0000","last_unstale":"2023-10-18T12:26:15.215078+0000","last_undegraded":"2023-10-18T12:26:15.215078+0000","last_fullsized":"2023-10-18T12:26:15.215078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:37.042773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:17.196681+0000","last_change":"2023-10-18T12:24:09.958469+0000","last_active":"2023-10-18T12:26:17.196681+0000","last_peered":"2023-10-18T12:26:17.196681+0000","last_clean":"2023-10-18T12:26:17.196681+0000","last_became_active":"2023-10-18T12:24:09.958122+0000","last_became_peered":"2023-10-18T12:24:09.958122+0000","last_unstale":"2023-10-18T12:26:17.196681+0000","last_undegraded":"2023-10-18T12:26:17.196681+0000","last_fullsized":"2023-10-18T12:26:17.196681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:29.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212471+0000","last_change":"2023-10-18T12:25:15.216813+0000","last_active":"2023-10-18T12:26:15.212471+0000","last_peered":"2023-10-18T12:26:15.212471+0000","last_clean":"2023-10-18T12:26:15.212471+0000","last_became_active":"2023-10-18T12:25:15.216467+0000","last_became_peered":"2023-10-18T12:25:15.216467+0000","last_unstale":"2023-10-18T12:26:15.212471+0000","last_undegraded":"2023-10-18T12:26:15.212471+0000","last_fullsized":"2023-10-18T12:26:15.212471+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:47.740658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212735+0000","last_change":"2023-10-18T12:24:14.157501+0000","last_active":"2023-10-18T12:26:15.212735+0000","last_peered":"2023-10-18T12:26:15.212735+0000","last_clean":"2023-10-18T12:26:15.212735+0000","last_became_active":"2023-10-18T12:24:14.157145+0000","last_became_peered":"2023-10-18T12:24:14.157145+0000","last_unstale":"2023-10-18T12:26:15.212735+0000","last_undegraded":"2023-10-18T12:26:15.212735+0000","last_fullsized":"2023-10-18T12:26:15.212735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:46.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805277+0000","last_change":"2023-10-18T12:24:08.978995+0000","last_active":"2023-10-18T12:26:15.805277+0000","last_peered":"2023-10-18T12:26:15.805277+0000","last_clean":"2023-10-18T12:26:15.805277+0000","last_became_active":"2023-10-18T12:24:08.978661+0000","last_became_peered":"2023-10-18T12:24:08.978661+0000","last_unstale":"2023-10-18T12:26:15.805277+0000","last_undegraded":"2023-10-18T12:26:15.805277+0000","last_fullsized":"2023-10-18T12:26:15.805277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:42.091334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:17.196955+0000","last_change":"2023-10-18T12:24:09.967209+0000","last_active":"2023-10-18T12:26:17.196955+0000","last_peered":"2023-10-18T12:26:17.196955+0000","last_clean":"2023-10-18T12:26:17.196955+0000","last_became_active":"2023-10-18T12:24:09.966899+0000","last_became_peered":"2023-10-18T12:24:09.966899+0000","last_unstale":"2023-10-18T12:26:17.196955+0000","last_undegraded":"2023-10-18T12:26:17.196955+0000","last_fullsized":"2023-10-18T12:26:17.196955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:14.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210022+0000","last_change":"2023-10-18T12:24:14.178562+0000","last_active":"2023-10-18T12:26:15.210022+0000","last_peered":"2023-10-18T12:26:15.210022+0000","last_clean":"2023-10-18T12:26:15.210022+0000","last_became_active":"2023-10-18T12:24:14.178171+0000","last_became_peered":"2023-10-18T12:24:14.178171+0000","last_unstale":"2023-10-18T12:26:15.210022+0000","last_undegraded":"2023-10-18T12:26:15.210022+0000","last_fullsized":"2023-10-18T12:26:15.210022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:08.729817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.216330+0000","last_change":"2023-10-18T12:24:08.967111+0000","last_active":"2023-10-18T12:26:16.216330+0000","last_peered":"2023-10-18T12:26:16.216330+0000","last_clean":"2023-10-18T12:26:16.216330+0000","last_became_active":"2023-10-18T12:24:08.966753+0000","last_became_peered":"2023-10-18T12:24:08.966753+0000","last_unstale":"2023-10-18T12:26:16.216330+0000","last_undegraded":"2023-10-18T12:26:16.216330+0000","last_fullsized":"2023-10-18T12:26:16.216330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:46:08.300445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.635861+0000","last_change":"2023-10-18T12:24:09.983265+0000","last_active":"2023-10-18T12:26:16.635861+0000","last_peered":"2023-10-18T12:26:16.635861+0000","last_clean":"2023-10-18T12:26:16.635861+0000","last_became_active":"2023-10-18T12:24:09.976597+0000","last_became_peered":"2023-10-18T12:24:09.976597+0000","last_unstale":"2023-10-18T12:26:16.635861+0000","last_undegraded":"2023-10-18T12:26:16.635861+0000","last_fullsized":"2023-10-18T12:26:16.635861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:55.270056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.221950+0000","last_change":"2023-10-18T12:24:14.151595+0000","last_active":"2023-10-18T12:26:16.221950+0000","last_peered":"2023-10-18T12:26:16.221950+0000","last_clean":"2023-10-18T12:26:16.221950+0000","last_became_active":"2023-10-18T12:24:14.151208+0000","last_became_peered":"2023-10-18T12:24:14.151208+0000","last_unstale":"2023-10-18T12:26:16.221950+0000","last_undegraded":"2023-10-18T12:26:16.221950+0000","last_fullsized":"2023-10-18T12:26:16.221950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:09.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.211145+0000","last_change":"2023-10-18T12:24:08.970002+0000","last_active":"2023-10-18T12:25:15.211145+0000","last_peered":"2023-10-18T12:25:15.211145+0000","last_clean":"2023-10-18T12:25:15.211145+0000","last_became_active":"2023-10-18T12:24:08.969659+0000","last_became_peered":"2023-10-18T12:24:08.969659+0000","last_unstale":"2023-10-18T12:25:15.211145+0000","last_undegraded":"2023-10-18T12:25:15.211145+0000","last_fullsized":"2023-10-18T12:25:15.211145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:08.661230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:17.326996+0000","last_change":"2023-10-18T12:24:09.986320+0000","last_active":"2023-10-18T12:26:17.326996+0000","last_peered":"2023-10-18T12:26:17.326996+0000","last_clean":"2023-10-18T12:26:17.326996+0000","last_became_active":"2023-10-18T12:24:09.984890+0000","last_became_peered":"2023-10-18T12:24:09.984890+0000","last_unstale":"2023-10-18T12:26:17.326996+0000","last_undegraded":"2023-10-18T12:26:17.326996+0000","last_fullsized":"2023-10-18T12:26:17.326996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:07.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.223680+0000","last_change":"2023-10-18T12:25:15.223250+0000","last_active":"2023-10-18T12:25:15.223680+0000","last_peered":"2023-10-18T12:25:15.223680+0000","last_clean":"2023-10-18T12:25:15.223680+0000","last_became_active":"2023-10-18T12:25:15.222753+0000","last_became_peered":"2023-10-18T12:25:15.222753+0000","last_unstale":"2023-10-18T12:25:15.223680+0000","last_undegraded":"2023-10-18T12:25:15.223680+0000","last_fullsized":"2023-10-18T12:25:15.223680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:55.677420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,1],"acting":[4,5,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.218389+0000","last_change":"2023-10-18T12:24:08.975834+0000","last_active":"2023-10-18T12:26:16.218389+0000","last_peered":"2023-10-18T12:26:16.218389+0000","last_clean":"2023-10-18T12:26:16.218389+0000","last_became_active":"2023-10-18T12:24:08.975504+0000","last_became_peered":"2023-10-18T12:24:08.975504+0000","last_unstale":"2023-10-18T12:26:16.218389+0000","last_undegraded":"2023-10-18T12:26:16.218389+0000","last_fullsized":"2023-10-18T12:26:16.218389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:56.455960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.215824+0000","last_change":"2023-10-18T12:24:09.970879+0000","last_active":"2023-10-18T12:26:16.215824+0000","last_peered":"2023-10-18T12:26:16.215824+0000","last_clean":"2023-10-18T12:26:16.215824+0000","last_became_active":"2023-10-18T12:24:09.970564+0000","last_became_peered":"2023-10-18T12:24:09.970564+0000","last_unstale":"2023-10-18T12:26:16.215824+0000","last_undegraded":"2023-10-18T12:26:16.215824+0000","last_fullsized":"2023-10-18T12:26:16.215824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:04.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.805400+0000","last_change":"2023-10-18T12:25:15.219514+0000","last_active":"2023-10-18T12:26:15.805400+0000","last_peered":"2023-10-18T12:26:15.805400+0000","last_clean":"2023-10-18T12:26:15.805400+0000","last_became_active":"2023-10-18T12:25:15.219010+0000","last_became_peered":"2023-10-18T12:25:15.219010+0000","last_unstale":"2023-10-18T12:26:15.805400+0000","last_undegraded":"2023-10-18T12:26:15.805400+0000","last_fullsized":"2023-10-18T12:26:15.805400+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:08.516553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.635113+0000","last_change":"2023-10-18T12:24:08.972638+0000","last_active":"2023-10-18T12:26:16.635113+0000","last_peered":"2023-10-18T12:26:16.635113+0000","last_clean":"2023-10-18T12:26:16.635113+0000","last_became_active":"2023-10-18T12:24:08.972080+0000","last_became_peered":"2023-10-18T12:24:08.972080+0000","last_unstale":"2023-10-18T12:26:16.635113+0000","last_undegraded":"2023-10-18T12:26:16.635113+0000","last_fullsized":"2023-10-18T12:26:16.635113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:45.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212690+0000","last_change":"2023-10-18T12:24:09.983924+0000","last_active":"2023-10-18T12:26:15.212690+0000","last_peered":"2023-10-18T12:26:15.212690+0000","last_clean":"2023-10-18T12:26:15.212690+0000","last_became_active":"2023-10-18T12:24:09.983586+0000","last_became_peered":"2023-10-18T12:24:09.983586+0000","last_unstale":"2023-10-18T12:26:15.212690+0000","last_undegraded":"2023-10-18T12:26:15.212690+0000","last_fullsized":"2023-10-18T12:26:15.212690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:21.974609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210778+0000","last_change":"2023-10-18T12:24:14.167275+0000","last_active":"2023-10-18T12:25:15.210778+0000","last_peered":"2023-10-18T12:25:15.210778+0000","last_clean":"2023-10-18T12:25:15.210778+0000","last_became_active":"2023-10-18T12:24:14.166859+0000","last_became_peered":"2023-10-18T12:24:14.166859+0000","last_unstale":"2023-10-18T12:25:15.210778+0000","last_undegraded":"2023-10-18T12:25:15.210778+0000","last_fullsized":"2023-10-18T12:25:15.210778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:22.320295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.222322+0000","last_change":"2023-10-18T12:24:08.965022+0000","last_active":"2023-10-18T12:26:16.222322+0000","last_peered":"2023-10-18T12:26:16.222322+0000","last_clean":"2023-10-18T12:26:16.222322+0000","last_became_active":"2023-10-18T12:24:08.964712+0000","last_became_peered":"2023-10-18T12:24:08.964712+0000","last_unstale":"2023-10-18T12:26:16.222322+0000","last_undegraded":"2023-10-18T12:26:16.222322+0000","last_fullsized":"2023-10-18T12:26:16.222322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.710115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210503+0000","last_change":"2023-10-18T12:24:09.984749+0000","last_active":"2023-10-18T12:26:15.210503+0000","last_peered":"2023-10-18T12:26:15.210503+0000","last_clean":"2023-10-18T12:26:15.210503+0000","last_became_active":"2023-10-18T12:24:09.984419+0000","last_became_peered":"2023-10-18T12:24:09.984419+0000","last_unstale":"2023-10-18T12:26:15.210503+0000","last_undegraded":"2023-10-18T12:26:15.210503+0000","last_fullsized":"2023-10-18T12:26:15.210503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:44.126763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:15.210651+0000","last_change":"2023-10-18T12:24:14.179845+0000","last_active":"2023-10-18T12:25:15.210651+0000","last_peered":"2023-10-18T12:25:15.210651+0000","last_clean":"2023-10-18T12:25:15.210651+0000","last_became_active":"2023-10-18T12:24:14.179451+0000","last_became_peered":"2023-10-18T12:24:14.179451+0000","last_unstale":"2023-10-18T12:25:15.210651+0000","last_undegraded":"2023-10-18T12:25:15.210651+0000","last_fullsized":"2023-10-18T12:25:15.210651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:07.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.804975+0000","last_change":"2023-10-18T12:24:08.967812+0000","last_active":"2023-10-18T12:26:15.804975+0000","last_peered":"2023-10-18T12:26:15.804975+0000","last_clean":"2023-10-18T12:26:15.804975+0000","last_became_active":"2023-10-18T12:24:08.967489+0000","last_became_peered":"2023-10-18T12:24:08.967489+0000","last_unstale":"2023-10-18T12:26:15.804975+0000","last_undegraded":"2023-10-18T12:26:15.804975+0000","last_fullsized":"2023-10-18T12:26:15.804975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:03.749700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.696098+0000","last_change":"2023-10-18T12:24:09.972378+0000","last_active":"2023-10-18T12:26:16.696098+0000","last_peered":"2023-10-18T12:26:16.696098+0000","last_clean":"2023-10-18T12:26:16.696098+0000","last_became_active":"2023-10-18T12:24:09.972085+0000","last_became_peered":"2023-10-18T12:24:09.972085+0000","last_unstale":"2023-10-18T12:26:16.696098+0000","last_undegraded":"2023-10-18T12:26:16.696098+0000","last_fullsized":"2023-10-18T12:26:16.696098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:20.784574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214199+0000","last_change":"2023-10-18T12:24:14.152675+0000","last_active":"2023-10-18T12:26:15.214199+0000","last_peered":"2023-10-18T12:26:15.214199+0000","last_clean":"2023-10-18T12:26:15.214199+0000","last_became_active":"2023-10-18T12:24:14.152189+0000","last_became_peered":"2023-10-18T12:24:14.152189+0000","last_unstale":"2023-10-18T12:26:15.214199+0000","last_undegraded":"2023-10-18T12:26:15.214199+0000","last_fullsized":"2023-10-18T12:26:15.214199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:22.633443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211510+0000","last_change":"2023-10-18T12:24:08.977451+0000","last_active":"2023-10-18T12:26:15.211510+0000","last_peered":"2023-10-18T12:26:15.211510+0000","last_clean":"2023-10-18T12:26:15.211510+0000","last_became_active":"2023-10-18T12:24:08.977132+0000","last_became_peered":"2023-10-18T12:24:08.977132+0000","last_unstale":"2023-10-18T12:26:15.211510+0000","last_undegraded":"2023-10-18T12:26:15.211510+0000","last_fullsized":"2023-10-18T12:26:15.211510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:22.237289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.644615+0000","last_change":"2023-10-18T12:24:09.968277+0000","last_active":"2023-10-18T12:25:43.644615+0000","last_peered":"2023-10-18T12:25:43.644615+0000","last_clean":"2023-10-18T12:25:43.644615+0000","last_became_active":"2023-10-18T12:24:09.967949+0000","last_became_peered":"2023-10-18T12:24:09.967949+0000","last_unstale":"2023-10-18T12:25:43.644615+0000","last_undegraded":"2023-10-18T12:25:43.644615+0000","last_fullsized":"2023-10-18T12:25:43.644615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:02.301421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926059+0000","last_change":"2023-10-18T12:24:14.152875+0000","last_active":"2023-10-18T12:26:15.926059+0000","last_peered":"2023-10-18T12:26:15.926059+0000","last_clean":"2023-10-18T12:26:15.926059+0000","last_became_active":"2023-10-18T12:24:14.152503+0000","last_became_peered":"2023-10-18T12:24:14.152503+0000","last_unstale":"2023-10-18T12:26:15.926059+0000","last_undegraded":"2023-10-18T12:26:15.926059+0000","last_fullsized":"2023-10-18T12:26:15.926059+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:54.442732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214930+0000","last_change":"2023-10-18T12:24:08.968928+0000","last_active":"2023-10-18T12:26:15.214930+0000","last_peered":"2023-10-18T12:26:15.214930+0000","last_clean":"2023-10-18T12:26:15.214930+0000","last_became_active":"2023-10-18T12:24:08.968621+0000","last_became_peered":"2023-10-18T12:24:08.968621+0000","last_unstale":"2023-10-18T12:26:15.214930+0000","last_undegraded":"2023-10-18T12:26:15.214930+0000","last_fullsized":"2023-10-18T12:26:15.214930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.846654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.639182+0000","last_change":"2023-10-18T12:24:09.984631+0000","last_active":"2023-10-18T12:25:43.639182+0000","last_peered":"2023-10-18T12:25:43.639182+0000","last_clean":"2023-10-18T12:25:43.639182+0000","last_became_active":"2023-10-18T12:24:09.984372+0000","last_became_peered":"2023-10-18T12:24:09.984372+0000","last_unstale":"2023-10-18T12:25:43.639182+0000","last_undegraded":"2023-10-18T12:25:43.639182+0000","last_fullsized":"2023-10-18T12:25:43.639182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:35.938990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214280+0000","last_change":"2023-10-18T12:24:14.151772+0000","last_active":"2023-10-18T12:26:15.214280+0000","last_peered":"2023-10-18T12:26:15.214280+0000","last_clean":"2023-10-18T12:26:15.214280+0000","last_became_active":"2023-10-18T12:24:14.151317+0000","last_became_peered":"2023-10-18T12:24:14.151317+0000","last_unstale":"2023-10-18T12:26:15.214280+0000","last_undegraded":"2023-10-18T12:26:15.214280+0000","last_fullsized":"2023-10-18T12:26:15.214280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:13.129172+0000","last_clean_scrub_stamp":"2023-10-18T12:24:13.129172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:30.630603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422728+0000","last_change":"2023-10-18T12:25:15.225441+0000","last_active":"2023-10-18T12:26:15.422728+0000","last_peered":"2023-10-18T12:26:15.422728+0000","last_clean":"2023-10-18T12:26:15.422728+0000","last_became_active":"2023-10-18T12:25:15.218656+0000","last_became_peered":"2023-10-18T12:25:15.218656+0000","last_unstale":"2023-10-18T12:26:15.422728+0000","last_undegraded":"2023-10-18T12:26:15.422728+0000","last_fullsized":"2023-10-18T12:26:15.422728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:07.943511+0000","last_clean_scrub_stamp":"2023-10-18T12:24:07.943511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:14.009834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T12:25:43.640317+0000","last_change":"2023-10-18T12:24:09.968331+0000","last_active":"2023-10-18T12:25:43.640317+0000","last_peered":"2023-10-18T12:25:43.640317+0000","last_clean":"2023-10-18T12:25:43.640317+0000","last_became_active":"2023-10-18T12:24:09.967979+0000","last_became_peered":"2023-10-18T12:24:09.967979+0000","last_unstale":"2023-10-18T12:25:43.640317+0000","last_undegraded":"2023-10-18T12:25:43.640317+0000","last_fullsized":"2023-10-18T12:25:43.640317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:06.726447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.223970+0000","last_change":"2023-10-18T12:26:16.223538+0000","last_active":"2023-10-18T12:26:16.223970+0000","last_peered":"2023-10-18T12:26:16.223970+0000","last_clean":"2023-10-18T12:26:16.223970+0000","last_became_active":"2023-10-18T12:26:16.223094+0000","last_became_peered":"2023-10-18T12:26:16.223094+0000","last_unstale":"2023-10-18T12:26:16.223970+0000","last_undegraded":"2023-10-18T12:26:16.223970+0000","last_fullsized":"2023-10-18T12:26:16.223970+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:23:31.237530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.222109+0000","last_change":"2023-10-18T12:24:09.966906+0000","last_active":"2023-10-18T12:26:16.222109+0000","last_peered":"2023-10-18T12:26:16.222109+0000","last_clean":"2023-10-18T12:26:16.222109+0000","last_became_active":"2023-10-18T12:24:09.966570+0000","last_became_peered":"2023-10-18T12:24:09.966570+0000","last_unstale":"2023-10-18T12:26:16.222109+0000","last_undegraded":"2023-10-18T12:26:16.222109+0000","last_fullsized":"2023-10-18T12:26:16.222109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.222132+0000","last_change":"2023-10-18T12:24:09.976660+0000","last_active":"2023-10-18T12:26:16.222132+0000","last_peered":"2023-10-18T12:26:16.222132+0000","last_clean":"2023-10-18T12:26:16.222132+0000","last_became_active":"2023-10-18T12:24:09.976341+0000","last_became_peered":"2023-10-18T12:24:09.976341+0000","last_unstale":"2023-10-18T12:26:16.222132+0000","last_undegraded":"2023-10-18T12:26:16.222132+0000","last_fullsized":"2023-10-18T12:26:16.222132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:14.074375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.926231+0000","last_change":"2023-10-18T12:24:09.982011+0000","last_active":"2023-10-18T12:26:15.926231+0000","last_peered":"2023-10-18T12:26:15.926231+0000","last_clean":"2023-10-18T12:26:15.926231+0000","last_became_active":"2023-10-18T12:24:09.979040+0000","last_became_peered":"2023-10-18T12:24:09.979040+0000","last_unstale":"2023-10-18T12:26:15.926231+0000","last_undegraded":"2023-10-18T12:26:15.926231+0000","last_fullsized":"2023-10-18T12:26:15.926231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:54.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422501+0000","last_change":"2023-10-18T12:24:09.977299+0000","last_active":"2023-10-18T12:26:15.422501+0000","last_peered":"2023-10-18T12:26:15.422501+0000","last_clean":"2023-10-18T12:26:15.422501+0000","last_became_active":"2023-10-18T12:24:09.975556+0000","last_became_peered":"2023-10-18T12:24:09.975556+0000","last_unstale":"2023-10-18T12:26:15.422501+0000","last_undegraded":"2023-10-18T12:26:15.422501+0000","last_fullsized":"2023-10-18T12:26:15.422501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212928+0000","last_change":"2023-10-18T12:24:09.969714+0000","last_active":"2023-10-18T12:26:15.212928+0000","last_peered":"2023-10-18T12:26:15.212928+0000","last_clean":"2023-10-18T12:26:15.212928+0000","last_became_active":"2023-10-18T12:24:09.969426+0000","last_became_peered":"2023-10-18T12:24:09.969426+0000","last_unstale":"2023-10-18T12:26:15.212928+0000","last_undegraded":"2023-10-18T12:26:15.212928+0000","last_fullsized":"2023-10-18T12:26:15.212928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:56.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.211073+0000","last_change":"2023-10-18T12:24:09.966261+0000","last_active":"2023-10-18T12:26:15.211073+0000","last_peered":"2023-10-18T12:26:15.211073+0000","last_clean":"2023-10-18T12:26:15.211073+0000","last_became_active":"2023-10-18T12:24:09.965605+0000","last_became_peered":"2023-10-18T12:24:09.965605+0000","last_unstale":"2023-10-18T12:26:15.211073+0000","last_undegraded":"2023-10-18T12:26:15.211073+0000","last_fullsized":"2023-10-18T12:26:15.211073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:17.336269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T12:26:16.635968+0000","last_change":"2023-10-18T12:24:09.987361+0000","last_active":"2023-10-18T12:26:16.635968+0000","last_peered":"2023-10-18T12:26:16.635968+0000","last_clean":"2023-10-18T12:26:16.635968+0000","last_became_active":"2023-10-18T12:24:09.984671+0000","last_became_peered":"2023-10-18T12:24:09.984671+0000","last_unstale":"2023-10-18T12:26:16.635968+0000","last_undegraded":"2023-10-18T12:26:16.635968+0000","last_fullsized":"2023-10-18T12:26:16.635968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:32.050067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214790+0000","last_change":"2023-10-18T12:24:09.977381+0000","last_active":"2023-10-18T12:26:15.214790+0000","last_peered":"2023-10-18T12:26:15.214790+0000","last_clean":"2023-10-18T12:26:15.214790+0000","last_became_active":"2023-10-18T12:24:09.977095+0000","last_became_peered":"2023-10-18T12:24:09.977095+0000","last_unstale":"2023-10-18T12:26:15.214790+0000","last_undegraded":"2023-10-18T12:26:15.214790+0000","last_fullsized":"2023-10-18T12:26:15.214790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:07.929904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.210102+0000","last_change":"2023-10-18T12:24:09.974058+0000","last_active":"2023-10-18T12:26:15.210102+0000","last_peered":"2023-10-18T12:26:15.210102+0000","last_clean":"2023-10-18T12:26:15.210102+0000","last_became_active":"2023-10-18T12:24:09.972955+0000","last_became_peered":"2023-10-18T12:24:09.972955+0000","last_unstale":"2023-10-18T12:26:15.210102+0000","last_undegraded":"2023-10-18T12:26:15.210102+0000","last_fullsized":"2023-10-18T12:26:15.210102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:19.231320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.214765+0000","last_change":"2023-10-18T12:24:09.987139+0000","last_active":"2023-10-18T12:26:15.214765+0000","last_peered":"2023-10-18T12:26:15.214765+0000","last_clean":"2023-10-18T12:26:15.214765+0000","last_became_active":"2023-10-18T12:24:09.986804+0000","last_became_peered":"2023-10-18T12:24:09.986804+0000","last_unstale":"2023-10-18T12:26:15.214765+0000","last_undegraded":"2023-10-18T12:26:15.214765+0000","last_fullsized":"2023-10-18T12:26:15.214765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:25:26.468530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.422543+0000","last_change":"2023-10-18T12:24:09.976122+0000","last_active":"2023-10-18T12:26:15.422543+0000","last_peered":"2023-10-18T12:26:15.422543+0000","last_clean":"2023-10-18T12:26:15.422543+0000","last_became_active":"2023-10-18T12:24:09.975724+0000","last_became_peered":"2023-10-18T12:24:09.975724+0000","last_unstale":"2023-10-18T12:26:15.422543+0000","last_undegraded":"2023-10-18T12:26:15.422543+0000","last_fullsized":"2023-10-18T12:26:15.422543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:05.845505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T12:26:15.212687+0000","last_change":"2023-10-18T12:24:09.974426+0000","last_active":"2023-10-18T12:26:15.212687+0000","last_peered":"2023-10-18T12:26:15.212687+0000","last_clean":"2023-10-18T12:26:15.212687+0000","last_became_active":"2023-10-18T12:24:09.974099+0000","last_became_peered":"2023-10-18T12:24:09.974099+0000","last_unstale":"2023-10-18T12:26:15.212687+0000","last_undegraded":"2023-10-18T12:26:15.212687+0000","last_fullsized":"2023-10-18T12:26:15.212687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:24:08.948983+0000","last_clean_scrub_stamp":"2023-10-18T12:24:08.948983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:15.152176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67627,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":232,"num_read_kb":184,"num_write":82,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":202881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":532844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":1.147,"15min":1.147},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88400000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":1.238,"15min":1.238},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93400000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75600000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":1.661,"15min":1.661},"last":0.73499999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90400000000000003,"5min":1.361,"15min":1.361},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.442,"15min":0.442},"max":{"1min":0.84099999999999997,"5min":1,"15min":1},"last":0.64300000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.622},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.752}]},{"osd":9,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.91100000000000003,"5min":1.726,"15min":1.726},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.012,"5min":1.401,"15min":1.401},"last":0.69999999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89000000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.76700000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1186117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.54500000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.497}]},{"osd":2,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63300000000000001,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.999,"15min":0.999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.66600000000000004,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72499999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84599999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.876,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.77600000000000002,"5min":1.099,"15min":1.099},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82799999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.97499999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90400000000000003,"5min":1.071,"15min":1.071},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89400000000000002,"5min":1.044,"15min":1.044},"last":0.76400000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80500000000000005,"15min":0.83499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.91700000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90400000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.81299999999999994}]}]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":537500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":3.9470000000000001,"5min":3.9470000000000001,"15min":3.9470000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0089999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.753},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.622}]},{"osd":3,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.71199999999999997,"15min":0.753},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.38,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.86199999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1739999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.90400000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1519999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.91500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.77300000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.94599999999999995,"15min":0.95199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":6.2220000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89200000000000002,"15min":0.90900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.83499999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.89100000000000001},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.875},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2430000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92700000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.92200000000000004,"15min":0.95199999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.995,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.97099999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":533404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85499999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.878}]},{"osd":2,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.34899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.86699999999999999},"min":{"1min":0.56200000000000006,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.79000000000000004},"min":{"1min":0.51300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":5.5010000000000003,"15min":5.5010000000000003},"last":0.74099999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":1.117,"15min":1.117},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.629,"5min":0.503,"15min":0.503},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.325,"15min":1.325},"last":0.89500000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.03,"5min":1.272,"15min":1.272},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76600000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0149999999999999,"5min":1.044,"15min":1.044},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.72399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0640000000000001,"5min":1.26,"15min":1.26},"last":0.95199999999999996}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.68300000000000005},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76800000000000002,"15min":0.72799999999999998},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.107,"15min":1.107},"last":1.0089999999999999}]},{"osd":11,"last update":"Wed Oct 18 12:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0209999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.98299999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1127868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.433,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.432},{"interface":"front","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.309,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.55300000000000005},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.999,"15min":0.999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68400000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.137,"15min":1.137},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81299999999999994,"5min":1.071,"15min":1.071},"last":0.69799999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.625,"5min":0.433,"15min":0.433},"max":{"1min":0.92800000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.65600000000000003}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.98199999999999998,"5min":1.02,"15min":1.02},"last":0.83599999999999997}]},{"osd":10,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95399999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56399999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79700000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83399999999999996},"min":{"1min":0.64500000000000002,"5min":0.63,"15min":0.63},"max":{"1min":0.97099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.80400000000000005},"min":{"1min":0.63600000000000001,"5min":0.62,"15min":0.62},"max":{"1min":0.94599999999999995,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.67100000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":528726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.53000000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.377},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.499}]},{"osd":2,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.68799999999999994,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.9199999999999999,"5min":6.9199999999999999,"15min":6.9199999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.026,"5min":0.81899999999999995,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":7.2270000000000003,"5min":7.2270000000000003,"15min":7.2270000000000003},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.016,"5min":1.234,"15min":1.234},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.996}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74299999999999999,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.28,"5min":1.849,"15min":1.849},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.68000000000000005}]},{"osd":8,"last update":"Wed Oct 18 12:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.88,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":7.7400000000000002,"5min":7.7400000000000002,"15min":7.7400000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.753}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.90800000000000003,"15min":0.88900000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.4969999999999999,"5min":7.4969999999999999,"15min":7.4969999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.66300000000000003}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95499999999999996,"15min":0.90300000000000002},"min":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3240000000000001,"5min":7.9290000000000003,"15min":7.9290000000000003},"last":0.90300000000000002}]},{"osd":11,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":8.1280000000000001,"5min":8.1280000000000001,"15min":8.1280000000000001},"last":0.98299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":595555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.161,"15min":1.161},"last":0.624},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94699999999999995,"5min":1.454,"15min":1.454},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.57299999999999995},"min":{"1min":0.52000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88500000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71899999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.315,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.42899999999999999},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76600000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.755},{"interface":"front","average":{"1min":0.749,"5min":0.68500000000000005,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.999,"5min":1.298,"15min":1.298},"last":0.875}]},{"osd":6,"last update":"Wed Oct 18 12:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81599999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82599999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.61399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.94599999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.98799999999999999}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.52700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89400000000000002,"5min":1.016,"15min":1.016},"last":0.95899999999999996}]},{"osd":11,"last update":"Wed Oct 18 12:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76900000000000002,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.91800000000000004,"15min":0.92500000000000004},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":1.0029999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":528816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.65200000000000002,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.30299999999999999},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65600000000000003,"15min":0.55200000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.32400000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.60699999999999998},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1479999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.024,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.65800000000000003,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.69699999999999995,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66800000000000004},"min":{"1min":0.497,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93700000000000006,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.879},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.66200000000000003},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.82399999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69899999999999995,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.751,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.96599999999999997,"15min":0.998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2949999999999999,"5min":3.0489999999999999,"15min":3.0489999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.874,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.044,"5min":3.222,"15min":3.222},"last":0.77400000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.095,"5min":0.85699999999999998,"15min":0.81499999999999995},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":2.984,"5min":2.984,"15min":2.984},"last":1.2410000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86799999999999999,"15min":0.84599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.86699999999999999,"15min":0.85699999999999998},"min":{"1min":0.61199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.73,"5min":3.73,"15min":3.73},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91700000000000004,"15min":0.89300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.89300000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.03,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":0.97599999999999998}]},{"osd":11,"last update":"Wed Oct 18 12:25:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93899999999999995,"15min":0.92400000000000004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.008},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":0.95899999999999996}]}]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":595316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.26100000000000001},"max":{"1min":0.92100000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60399999999999998,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69099999999999995}]},{"osd":3,"last update":"Wed Oct 18 12:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.54600000000000004},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.748}]},{"osd":5,"last update":"Wed Oct 18 12:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.38,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.876,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.52100000000000002},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.88800000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.58399999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79800000000000004,"15min":0.79400000000000004},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.96499999999999997}]},{"osd":8,"last update":"Wed Oct 18 12:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60599999999999998},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.95199999999999996,"5min":1.367,"15min":1.367},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.46300000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":1.0349999999999999}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76500000000000001,"15min":0.74099999999999999},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.254,"15min":1.254},"last":0.872}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.74299999999999999,"15min":0.71299999999999997},"min":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":1.079},{"interface":"front","average":{"1min":0.878,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0169999999999999}]},{"osd":11,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.64600000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.002},{"interface":"front","average":{"1min":0.877,"5min":0.80400000000000005,"15min":0.79200000000000004},"min":{"1min":0.63,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.1060000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":528830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.51400000000000001,"15min":0.436},"min":{"1min":0.32000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7880000000000003,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.52700000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.73599999999999999,"15min":0.63600000000000001},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.68999999999999995,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":5.0579999999999998,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.73699999999999999,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.747}]},{"osd":4,"last update":"Wed Oct 18 12:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.085,"5min":0.77800000000000002,"15min":0.70599999999999996},"min":{"1min":0.60999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.78700000000000003,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91700000000000004,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.752,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.87,"5min":3.806,"15min":3.806},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.65200000000000002,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.504}]},{"osd":7,"last update":"Wed Oct 18 12:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.752,"15min":0.69499999999999995},"min":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.83299999999999996,"15min":0.753},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.984,"5min":4.984,"15min":4.984},"last":0.93999999999999995}]},{"osd":8,"last update":"Wed Oct 18 12:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Oct 18 12:25:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3789999999999996,"5min":5.3789999999999996,"15min":5.3789999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.89600000000000002,"15min":0.86199999999999999},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":0.89400000000000002}]},{"osd":10,"last update":"Wed Oct 18 12:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93799999999999994,"15min":0.90500000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":4.9530000000000003},"last":0.92000000000000004}]},{"osd":11,"last update":"Wed Oct 18 12:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.016,"15min":1.052},"min":{"1min":0.61899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2030000000000001,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.111,"15min":1.155},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2170000000000001,"5min":4.8879999999999999,"15min":4.8879999999999999},"last":0.95999999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":513146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54800000000000004,"5min":0.621,"15min":0.621},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.57099999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.876,"5min":1.04,"15min":1.04},"last":0.64700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.55700000000000005},"min":{"1min":0.52200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":1.22,"15min":1.22},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.181,"15min":1.181},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.51900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94599999999999995,"5min":1.077,"15min":1.077},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52800000000000002}]},{"osd":8,"last update":"Wed Oct 18 12:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Oct 18 12:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80200000000000005,"15min":0.81100000000000005},"min":{"1min":0.65900000000000003,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.02,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.55700000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72699999999999998}]},{"osd":10,"last update":"Wed Oct 18 12:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed Oct 18 12:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.996,"5min":0.997,"15min":0.997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85599999999999998,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1123041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76200000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":0.878},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.38200000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435}]},{"osd":4,"last update":"Wed Oct 18 12:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.47299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89200000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67600000000000005,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":0.999},"last":0.754},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed Oct 18 12:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63500000000000001}]},{"osd":9,"last update":"Wed Oct 18 12:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.752},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":4.548,"15min":4.548},"last":0.57399999999999995}]},{"osd":10,"last update":"Wed Oct 18 12:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.745,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73999999999999999,"15min":0.70799999999999996},"min":{"1min":0.60699999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:26:22.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:26:22.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 -- 172.21.15.70:0/1184499325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 msgr2=0x7f0508100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 --2- 172.21.15.70:0/1184499325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508100b20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009990 tx=0x7f04f0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 -- 172.21.15.70:0/1184499325 shutdown_connections 2023-10-18T12:26:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 --2- 172.21.15.70:0/1184499325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0508101060 0x7f050810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 --2- 172.21.15.70:0/1184499325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05081066b0 0x7f0508106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 --2- 172.21.15.70:0/1184499325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508100b20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 -- 172.21.15.70:0/1184499325 >> 172.21.15.70:0/1184499325 conn(0x7f05080fc170 msgr2=0x7f05080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 -- 172.21.15.70:0/1184499325 shutdown_connections 2023-10-18T12:26:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.585+0000 7f050e7ff700 1 -- 172.21.15.70:0/1184499325 wait complete. 2023-10-18T12:26:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.586+0000 7f050e7ff700 1 Processor -- start 2023-10-18T12:26:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 -- start start 2023-10-18T12:26:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0508101060 0x7f0508199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05081066b0 0x7f050819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050819cb40 con 0x7f05081006a0 2023-10-18T12:26:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050819ccb0 con 0x7f0508101060 2023-10-18T12:26:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f050e7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050819ce70 con 0x7f05081066b0 2023-10-18T12:26:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.587+0000 7f0507fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55436/0 (socket says 172.21.15.70:55436) 2023-10-18T12:26:22.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 -- 172.21.15.70:0/3854230286 learned_addr learned my addr 172.21.15.70:0/3854230286 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:22.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f04fffff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0508101060 0x7f0508199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f050cd9c700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05081066b0 0x7f050819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 -- 172.21.15.70:0/3854230286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05081066b0 msgr2=0x7f050819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05081066b0 0x7f050819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 -- 172.21.15.70:0/3854230286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0508101060 msgr2=0x7f0508199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0508101060 0x7f0508199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 -- 172.21.15.70:0/3854230286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00095f0 con 0x7f05081006a0 2023-10-18T12:26:22.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0507fff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508198d10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f04f800e9a0 tx=0x7f04f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8009790 con 0x7f05081006a0 2023-10-18T12:26:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f800cbd0 con 0x7f05081006a0 2023-10-18T12:26:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050819d150 con 0x7f05081006a0 2023-10-18T12:26:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.588+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8018680 con 0x7f05081006a0 2023-10-18T12:26:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.589+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05081a5ab0 con 0x7f05081006a0 2023-10-18T12:26:22.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.589+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0508101de0 con 0x7f05081006a0 2023-10-18T12:26:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.591+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f04f800cd40 con 0x7f05081006a0 2023-10-18T12:26:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.593+0000 7f0505ffb700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04f40a14d0 0x7f04f40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.593+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f04f8014070 con 0x7f05081006a0 2023-10-18T12:26:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.593+0000 7f04fffff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04f40a14d0 0x7f04f40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.593+0000 7f04fffff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04f40a14d0 0x7f04f40a3990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f050819a220 tx=0x7f04f000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.596+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f04f8052100 con 0x7f05081006a0 2023-10-18T12:26:22.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.801+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f050819a070 con 0x7f05081006a0 2023-10-18T12:26:22.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.803+0000 7f0505ffb700 1 -- 172.21.15.70:0/3854230286 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24904 (secure 0 0 0) 0x7f04f80965d0 con 0x7f05081006a0 2023-10-18T12:26:22.806 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:22.807 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":93,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:26:16.208557+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:26:22.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.805+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04f40a14d0 msgr2=0x7f04f40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.805+0000 7f050e7ff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04f40a14d0 0x7f04f40a3990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f050819a220 tx=0x7f04f000b260 comp rx=0 tx=0).stop 2023-10-18T12:26:22.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.805+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 msgr2=0x7f0508198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:22.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.805+0000 7f050e7ff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508198d10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f04f800e9a0 tx=0x7f04f800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 shutdown_connections 2023-10-18T12:26:22.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05081066b0 0x7f050819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05081006a0 0x7f0508198d10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04f40a14d0 0x7f04f40a3990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 --2- 172.21.15.70:0/3854230286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0508101060 0x7f0508199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 >> 172.21.15.70:0/3854230286 conn(0x7f05080fc170 msgr2=0x7f0508072e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 shutdown_connections 2023-10-18T12:26:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:22.806+0000 7f050e7ff700 1 -- 172.21.15.70:0/3854230286 wait complete. 2023-10-18T12:26:22.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:26:22.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:26:23.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 -- 172.21.15.70:0/1093635993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 msgr2=0x7fba2c10d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 --2- 172.21.15.70:0/1093635993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 0x7fba2c10d5e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba24009f90 tx=0x7fba2400b330 comp rx=0 tx=0).stop 2023-10-18T12:26:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 -- 172.21.15.70:0/1093635993 shutdown_connections 2023-10-18T12:26:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 --2- 172.21.15.70:0/1093635993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 0x7fba2c10d5e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 --2- 172.21.15.70:0/1093635993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba2c106720 0x7fba2c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 --2- 172.21.15.70:0/1093635993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c100710 0x7fba2c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 -- 172.21.15.70:0/1093635993 >> 172.21.15.70:0/1093635993 conn(0x7fba2c0fc200 msgr2=0x7fba2c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:23.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 -- 172.21.15.70:0/1093635993 shutdown_connections 2023-10-18T12:26:23.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.027+0000 7fba331ff700 1 -- 172.21.15.70:0/1093635993 wait complete. 2023-10-18T12:26:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.028+0000 7fba331ff700 1 Processor -- start 2023-10-18T12:26:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.028+0000 7fba331ff700 1 -- start start 2023-10-18T12:26:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba331ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba2c100710 0x7fba2c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba331ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 0x7fba2c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba331ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 0x7fba2c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba331ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19cb30 con 0x7fba2c106720 2023-10-18T12:26:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba331ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19cca0 con 0x7fba2c100710 2023-10-18T12:26:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba331ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19ce60 con 0x7fba2c1010d0 2023-10-18T12:26:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 0x7fba2c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 0x7fba2c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55460/0 (socket says 172.21.15.70:55460) 2023-10-18T12:26:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 -- 172.21.15.70:0/2789145070 learned_addr learned my addr 172.21.15.70:0/2789145070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba30f9b700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba2c100710 0x7fba2c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba2bfff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 0x7fba2c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 -- 172.21.15.70:0/2789145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 msgr2=0x7fba2c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 0x7fba2c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 -- 172.21.15.70:0/2789145070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba2c100710 msgr2=0x7fba2c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.029+0000 7fba3179c700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba2c100710 0x7fba2c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba3179c700 1 -- 172.21.15.70:0/2789145070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009c40 con 0x7fba2c106720 2023-10-18T12:26:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba3179c700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 0x7fba2c19c530 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fba24008fb0 tx=0x7fba240094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2400e040 con 0x7fba2c106720 2023-10-18T12:26:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c19d0e0 con 0x7fba2c106720 2023-10-18T12:26:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba2400bbf0 con 0x7fba2c106720 2023-10-18T12:26:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24005040 con 0x7fba2c106720 2023-10-18T12:26:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.030+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c1a5aa0 con 0x7fba2c106720 2023-10-18T12:26:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.031+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c101e50 con 0x7fba2c106720 2023-10-18T12:26:23.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.033+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fba2401d470 con 0x7fba2c106720 2023-10-18T12:26:23.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.034+0000 7fba29ffb700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fba1c0a14d0 0x7fba1c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.035+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fba240ce9f0 con 0x7fba2c106720 2023-10-18T12:26:23.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.035+0000 7fba30f9b700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fba1c0a14d0 0x7fba1c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.035+0000 7fba30f9b700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fba1c0a14d0 0x7fba1c0a3990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba20009be0 tx=0x7fba20009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:23.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.036+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fba24097c60 con 0x7fba2c106720 2023-10-18T12:26:23.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:22 smithi070 ceph-mon[98639]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 895 B/s wr, 2 op/s 2023-10-18T12:26:23.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:22 smithi070 ceph-mon[98639]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:23.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:22 smithi152 ceph-mon[134767]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 895 B/s wr, 2 op/s 2023-10-18T12:26:23.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:22 smithi152 ceph-mon[134767]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:23.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:22 smithi186 ceph-mon[135253]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 895 B/s wr, 2 op/s 2023-10-18T12:26:23.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:22 smithi186 ceph-mon[135253]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.237+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba2c19a050 con 0x7fba2c106720 2023-10-18T12:26:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.237+0000 7fba29ffb700 1 -- 172.21.15.70:0/2789145070 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba2409b280 con 0x7fba2c106720 2023-10-18T12:26:23.241 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:26:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fba1c0a14d0 msgr2=0x7fba1c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fba1c0a14d0 0x7fba1c0a3990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba20009be0 tx=0x7fba20009260 comp rx=0 tx=0).stop 2023-10-18T12:26:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 msgr2=0x7fba2c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 0x7fba2c19c530 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fba24008fb0 tx=0x7fba240094f0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 shutdown_connections 2023-10-18T12:26:23.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba2c106720 0x7fba2c19c530 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fba1c0a14d0 0x7fba1c0a3990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba2c100710 0x7fba2c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 --2- 172.21.15.70:0/2789145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba2c1010d0 0x7fba2c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 >> 172.21.15.70:0/2789145070 conn(0x7fba2c0fc200 msgr2=0x7fba2c072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:23.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 shutdown_connections 2023-10-18T12:26:23.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.240+0000 7fba331ff700 1 -- 172.21.15.70:0/2789145070 wait complete. 2023-10-18T12:26:23.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:26:23.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 -- 172.21.15.70:0/1449600747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668072c20 msgr2=0x7f8668073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 --2- 172.21.15.70:0/1449600747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668072c20 0x7f8668073000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8664009990 tx=0x7f8664009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 -- 172.21.15.70:0/1449600747 shutdown_connections 2023-10-18T12:26:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 --2- 172.21.15.70:0/1449600747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f866810cf60 0x7f866810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 --2- 172.21.15.70:0/1449600747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668072c20 0x7f8668073000 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 --2- 172.21.15.70:0/1449600747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8668073540 0x7f866810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 -- 172.21.15.70:0/1449600747 >> 172.21.15.70:0/1449600747 conn(0x7f86680fc1b0 msgr2=0x7f86680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 -- 172.21.15.70:0/1449600747 shutdown_connections 2023-10-18T12:26:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.461+0000 7f866ffff700 1 -- 172.21.15.70:0/1449600747 wait complete. 2023-10-18T12:26:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.462+0000 7f866ffff700 1 Processor -- start 2023-10-18T12:26:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.462+0000 7f866ffff700 1 -- start start 2023-10-18T12:26:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8668072c20 0x7f8668198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 0x7f8668199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f866810cf60 0x7f866819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866819dd00 con 0x7f8668073540 2023-10-18T12:26:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866819de70 con 0x7f866810cf60 2023-10-18T12:26:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866819e030 con 0x7f8668072c20 2023-10-18T12:26:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 0x7f8668199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 0x7f8668199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55488/0 (socket says 172.21.15.70:55488) 2023-10-18T12:26:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 -- 172.21.15.70:0/2715004263 learned_addr learned my addr 172.21.15.70:0/2715004263 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866dd9b700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8668072c20 0x7f8668198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866e59c700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f866810cf60 0x7f866819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 -- 172.21.15.70:0/2715004263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8668072c20 msgr2=0x7f8668198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8668072c20 0x7f8668198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 -- 172.21.15.70:0/2715004263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f866810cf60 msgr2=0x7f866819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f866810cf60 0x7f866819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866d59a700 1 -- 172.21.15.70:0/2715004263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640095f0 con 0x7f8668073540 2023-10-18T12:26:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.463+0000 7f866dd9b700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8668072c20 0x7f8668198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.464+0000 7f866d59a700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 0x7f8668199320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9a0 tx=0x7f865800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.464+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658009790 con 0x7f8668073540 2023-10-18T12:26:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.464+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865800cbd0 con 0x7f8668073540 2023-10-18T12:26:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.464+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866819e310 con 0x7f8668073540 2023-10-18T12:26:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.464+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658018680 con 0x7f8668073540 2023-10-18T12:26:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.464+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86681a5b50 con 0x7f8668073540 2023-10-18T12:26:23.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.465+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8668108170 con 0x7f8668073540 2023-10-18T12:26:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.467+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f865800cd40 con 0x7f8668073540 2023-10-18T12:26:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.469+0000 7f865effd700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f86540a14d0 0x7f86540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.469+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8658014070 con 0x7f8668073540 2023-10-18T12:26:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.469+0000 7f866dd9b700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f86540a14d0 0x7f86540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.470+0000 7f866dd9b700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f86540a14d0 0x7f86540a3990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f866400b2e0 tx=0x7f8664005fb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.473+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8658052100 con 0x7f8668073540 2023-10-18T12:26:23.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.678+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f866819a0e0 con 0x7f8668073540 2023-10-18T12:26:23.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.679+0000 7f865effd700 1 -- 172.21.15.70:0/2715004263 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8658096660 con 0x7f8668073540 2023-10-18T12:26:23.682 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f86540a14d0 msgr2=0x7f86540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f86540a14d0 0x7f86540a3990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f866400b2e0 tx=0x7f8664005fb0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 msgr2=0x7f8668199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 0x7f8668199320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9a0 tx=0x7f865800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 shutdown_connections 2023-10-18T12:26:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f86540a14d0 0x7f86540a3990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f866810cf60 0x7f866819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8668072c20 0x7f8668198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 --2- 172.21.15.70:0/2715004263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8668073540 0x7f8668199320 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 >> 172.21.15.70:0/2715004263 conn(0x7f86680fc1b0 msgr2=0x7f86680fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 shutdown_connections 2023-10-18T12:26:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.681+0000 7f866ffff700 1 -- 172.21.15.70:0/2715004263 wait complete. 2023-10-18T12:26:23.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:26:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 -- 172.21.15.70:0/2436335027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20106630 msgr2=0x7f6e20106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2436335027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20106630 0x7f6e20106a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009990 tx=0x7f6e1c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 -- 172.21.15.70:0/2436335027 shutdown_connections 2023-10-18T12:26:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2436335027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e20100fe0 0x7f6e2010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2436335027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20106630 0x7f6e20106a10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2436335027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e20100620 0x7f6e20100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 -- 172.21.15.70:0/2436335027 >> 172.21.15.70:0/2436335027 conn(0x7f6e200fc0f0 msgr2=0x7f6e200fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 -- 172.21.15.70:0/2436335027 shutdown_connections 2023-10-18T12:26:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.901+0000 7f6e279ff700 1 -- 172.21.15.70:0/2436335027 wait complete. 2023-10-18T12:26:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 Processor -- start 2023-10-18T12:26:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 -- start start 2023-10-18T12:26:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 0x7f6e2019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e20100fe0 0x7f6e2019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e20106630 0x7f6e20198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e20199120 con 0x7f6e20100620 2023-10-18T12:26:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e20199290 con 0x7f6e20106630 2023-10-18T12:26:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.902+0000 7f6e279ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e20199450 con 0x7f6e20100fe0 2023-10-18T12:26:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 0x7f6e2019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 0x7f6e2019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55518/0 (socket says 172.21.15.70:55518) 2023-10-18T12:26:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 -- 172.21.15.70:0/2463144409 learned_addr learned my addr 172.21.15.70:0/2463144409 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e25f9c700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e20106630 0x7f6e20198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e24f9a700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e20100fe0 0x7f6e2019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 -- 172.21.15.70:0/2463144409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e20100fe0 msgr2=0x7f6e2019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e20100fe0 0x7f6e2019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 -- 172.21.15.70:0/2463144409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e20106630 msgr2=0x7f6e20198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e20106630 0x7f6e20198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 -- 172.21.15.70:0/2463144409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0095f0 con 0x7f6e20100620 2023-10-18T12:26:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e2579b700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 0x7f6e2019e990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.903+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c01d070 con 0x7f6e20100620 2023-10-18T12:26:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.904+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e1c022470 con 0x7f6e20100620 2023-10-18T12:26:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.904+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e201996d0 con 0x7f6e20100620 2023-10-18T12:26:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.904+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c017650 con 0x7f6e20100620 2023-10-18T12:26:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.904+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e201a5980 con 0x7f6e20100620 2023-10-18T12:26:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.905+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e20101d60 con 0x7f6e20100620 2023-10-18T12:26:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.907+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6e1c0225e0 con 0x7f6e20100620 2023-10-18T12:26:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.908+0000 7f6e167fc700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e0c0a1540 0x7f6e0c0a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:23.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.908+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6e1c0ce380 con 0x7f6e20100620 2023-10-18T12:26:23.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.908+0000 7f6e24f9a700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e0c0a1540 0x7f6e0c0a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:23.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.909+0000 7f6e24f9a700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e0c0a1540 0x7f6e0c0a3a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e20199c80 tx=0x7f6e10009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:23.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:23.912+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e1c05d100 con 0x7f6e20100620 2023-10-18T12:26:24.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:23 smithi070 ceph-mon[98639]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:24.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:23 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3854230286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:24.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:23 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2789145070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:23 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2715004263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.110+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e20110560 con 0x7f6e20100620 2023-10-18T12:26:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.111+0000 7f6e167fc700 1 -- 172.21.15.70:0/2463144409 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e1c0975f0 con 0x7f6e20100620 2023-10-18T12:26:24.114 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:26:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e0c0a1540 msgr2=0x7f6e0c0a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e0c0a1540 0x7f6e0c0a3a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e20199c80 tx=0x7f6e10009260 comp rx=0 tx=0).stop 2023-10-18T12:26:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 msgr2=0x7f6e2019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 0x7f6e2019e990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c005740 comp rx=0 tx=0).stop 2023-10-18T12:26:24.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 shutdown_connections 2023-10-18T12:26:24.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e0c0a1540 0x7f6e0c0a3a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e20106630 0x7f6e20198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e20100620 0x7f6e2019e990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 --2- 172.21.15.70:0/2463144409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e20100fe0 0x7f6e2019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 >> 172.21.15.70:0/2463144409 conn(0x7f6e200fc0f0 msgr2=0x7f6e2010bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:24.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 shutdown_connections 2023-10-18T12:26:24.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.113+0000 7f6e279ff700 1 -- 172.21.15.70:0/2463144409 wait complete. 2023-10-18T12:26:24.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:26:24.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:23 smithi152 ceph-mon[134767]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:23 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3854230286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:23 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2789145070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:23 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2715004263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:23 smithi186 ceph-mon[135253]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T12:26:24.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:23 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3854230286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:24.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:23 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2789145070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:23 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2715004263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 -- 172.21.15.70:0/911329960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c106730 msgr2=0x7fcd8c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 --2- 172.21.15.70:0/911329960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c106730 0x7fcd8c106b10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009990 tx=0x7fcd7c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 -- 172.21.15.70:0/911329960 shutdown_connections 2023-10-18T12:26:24.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 --2- 172.21.15.70:0/911329960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 --2- 172.21.15.70:0/911329960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c106730 0x7fcd8c106b10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 --2- 172.21.15.70:0/911329960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c100720 0x7fcd8c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.333+0000 7fcd917ff700 1 -- 172.21.15.70:0/911329960 >> 172.21.15.70:0/911329960 conn(0x7fcd8c0fc1b0 msgr2=0x7fcd8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.334+0000 7fcd917ff700 1 -- 172.21.15.70:0/911329960 shutdown_connections 2023-10-18T12:26:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.334+0000 7fcd917ff700 1 -- 172.21.15.70:0/911329960 wait complete. 2023-10-18T12:26:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 Processor -- start 2023-10-18T12:26:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 -- start start 2023-10-18T12:26:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c100720 0x7fcd8c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c106730 0x7fcd8c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c199250 con 0x7fcd8c1010e0 2023-10-18T12:26:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1993c0 con 0x7fcd8c100720 2023-10-18T12:26:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd917ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c199580 con 0x7fcd8c106730 2023-10-18T12:26:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd8a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c100720 0x7fcd8c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd8a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55532/0 (socket says 172.21.15.70:55532) 2023-10-18T12:26:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/1016430286 learned_addr learned my addr 172.21.15.70:0/1016430286 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.335+0000 7fcd8b7fe700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c106730 0x7fcd8c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/1016430286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c106730 msgr2=0x7fcd8c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd8a7fc700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c106730 0x7fcd8c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/1016430286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c100720 msgr2=0x7fcd8c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd8a7fc700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c100720 0x7fcd8c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/1016430286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0095f0 con 0x7fcd8c1010e0 2023-10-18T12:26:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd8a7fc700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c19efb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000e9a0 tx=0x7fcd8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80009790 con 0x7fcd8c1010e0 2023-10-18T12:26:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd80010430 con 0x7fcd8c1010e0 2023-10-18T12:26:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c199860 con 0x7fcd8c1010e0 2023-10-18T12:26:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80018680 con 0x7fcd8c1010e0 2023-10-18T12:26:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.336+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1a5a60 con 0x7fcd8c1010e0 2023-10-18T12:26:24.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.337+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c101e60 con 0x7fcd8c1010e0 2023-10-18T12:26:24.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.339+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcd800105a0 con 0x7fcd8c1010e0 2023-10-18T12:26:24.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.341+0000 7fcd73fff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd740a14d0 0x7fcd740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.341+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcd80014070 con 0x7fcd8c1010e0 2023-10-18T12:26:24.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.341+0000 7fcd8affd700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd740a14d0 0x7fcd740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.342+0000 7fcd8affd700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd740a14d0 0x7fcd740a3990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00b2e0 tx=0x7fcd7c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.345+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcd80052100 con 0x7fcd8c1010e0 2023-10-18T12:26:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.545+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcd8c19a020 con 0x7fcd8c1010e0 2023-10-18T12:26:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.545+0000 7fcd73fff700 1 -- 172.21.15.70:0/1016430286 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcd80096570 con 0x7fcd8c1010e0 2023-10-18T12:26:24.549 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:26:24.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd740a14d0 msgr2=0x7fcd740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd740a14d0 0x7fcd740a3990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00b2e0 tx=0x7fcd7c005fd0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 msgr2=0x7fcd8c19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c19efb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000e9a0 tx=0x7fcd8000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 shutdown_connections 2023-10-18T12:26:24.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c106730 0x7fcd8c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd740a14d0 0x7fcd740a3990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c100720 0x7fcd8c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 --2- 172.21.15.70:0/1016430286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c1010e0 0x7fcd8c19efb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 >> 172.21.15.70:0/1016430286 conn(0x7fcd8c0fc1b0 msgr2=0x7fcd8c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:24.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 shutdown_connections 2023-10-18T12:26:24.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.548+0000 7fcd917ff700 1 -- 172.21.15.70:0/1016430286 wait complete. 2023-10-18T12:26:24.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:26:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 -- 172.21.15.70:0/3461224125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 msgr2=0x7ff30c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 --2- 172.21.15.70:0/3461224125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c106a90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009990 tx=0x7ff2fc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 -- 172.21.15.70:0/3461224125 shutdown_connections 2023-10-18T12:26:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 --2- 172.21.15.70:0/3461224125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff30c101060 0x7ff30c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 --2- 172.21.15.70:0/3461224125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c106a90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 --2- 172.21.15.70:0/3461224125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 0x7ff30c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 -- 172.21.15.70:0/3461224125 >> 172.21.15.70:0/3461224125 conn(0x7ff30c0fc170 msgr2=0x7ff30c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:24.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 -- 172.21.15.70:0/3461224125 shutdown_connections 2023-10-18T12:26:24.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.767+0000 7ff3121ff700 1 -- 172.21.15.70:0/3461224125 wait complete. 2023-10-18T12:26:24.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.768+0000 7ff3121ff700 1 Processor -- start 2023-10-18T12:26:24.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 -- start start 2023-10-18T12:26:24.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 0x7ff30c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff30c101060 0x7ff30c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1991d0 con 0x7ff30c1066b0 2023-10-18T12:26:24.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c199340 con 0x7ff30c1006a0 2023-10-18T12:26:24.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff3121ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c199500 con 0x7ff30c101060 2023-10-18T12:26:24.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff30b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 0x7ff30c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55546/0 (socket says 172.21.15.70:55546) 2023-10-18T12:26:24.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff30bfff700 1 -- 172.21.15.70:0/1323551903 learned_addr learned my addr 172.21.15.70:0/1323551903 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:24.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff30b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 0x7ff30c19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:42724/0 (socket says 172.21.15.70:42724) 2023-10-18T12:26:24.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.769+0000 7ff30affd700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff30c101060 0x7ff30c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff30bfff700 1 -- 172.21.15.70:0/1323551903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff30c101060 msgr2=0x7ff30c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff30bfff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff30c101060 0x7ff30c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff30bfff700 1 -- 172.21.15.70:0/1323551903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 msgr2=0x7ff30c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff30bfff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 0x7ff30c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:24.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff30bfff700 1 -- 172.21.15.70:0/1323551903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc0095f0 con 0x7ff30c1066b0 2023-10-18T12:26:24.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff30bfff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c198b70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff30400ef30 tx=0x7ff30400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:24.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304018070 con 0x7ff30c1066b0 2023-10-18T12:26:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30400f040 con 0x7ff30c1066b0 2023-10-18T12:26:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c199780 con 0x7ff30c1066b0 2023-10-18T12:26:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400cd00 con 0x7ff30c1066b0 2023-10-18T12:26:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.770+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c1a59e0 con 0x7ff30c1066b0 2023-10-18T12:26:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.771+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c101de0 con 0x7ff30c1066b0 2023-10-18T12:26:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.775+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff304003a00 con 0x7ff30c1066b0 2023-10-18T12:26:24.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.776+0000 7ff308ff9700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2f40a14d0 0x7ff2f40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:24.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.776+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff3040cd620 con 0x7ff30c1066b0 2023-10-18T12:26:24.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.776+0000 7ff30b7fe700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2f40a14d0 0x7ff2f40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:24.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.777+0000 7ff30b7fe700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2f40a14d0 0x7ff2f40a3990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006010 tx=0x7ff2fc00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:24.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:24.778+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff3040968f0 con 0x7ff30c1066b0 2023-10-18T12:26:24.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:24 smithi070 ceph-mon[98639]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 851 B/s wr, 2 op/s 2023-10-18T12:26:24.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:24 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2463144409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:24.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:24 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1016430286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.029+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30c199fe0 con 0x7ff30c1066b0 2023-10-18T12:26:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.030+0000 7ff308ff9700 1 -- 172.21.15.70:0/1323551903 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7ff304099f10 con 0x7ff30c1066b0 2023-10-18T12:26:25.033 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:25.033 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:26:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2f40a14d0 msgr2=0x7ff2f40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2f40a14d0 0x7ff2f40a3990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006010 tx=0x7ff2fc00b260 comp rx=0 tx=0).stop 2023-10-18T12:26:25.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 msgr2=0x7ff30c198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c198b70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff30400ef30 tx=0x7ff30400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 shutdown_connections 2023-10-18T12:26:25.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c1066b0 0x7ff30c198b70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff2f40a14d0 0x7ff2f40a3990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff30c1006a0 0x7ff30c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 --2- 172.21.15.70:0/1323551903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff30c101060 0x7ff30c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 >> 172.21.15.70:0/1323551903 conn(0x7ff30c0fc170 msgr2=0x7ff30c0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 shutdown_connections 2023-10-18T12:26:25.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.032+0000 7ff3121ff700 1 -- 172.21.15.70:0/1323551903 wait complete. 2023-10-18T12:26:25.039 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:26:25.054 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:26:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:24 smithi152 ceph-mon[134767]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 851 B/s wr, 2 op/s 2023-10-18T12:26:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:24 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2463144409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:24 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1016430286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:24 smithi186 ceph-mon[135253]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 851 B/s wr, 2 op/s 2023-10-18T12:26:25.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:24 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2463144409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:26:25.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:24 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1016430286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:26:25.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 -- 172.21.15.70:0/763725210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1066b0 msgr2=0x7f9c3c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 --2- 172.21.15.70:0/763725210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1066b0 0x7f9c3c106a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009990 tx=0x7f9c2c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 -- 172.21.15.70:0/763725210 shutdown_connections 2023-10-18T12:26:25.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 --2- 172.21.15.70:0/763725210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c3c101060 0x7f9c3c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 --2- 172.21.15.70:0/763725210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1066b0 0x7f9c3c106a90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 --2- 172.21.15.70:0/763725210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 -- 172.21.15.70:0/763725210 >> 172.21.15.70:0/763725210 conn(0x7f9c3c0fc170 msgr2=0x7f9c3c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 -- 172.21.15.70:0/763725210 shutdown_connections 2023-10-18T12:26:25.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.252+0000 7f9c425ff700 1 -- 172.21.15.70:0/763725210 wait complete. 2023-10-18T12:26:25.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.254+0000 7f9c425ff700 1 Processor -- start 2023-10-18T12:26:25.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.254+0000 7f9c425ff700 1 -- start start 2023-10-18T12:26:25.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c425ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c425ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c3c101060 0x7f9c3c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c425ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c3c1066b0 0x7f9c3c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c425ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1a4ea0 con 0x7f9c3c1066b0 2023-10-18T12:26:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c425ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1a5010 con 0x7f9c3c1006a0 2023-10-18T12:26:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c425ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c1a51d0 con 0x7f9c3c101060 2023-10-18T12:26:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:42738/0 (socket says 172.21.15.70:42738) 2023-10-18T12:26:25.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 -- 172.21.15.70:0/3738686486 learned_addr learned my addr 172.21.15.70:0/3738686486 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3b7fe700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c3c101060 0x7f9c3c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c40b9c700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c3c1066b0 0x7f9c3c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 -- 172.21.15.70:0/3738686486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c3c101060 msgr2=0x7f9c3c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c3c101060 0x7f9c3c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 -- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c3c1066b0 msgr2=0x7f9c3c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c3c1066b0 0x7f9c3c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.255+0000 7f9c3bfff700 1 -- 172.21.15.70:0/3738686486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0095f0 con 0x7f9c3c1006a0 2023-10-18T12:26:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.256+0000 7f9c3bfff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c072af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.256+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c01d070 con 0x7f9c3c1006a0 2023-10-18T12:26:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.256+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c1a5450 con 0x7f9c3c1006a0 2023-10-18T12:26:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.256+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2c022470 con 0x7f9c3c1006a0 2023-10-18T12:26:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.256+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c017650 con 0x7f9c3c1006a0 2023-10-18T12:26:25.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.256+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c1a5930 con 0x7f9c3c1006a0 2023-10-18T12:26:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.258+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c108c30 con 0x7f9c3c1006a0 2023-10-18T12:26:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.260+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9c2c0225e0 con 0x7f9c3c1006a0 2023-10-18T12:26:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.261+0000 7f9c397fa700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c240a14d0 0x7f9c240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.261+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9c2c0ce4b0 con 0x7f9c3c1006a0 2023-10-18T12:26:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.262+0000 7f9c3b7fe700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c240a14d0 0x7f9c240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.262+0000 7f9c3b7fe700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c240a14d0 0x7f9c240a3990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c074220 tx=0x7f9c30009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.266+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9c2c097720 con 0x7f9c3c1006a0 2023-10-18T12:26:25.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.479+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c3c101de0 con 0x7f9c3c1006a0 2023-10-18T12:26:25.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.481+0000 7f9c397fa700 1 -- 172.21.15.70:0/3738686486 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24904 (secure 0 0 0) 0x7f9c2c09ad40 con 0x7f9c3c1006a0 2023-10-18T12:26:25.484 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":93,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:26:16.208557+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:26:25.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c240a14d0 msgr2=0x7f9c240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c240a14d0 0x7f9c240a3990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c074220 tx=0x7f9c30009260 comp rx=0 tx=0).stop 2023-10-18T12:26:25.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 msgr2=0x7f9c3c072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c072af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c005740 comp rx=0 tx=0).stop 2023-10-18T12:26:25.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 shutdown_connections 2023-10-18T12:26:25.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c3c1066b0 0x7f9c3c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9c240a14d0 0x7f9c240a3990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c1006a0 0x7f9c3c072af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 --2- 172.21.15.70:0/3738686486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c3c101060 0x7f9c3c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 >> 172.21.15.70:0/3738686486 conn(0x7f9c3c0fc170 msgr2=0x7f9c3c0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 shutdown_connections 2023-10-18T12:26:25.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.484+0000 7f9c425ff700 1 -- 172.21.15.70:0/3738686486 wait complete. 2023-10-18T12:26:25.507 DEBUG:tasks.fwd_scrub:Yielding 2023-10-18T12:26:25.509 INFO:teuthology.run_tasks:Running task exec... 2023-10-18T12:26:25.521 INFO:teuthology.task.exec:Executing custom commands... 2023-10-18T12:26:25.521 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:26:25.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-10-18T12:26:25.522 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-18T12:26:25.523 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:26:25.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:26:25.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 -- 172.21.15.70:0/1808008681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 msgr2=0x7f98fc106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 --2- 172.21.15.70:0/1808008681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc106b10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0098f0 tx=0x7f98ec009c00 comp rx=0 tx=0).stop 2023-10-18T12:26:25.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 -- 172.21.15.70:0/1808008681 shutdown_connections 2023-10-18T12:26:25.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 --2- 172.21.15.70:0/1808008681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 0x7f98fc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 --2- 172.21.15.70:0/1808008681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc106b10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 --2- 172.21.15.70:0/1808008681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 0x7f98fc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.748+0000 7f99015ff700 1 -- 172.21.15.70:0/1808008681 >> 172.21.15.70:0/1808008681 conn(0x7f98fc0fc1b0 msgr2=0x7f98fc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.749+0000 7f99015ff700 1 -- 172.21.15.70:0/1808008681 shutdown_connections 2023-10-18T12:26:25.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.749+0000 7f99015ff700 1 -- 172.21.15.70:0/1808008681 wait complete. 2023-10-18T12:26:25.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.750+0000 7f99015ff700 1 Processor -- start 2023-10-18T12:26:25.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.750+0000 7f99015ff700 1 -- start start 2023-10-18T12:26:25.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f99015ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 0x7f98fc19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f99015ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 0x7f98fc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f99015ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f99015ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc197030 con 0x7f98fc1010e0 2023-10-18T12:26:25.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f99015ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1971a0 con 0x7f98fc106730 2023-10-18T12:26:25.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f99015ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc197360 con 0x7f98fc100720 2023-10-18T12:26:25.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f98fa7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 0x7f98fc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f98fb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f98fa7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 0x7f98fc19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55574/0 (socket says 172.21.15.70:55574) 2023-10-18T12:26:25.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f98fb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc1969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:42746/0 (socket says 172.21.15.70:42746) 2023-10-18T12:26:25.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.751+0000 7f98fa7fc700 1 -- 172.21.15.70:0/869280170 learned_addr learned my addr 172.21.15.70:0/869280170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98faffd700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 0x7f98fc19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98fb7fe700 1 -- 172.21.15.70:0/869280170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 msgr2=0x7f98fc19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 0x7f98fc19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98fb7fe700 1 -- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 msgr2=0x7f98fc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 0x7f98fc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98fb7fe700 1 -- 172.21.15.70:0/869280170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00095f0 con 0x7f98fc106730 2023-10-18T12:26:25.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98faffd700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 0x7f98fc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:26:25.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc1969d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f98f400efc0 tx=0x7f98f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98e3fff700 1 -- 172.21.15.70:0/869280170 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4018070 con 0x7f98fc106730 2023-10-18T12:26:25.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98e3fff700 1 -- 172.21.15.70:0/869280170 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f400f040 con 0x7f98fc106730 2023-10-18T12:26:25.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f98e3fff700 1 -- 172.21.15.70:0/869280170 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400cdf0 con 0x7f98fc106730 2023-10-18T12:26:25.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec009520 con 0x7f98fc106730 2023-10-18T12:26:25.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.752+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc1a36b0 con 0x7f98fc106730 2023-10-18T12:26:25.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.755+0000 7f98e3fff700 1 -- 172.21.15.70:0/869280170 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f98f4003a00 con 0x7f98fc106730 2023-10-18T12:26:25.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.757+0000 7f9914f4ab80 1 Processor -- start 2023-10-18T12:26:25.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.757+0000 7f98e3fff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98e40a14d0 0x7f98e40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 -- start start 2023-10-18T12:26:25.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f98e3fff700 1 -- 172.21.15.70:0/869280170 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f98f40cce30 con 0x7f98fc106730 2023-10-18T12:26:25.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98e40a14d0 0x7f98e40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 0x55853f9de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e2070 0x55853f9de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9deb70 0x55853f9f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f954760 con 0x55853f9e1c90 2023-10-18T12:26:25.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f93ace0 con 0x55853f9deb70 2023-10-18T12:26:25.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f938600 con 0x55853f9e2070 2023-10-18T12:26:25.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f98faffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 0x55853f9de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f98fa7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e2070 0x55853f9de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.758+0000 7f98faffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 0x55853f9de0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55594/0 (socket says 172.21.15.70:55594) 2023-10-18T12:26:25.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98faffd700 1 -- 172.21.15.70:0/322095237 learned_addr learned my addr 172.21.15.70:0/322095237 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9deb70 0x55853f9f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 -- 172.21.15.70:0/322095237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e2070 msgr2=0x55853f9de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e2070 0x55853f9de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 -- 172.21.15.70:0/322095237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 msgr2=0x55853f9de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 0x55853f9de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 -- 172.21.15.70:0/322095237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853f8f1470 con 0x55853f9deb70 2023-10-18T12:26:25.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98faffd700 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 0x55853f9de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:25.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98e40a14d0 0x7f98e40a3990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98fc197b90 tx=0x7f98f0011b70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9deb70 0x55853f9f5d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98f40ca480 tx=0x7f98f4099c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=837f4d4ba75c4cc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98e17fa700 1 -- 172.21.15.70:0/322095237 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40996c0 con 0x55853f9deb70 2023-10-18T12:26:25.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98e17fa700 1 -- 172.21.15.70:0/322095237 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f4004ad0 con 0x55853f9deb70 2023-10-18T12:26:25.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f98e17fa700 1 -- 172.21.15.70:0/322095237 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4097f30 con 0x55853f9deb70 2023-10-18T12:26:25.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f11b9fff700 1 -- 172.21.15.70:0/2562518188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 msgr2=0x7f11b4102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.759+0000 7f11b9fff700 1 --2- 172.21.15.70:0/2562518188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b4102d70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009990 tx=0x7f11a8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 -- 172.21.15.70:0/2562518188 shutdown_connections 2023-10-18T12:26:25.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 --2- 172.21.15.70:0/2562518188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b41032b0 0x7f11b410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 --2- 172.21.15.70:0/2562518188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b4108900 0x7f11b4108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 --2- 172.21.15.70:0/2562518188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b4102d70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 -- 172.21.15.70:0/2562518188 >> 172.21.15.70:0/2562518188 conn(0x7f11b40fe3a0 msgr2=0x7f11b41007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 -- 172.21.15.70:0/2562518188 shutdown_connections 2023-10-18T12:26:25.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f11b9fff700 1 -- 172.21.15.70:0/2562518188 wait complete. 2023-10-18T12:26:25.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 -- 172.21.15.70:0/322095237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9deb70 msgr2=0x55853f9f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9deb70 0x55853f9f5d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98f40ca480 tx=0x7f98f4099c00 comp rx=0 tx=0).stop 2023-10-18T12:26:25.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 -- 172.21.15.70:0/322095237 shutdown_connections 2023-10-18T12:26:25.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9deb70 0x55853f9f5d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e1c90 0x55853f9de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/322095237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e2070 0x55853f9de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 -- 172.21.15.70:0/322095237 >> 172.21.15.70:0/322095237 conn(0x55853f9557a0 msgr2=0x55853f9dbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 -- 172.21.15.70:0/322095237 shutdown_connections 2023-10-18T12:26:25.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.760+0000 7f9914f4ab80 1 -- 172.21.15.70:0/322095237 wait complete. 2023-10-18T12:26:25.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.761+0000 7f11b9fff700 1 Processor -- start 2023-10-18T12:26:25.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.761+0000 7f9914f4ab80 1 -- start start 2023-10-18T12:26:25.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.761+0000 7f11b9fff700 1 -- start start 2023-10-18T12:26:25.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.761+0000 7f9914f4ab80 1 Processor -- start 2023-10-18T12:26:25.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.761+0000 7f11b9fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.761+0000 7f11b9fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b41032b0 0x7f11b419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b9fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b4108900 0x7f11b4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9deb70 0x55853f9f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e1c90 0x55853f9f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e2070 0x55853f9f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f933ef0 con 0x55853f9e2070 2023-10-18T12:26:25.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f8d0790 con 0x55853f9e1c90 2023-10-18T12:26:25.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f8d0430 con 0x55853f9deb70 2023-10-18T12:26:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f9914f4ab80 10 client.0 _ll_register_callbacks cb 0x55853f81ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f98faffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9deb70 0x55853f9f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f98faffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9deb70 0x55853f9f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58086/0 (socket says 172.21.15.70:58086) 2023-10-18T12:26:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f98faffd700 1 -- 172.21.15.70:0/2188741199 learned_addr learned my addr 172.21.15.70:0/2188741199 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b9fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4199200 con 0x7f11b4108900 2023-10-18T12:26:25.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b9fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4199370 con 0x7f11b41032b0 2023-10-18T12:26:25.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b9fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4199530 con 0x7f11b41028f0 2023-10-18T12:26:25.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e2070 0x55853f9f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e1c90 0x55853f9f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58070/0 (socket says 172.21.15.70:58070) 2023-10-18T12:26:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 -- 172.21.15.70:0/3260566317 learned_addr learned my addr 172.21.15.70:0/3260566317 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b2ffd700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b41032b0 0x7f11b419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 -- 172.21.15.70:0/3260566317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b41032b0 msgr2=0x7f11b419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b41032b0 0x7f11b419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b3fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b4108900 0x7f11b4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 -- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b4108900 msgr2=0x7f11b4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b4108900 0x7f11b4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b37fe700 1 -- 172.21.15.70:0/3260566317 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80095f0 con 0x7f11b41028f0 2023-10-18T12:26:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.762+0000 7f11b3fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b4108900 0x7f11b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:26:25.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98fa7fc700 1 -- 172.21.15.70:0/2188741199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9deb70 msgr2=0x55853f9f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9deb70 0x55853f9f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98fa7fc700 1 -- 172.21.15.70:0/2188741199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e2070 msgr2=0x55853f9f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e2070 0x55853f9f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98fa7fc700 1 -- 172.21.15.70:0/2188741199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc10def0 con 0x55853f9e1c90 2023-10-18T12:26:25.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f11b37fe700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b419ea70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11a400e9a0 tx=0x7f11a400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e1c90 0x55853f9f3800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f98f001d000 tx=0x7f98f00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb47c735bdedae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98deff5700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:26:25.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4009790 con 0x7f11b41028f0 2023-10-18T12:26:25.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11a400cbd0 con 0x7f11b41028f0 2023-10-18T12:26:25.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b4199810 con 0x7f11b41028f0 2023-10-18T12:26:25.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4018680 con 0x7f11b41028f0 2023-10-18T12:26:25.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b41a5a60 con 0x7f11b41028f0 2023-10-18T12:26:25.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98deff5700 1 -- 172.21.15.70:0/2188741199 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0020c40 con 0x55853f9e1c90 2023-10-18T12:26:25.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f98deff5700 1 -- 172.21.15.70:0/2188741199 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f98f0020da0 con 0x55853f9e1c90 2023-10-18T12:26:25.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.763+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55853f938f70 con 0x55853f9e1c90 2023-10-18T12:26:25.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f98deff5700 1 -- 172.21.15.70:0/2188741199 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f000fe80 con 0x55853f9e1c90 2023-10-18T12:26:25.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f9914f4ab80 10 client.25619 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:25.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f98fc1a36b0 con 0x55853f9e1c90 2023-10-18T12:26:25.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f98deff5700 1 -- 172.21.15.70:0/2188741199 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55853f938f70 con 0x55853f9e1c90 2023-10-18T12:26:25.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b410aef0 con 0x7f11b41028f0 2023-10-18T12:26:25.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f98deff5700 1 -- 172.21.15.70:0/2188741199 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f98f0026030 con 0x55853f9e1c90 2023-10-18T12:26:25.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f9914f4ab80 10 client.25619 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:25.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.764+0000 7f9914f4ab80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:25.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f9914f4ab80 20 client.25619 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:25.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa03980 0x55853fa05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f9914f4ab80 4 client.25619 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:25.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55853f9b0930 con 0x55853fa03980 2023-10-18T12:26:25.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f98faffd700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa03980 0x55853fa05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f98deff5700 10 client.25619 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:25.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.765+0000 7f98faffd700 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa03980 0x55853fa05da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78aac8a5460c745c server_cookie=d5111803d63e6833 in_seq=0 out_seq=0 2023-10-18T12:26:25.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f98deff5700 1 -- 172.21.15.70:0/2188741199 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55853f9b0930 con 0x55853fa03980 2023-10-18T12:26:25.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f98deff5700 10 client.25619 handle_command_reply: tid=0 2023-10-18T12:26:25.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 client.25619 shutdown 2023-10-18T12:26:25.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e1c90 msgr2=0x55853f9f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e1c90 0x55853f9f3800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f98f001d000 tx=0x7f98f00047c0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f11a400cd40 con 0x7f11b41028f0 2023-10-18T12:26:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 shutdown_connections 2023-10-18T12:26:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa03980 0x55853fa05da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9e2070 0x55853f9f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9deb70 0x55853f9f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f98deff5700 0 client.25619 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:25.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/2188741199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e1c90 0x55853f9f3800 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.766+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 >> 172.21.15.70:0/2188741199 conn(0x55853f9557a0 msgr2=0x55853f9de0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.767+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 shutdown_connections 2023-10-18T12:26:25.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.767+0000 7f9914f4ab80 1 -- 172.21.15.70:0/2188741199 wait complete. 2023-10-18T12:26:25.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.767+0000 7f9914f4ab80 20 client.25619 trim_cache size 0 max 16384 2023-10-18T12:26:25.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.768+0000 7f11b0ff9700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f119c0a1550 0x7f119c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.768+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f11a4014070 con 0x7f11b41028f0 2023-10-18T12:26:25.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.768+0000 7f11b2ffd700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f119c0a1550 0x7f119c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.769+0000 7f11b2ffd700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f119c0a1550 0x7f119c0a3a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f11b419a240 tx=0x7f11a8005de0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.771+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f11a4052100 con 0x7f11b41028f0 2023-10-18T12:26:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 Processor -- start 2023-10-18T12:26:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 -- start start 2023-10-18T12:26:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853f9fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853f9d5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853f9f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f7b49a0 con 0x55853f9deb70 2023-10-18T12:26:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f9fac70 con 0x55853f9e2070 2023-10-18T12:26:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.784+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853f9d6180 con 0x55853f9e1c90 2023-10-18T12:26:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853f9f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853f9f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:42784/0 (socket says 172.21.15.70:42784) 2023-10-18T12:26:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98faffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853f9fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 -- 172.21.15.70:0/3446974059 learned_addr learned my addr 172.21.15.70:0/3446974059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fa7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853f9d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 -- 172.21.15.70:0/3446974059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 msgr2=0x55853f9d5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853f9d5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 -- 172.21.15.70:0/3446974059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 msgr2=0x55853f9fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853f9fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 -- 172.21.15.70:0/3446974059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853f75ee60 con 0x55853f9e2070 2023-10-18T12:26:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853f9f3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98f4099a60 tx=0x7f98f4004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb4c1242915e43a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.785+0000 7f98e1ffb700 1 -- 172.21.15.70:0/3446974059 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40996c0 con 0x55853f9e2070 2023-10-18T12:26:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f98e1ffb700 1 -- 172.21.15.70:0/3446974059 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f409a460 con 0x55853f9e2070 2023-10-18T12:26:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f98e1ffb700 1 -- 172.21.15.70:0/3446974059 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4099250 con 0x55853f9e2070 2023-10-18T12:26:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 -- 172.21.15.70:0/3446974059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 msgr2=0x55853f9f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853f9f3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98f4099a60 tx=0x7f98f4004dc0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 -- 172.21.15.70:0/3446974059 shutdown_connections 2023-10-18T12:26:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853f9f3810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853f9fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/3446974059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853f9d5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 -- 172.21.15.70:0/3446974059 >> 172.21.15.70:0/3446974059 conn(0x55853f9557a0 msgr2=0x55853f9d5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 -- 172.21.15.70:0/3446974059 shutdown_connections 2023-10-18T12:26:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.786+0000 7f9914f4ab80 1 -- 172.21.15.70:0/3446974059 wait complete. 2023-10-18T12:26:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.787+0000 7f9914f4ab80 1 -- start start 2023-10-18T12:26:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.787+0000 7f9914f4ab80 1 Processor -- start 2023-10-18T12:26:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853fa4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853fa4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853fa4e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853fa58880 con 0x55853f9deb70 2023-10-18T12:26:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853fa589f0 con 0x55853f9e2070 2023-10-18T12:26:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853fa58bb0 con 0x55853f9e1c90 2023-10-18T12:26:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f9914f4ab80 10 client.0 _ll_register_callbacks cb 0x55853f81ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853fa4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853fa4d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55644/0 (socket says 172.21.15.70:55644) 2023-10-18T12:26:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 -- 172.21.15.70:0/770799140 learned_addr learned my addr 172.21.15.70:0/770799140 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853fa4e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853fa4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 -- 172.21.15.70:0/770799140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 msgr2=0x55853fa4dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853fa4dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 -- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 msgr2=0x55853fa4e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853fa4e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98faffd700 1 -- 172.21.15.70:0/770799140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55853f8f1470 con 0x55853f9deb70 2023-10-18T12:26:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.788+0000 7f98fb7fe700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853fa4e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853fa4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f98faffd700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853fa4d710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c010 tx=0x7f98ec00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214191c42a8c1411 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f98deff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f98deff5700 1 -- 172.21.15.70:0/770799140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec01c040 con 0x55853f9deb70 2023-10-18T12:26:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f98deff5700 1 -- 172.21.15.70:0/770799140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f98ec025c60 con 0x55853f9deb70 2023-10-18T12:26:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98f4004ad0 con 0x55853f9deb70 2023-10-18T12:26:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f98deff5700 1 -- 172.21.15.70:0/770799140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98ec00f950 con 0x55853f9deb70 2023-10-18T12:26:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f9914f4ab80 10 client.15603 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.789+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55853f747ce0 con 0x55853f9deb70 2023-10-18T12:26:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f98deff5700 1 -- 172.21.15.70:0/770799140 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f4004ad0 con 0x55853f9deb70 2023-10-18T12:26:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f98deff5700 1 -- 172.21.15.70:0/770799140 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f98ec015020 con 0x55853f9deb70 2023-10-18T12:26:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f9914f4ab80 10 client.15603 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f9914f4ab80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f9914f4ab80 20 client.15603 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa5a4e0 0x55853fa5c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f9914f4ab80 4 client.15603 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55853f92ced0 con 0x55853fa5a4e0 2023-10-18T12:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.790+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa5a4e0 0x55853fa5c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.791+0000 7f98deff5700 10 client.15603 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.791+0000 7f98fa7fc700 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa5a4e0 0x55853fa5c900 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1a8c32816dbde5 server_cookie=da197a5314f94532 in_seq=0 out_seq=0 2023-10-18T12:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f98deff5700 1 -- 172.21.15.70:0/770799140 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55853f92ced0 con 0x55853fa5a4e0 2023-10-18T12:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f98deff5700 10 client.15603 handle_command_reply: tid=0 2023-10-18T12:26:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 client.15603 shutdown 2023-10-18T12:26:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 msgr2=0x55853fa4d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853fa4d710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c010 tx=0x7f98ec00f460 comp rx=0 tx=0).stop 2023-10-18T12:26:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 shutdown_connections 2023-10-18T12:26:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55853fa5a4e0 0x55853fa5c900 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55853f9e2070 0x55853fa4e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55853f9deb70 0x55853fa4d710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 --2- 172.21.15.70:0/770799140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55853f9e1c90 0x55853fa4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f98deff5700 0 client.15603 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 >> 172.21.15.70:0/770799140 conn(0x55853f9557a0 msgr2=0x55853f9e4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 shutdown_connections 2023-10-18T12:26:25.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 1 -- 172.21.15.70:0/770799140 wait complete. 2023-10-18T12:26:25.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.792+0000 7f9914f4ab80 20 client.15603 trim_cache size 0 max 16384 2023-10-18T12:26:25.820 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:26:25.821 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:26:25.821 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "729d9d81-1e98-4707-81f6-cb28224ba555", 2023-10-18T12:26:25.821 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:26:25.821 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:26:25.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98e40a14d0 msgr2=0x7f98e40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98e40a14d0 0x7f98e40a3990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98fc197b90 tx=0x7f98f0011b70 comp rx=0 tx=0).stop 2023-10-18T12:26:25.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 msgr2=0x7f98fc1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:25.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc1969d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f98f400efc0 tx=0x7f98f400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 shutdown_connections 2023-10-18T12:26:25.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98e40a14d0 0x7f98e40a3990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98fc106730 0x7f98fc1969d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98fc100720 0x7f98fc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 --2- 172.21.15.70:0/869280170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc1010e0 0x7f98fc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:25.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 >> 172.21.15.70:0/869280170 conn(0x7f98fc0fc1b0 msgr2=0x7f98fc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:25.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 shutdown_connections 2023-10-18T12:26:25.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:25.793+0000 7f99015ff700 1 -- 172.21.15.70:0/869280170 wait complete. 2023-10-18T12:26:25.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:25 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1323551903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:25.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:25 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:26:25.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:25 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3738686486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:25.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:26:26.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.026+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b4199ff0 con 0x7f119c0a1550 2023-10-18T12:26:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.038+0000 7f800299d700 1 -- 172.21.15.70:0/2048187295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 msgr2=0x7f7ffc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.038+0000 7f800299d700 1 --2- 172.21.15.70:0/2048187295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc106a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009990 tx=0x7f7ff8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 -- 172.21.15.70:0/2048187295 shutdown_connections 2023-10-18T12:26:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 --2- 172.21.15.70:0/2048187295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ffc100f80 0x7f7ffc10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 --2- 172.21.15.70:0/2048187295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc106a40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 --2- 172.21.15.70:0/2048187295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc1005c0 0x7f7ffc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 -- 172.21.15.70:0/2048187295 >> 172.21.15.70:0/2048187295 conn(0x7f7ffc0fc0f0 msgr2=0x7f7ffc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 -- 172.21.15.70:0/2048187295 shutdown_connections 2023-10-18T12:26:26.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.039+0000 7f800299d700 1 -- 172.21.15.70:0/2048187295 wait complete. 2023-10-18T12:26:26.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 Processor -- start 2023-10-18T12:26:26.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 -- start start 2023-10-18T12:26:26.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc1005c0 0x7f7ffc19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ffc100f80 0x7f7ffc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc196e30 con 0x7f7ffc1005c0 2023-10-18T12:26:26.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc196fa0 con 0x7f7ffc100f80 2023-10-18T12:26:26.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.040+0000 7f800299d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc197160 con 0x7f7ffc106660 2023-10-18T12:26:26.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc1967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58132/0 (socket says 172.21.15.70:58132) 2023-10-18T12:26:26.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 -- 172.21.15.70:0/951416706 learned_addr learned my addr 172.21.15.70:0/951416706 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800119a700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ffc100f80 0x7f7ffc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800199b700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc1005c0 0x7f7ffc19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 -- 172.21.15.70:0/951416706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ffc100f80 msgr2=0x7f7ffc19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ffc100f80 0x7f7ffc19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 -- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc1005c0 msgr2=0x7f7ffc19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc1005c0 0x7f7ffc19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.041+0000 7f800219c700 1 -- 172.21.15.70:0/951416706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80095f0 con 0x7f7ffc106660 2023-10-18T12:26:26.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.042+0000 7f800219c700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc1967c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400ef30 tx=0x7f7ff400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.042+0000 7f7ff2ffd700 1 -- 172.21.15.70:0/951416706 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4018070 con 0x7f7ffc106660 2023-10-18T12:26:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.042+0000 7f7ff2ffd700 1 -- 172.21.15.70:0/951416706 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff400f040 con 0x7f7ffc106660 2023-10-18T12:26:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.042+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc1973e0 con 0x7f7ffc106660 2023-10-18T12:26:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.042+0000 7f7ff2ffd700 1 -- 172.21.15.70:0/951416706 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400cd00 con 0x7f7ffc106660 2023-10-18T12:26:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.042+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc1a3680 con 0x7f7ffc106660 2023-10-18T12:26:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.046+0000 7f7ff2ffd700 1 -- 172.21.15.70:0/951416706 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7ff4003a00 con 0x7f7ffc106660 2023-10-18T12:26:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.047+0000 7f80174efb80 1 Processor -- start 2023-10-18T12:26:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.047+0000 7f80174efb80 1 -- start start 2023-10-18T12:26:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f80174efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe1255190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f80174efb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1251740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f80174efb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1251c80 0x55afe1265f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f80174efb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe11c46e0 con 0x55afe12512e0 2023-10-18T12:26:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f80174efb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe11aac60 con 0x55afe1251c80 2023-10-18T12:26:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f80174efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe11a8580 con 0x55afe1254db0 2023-10-18T12:26:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f800119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe1255190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f800199b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1251740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f800119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe1255190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58152/0 (socket says 172.21.15.70:58152) 2023-10-18T12:26:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f800119a700 1 -- 172.21.15.70:0/68695065 learned_addr learned my addr 172.21.15.70:0/68695065 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.048+0000 7f800119a700 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1251c80 0x55afe1265f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f800119a700 1 -- 172.21.15.70:0/68695065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1251c80 msgr2=0x55afe1265f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f800119a700 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1251c80 0x55afe1265f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f800119a700 1 -- 172.21.15.70:0/68695065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 msgr2=0x55afe1251740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f800119a700 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1251740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f7ff2ffd700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7fe80a14d0 0x7f7fe80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f800119a700 1 -- 172.21.15.70:0/68695065 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe11613f0 con 0x55afe1254db0 2023-10-18T12:26:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f7ff2ffd700 1 -- 172.21.15.70:0/951416706 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7ff40ccd50 con 0x7f7ffc106660 2023-10-18T12:26:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f800119a700 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe1255190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0136a0 tx=0x7f7fec009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30114a753e1a196c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f7fe7fff700 1 -- 172.21.15.70:0/68695065 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec004050 con 0x55afe1254db0 2023-10-18T12:26:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f7fe7fff700 1 -- 172.21.15.70:0/68695065 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec003680 con 0x55afe1254db0 2023-10-18T12:26:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.049+0000 7f7fe7fff700 1 -- 172.21.15.70:0/68695065 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec010690 con 0x55afe1254db0 2023-10-18T12:26:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f800219c700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7fe80a14d0 0x7f7fe80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f80174efb80 1 -- 172.21.15.70:0/68695065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 msgr2=0x55afe1255190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f800219c700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7fe80a14d0 0x7f7fe80a3990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400ef00 tx=0x7f7ff40092e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f80174efb80 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe1255190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0136a0 tx=0x7f7fec009ab0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f80174efb80 1 -- 172.21.15.70:0/68695065 shutdown_connections 2023-10-18T12:26:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f80174efb80 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1251c80 0x55afe1265f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f80174efb80 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe1255190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.050+0000 7f80174efb80 1 --2- 172.21.15.70:0/68695065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1251740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.051+0000 7f80174efb80 1 -- 172.21.15.70:0/68695065 >> 172.21.15.70:0/68695065 conn(0x55afe11c5720 msgr2=0x55afe124bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.051+0000 7f80174efb80 1 -- 172.21.15.70:0/68695065 shutdown_connections 2023-10-18T12:26:26.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.051+0000 7f80174efb80 1 -- 172.21.15.70:0/68695065 wait complete. 2023-10-18T12:26:26.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.051+0000 7f80174efb80 1 -- start start 2023-10-18T12:26:26.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.051+0000 7f80174efb80 1 Processor -- start 2023-10-18T12:26:26.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1265e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe1263180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe1263800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe11a3e70 con 0x55afe12512e0 2023-10-18T12:26:26.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe1140710 con 0x55afe1254db0 2023-10-18T12:26:26.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe11403b0 con 0x55afe1251c80 2023-10-18T12:26:26.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f80174efb80 10 client.0 _ll_register_callbacks cb 0x55afe108aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:26.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe1263180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe1263180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58172/0 (socket says 172.21.15.70:58172) 2023-10-18T12:26:26.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 -- 172.21.15.70:0/448811757 learned_addr learned my addr 172.21.15.70:0/448811757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.052+0000 7f800119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1265e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 -- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 msgr2=0x55afe1263800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe1263800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 -- 172.21.15.70:0/448811757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 msgr2=0x55afe1265e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1265e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.053+0000 7f800199b700 1 -- 172.21.15.70:0/448811757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec009e70 con 0x55afe1251c80 2023-10-18T12:26:26.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f800199b700 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe1263180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc101700 tx=0x7f7ff8005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4abc6d078643aded server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f7fe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:26:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f7fe57fa700 1 -- 172.21.15.70:0/448811757 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff801d070 con 0x55afe1251c80 2023-10-18T12:26:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f7fe57fa700 1 -- 172.21.15.70:0/448811757 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7ff8004500 con 0x55afe1251c80 2023-10-18T12:26:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afe11a8ef0 con 0x55afe1251c80 2023-10-18T12:26:26.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f7fe57fa700 1 -- 172.21.15.70:0/448811757 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff80046c0 con 0x55afe1251c80 2023-10-18T12:26:26.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f80174efb80 10 client.25642 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:26.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55afe0fcede0 con 0x55afe1251c80 2023-10-18T12:26:26.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.054+0000 7f7fe57fa700 1 -- 172.21.15.70:0/448811757 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55afe11a8ef0 con 0x55afe1251c80 2023-10-18T12:26:26.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f7fe57fa700 1 -- 172.21.15.70:0/448811757 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ff800f7c0 con 0x55afe1251c80 2023-10-18T12:26:26.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f80174efb80 10 client.25642 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:26.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f80174efb80 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:26.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f80174efb80 20 client.25642 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:26.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f80174efb80 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12739c0 0x55afe1275de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f80174efb80 4 client.25642 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:26.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afe12208b0 con 0x55afe12739c0 2023-10-18T12:26:26.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.055+0000 7f800119a700 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12739c0 0x55afe1275de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.056+0000 7f7fe57fa700 10 client.25642 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:26.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.056+0000 7f800119a700 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12739c0 0x55afe1275de0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf5a7c633df8fd6 server_cookie=766f481aae126a89 in_seq=0 out_seq=0 2023-10-18T12:26:26.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f7fe57fa700 1 -- 172.21.15.70:0/448811757 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55afe12208b0 con 0x55afe12739c0 2023-10-18T12:26:26.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f7fe57fa700 10 client.25642 handle_command_reply: tid=0 2023-10-18T12:26:26.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 client.25642 shutdown 2023-10-18T12:26:26.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 msgr2=0x55afe1263180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe1263180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc101700 tx=0x7f7ff8005800 comp rx=0 tx=0).stop 2023-10-18T12:26:26.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 shutdown_connections 2023-10-18T12:26:26.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12739c0 0x55afe1275de0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe1263800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1265e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 --2- 172.21.15.70:0/448811757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe1263180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 >> 172.21.15.70:0/448811757 conn(0x55afe11c5720 msgr2=0x55afe124e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f7fe57fa700 0 client.25642 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:26.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 shutdown_connections 2023-10-18T12:26:26.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.057+0000 7f80174efb80 1 -- 172.21.15.70:0/448811757 wait complete. 2023-10-18T12:26:26.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.058+0000 7f80174efb80 20 client.25642 trim_cache size 0 max 16384 2023-10-18T12:26:26.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.086+0000 7f80174efb80 1 Processor -- start 2023-10-18T12:26:26.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.086+0000 7f80174efb80 1 -- start start 2023-10-18T12:26:26.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.086+0000 7f80174efb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1246820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f80174efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe126a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f80174efb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe12626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f80174efb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe1024920 con 0x55afe12512e0 2023-10-18T12:26:26.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f80174efb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe124d700 con 0x55afe1254db0 2023-10-18T12:26:26.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f80174efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe12483f0 con 0x55afe1251c80 2023-10-18T12:26:26.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe126a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe126a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58180/0 (socket says 172.21.15.70:58180) 2023-10-18T12:26:26.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 -- 172.21.15.70:0/3740257916 learned_addr learned my addr 172.21.15.70:0/3740257916 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800119a700 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1246820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 -- 172.21.15.70:0/3740257916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 msgr2=0x55afe12626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800119a700 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe12626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe12626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 -- 172.21.15.70:0/3740257916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 msgr2=0x55afe1246820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1246820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800199b700 1 -- 172.21.15.70:0/3740257916 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe0fb7c60 con 0x55afe1251c80 2023-10-18T12:26:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.087+0000 7f800119a700 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe12626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:26:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.088+0000 7f800199b700 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe126a8e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800fbf0 tx=0x7f7ff8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a57ed2dfbfb671 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.088+0000 7f7ff0bff700 1 -- 172.21.15.70:0/3740257916 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff801d070 con 0x55afe1251c80 2023-10-18T12:26:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.088+0000 7f7ff0bff700 1 -- 172.21.15.70:0/3740257916 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff801e040 con 0x55afe1251c80 2023-10-18T12:26:26.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.088+0000 7f7ff0bff700 1 -- 172.21.15.70:0/3740257916 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8024a00 con 0x55afe1251c80 2023-10-18T12:26:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 -- 172.21.15.70:0/3740257916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 msgr2=0x55afe126a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe126a8e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800fbf0 tx=0x7f7ff8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 -- 172.21.15.70:0/3740257916 shutdown_connections 2023-10-18T12:26:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe1254db0 0x55afe12626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe12512e0 0x55afe1246820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 --2- 172.21.15.70:0/3740257916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1251c80 0x55afe126a8e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.089+0000 7f80174efb80 1 -- 172.21.15.70:0/3740257916 >> 172.21.15.70:0/3740257916 conn(0x55afe11c5720 msgr2=0x55afe124ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.090+0000 7f80174efb80 1 -- 172.21.15.70:0/3740257916 shutdown_connections 2023-10-18T12:26:26.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.090+0000 7f80174efb80 1 -- 172.21.15.70:0/3740257916 wait complete. 2023-10-18T12:26:26.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.090+0000 7f80174efb80 1 -- start start 2023-10-18T12:26:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.091+0000 7f80174efb80 1 Processor -- start 2023-10-18T12:26:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.091+0000 7f80174efb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 0x55afe1268ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.091+0000 7f80174efb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe1251c80 0x55afe1268ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.091+0000 7f80174efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe12bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.091+0000 7f80174efb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe12bfcf0 con 0x55afe1251c80 2023-10-18T12:26:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f80174efb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe12bfe60 con 0x55afe12512e0 2023-10-18T12:26:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f80174efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe12c0020 con 0x55afe1254db0 2023-10-18T12:26:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f80174efb80 10 client.0 _ll_register_callbacks cb 0x55afe108aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 0x55afe1268ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 0x55afe1268ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:42872/0 (socket says 172.21.15.70:42872) 2023-10-18T12:26:26.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 -- 172.21.15.70:0/1480222342 learned_addr learned my addr 172.21.15.70:0/1480222342 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe12bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800199b700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe1251c80 0x55afe1268ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 -- 172.21.15.70:0/1480222342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 msgr2=0x55afe12bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe12bf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 -- 172.21.15.70:0/1480222342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe1251c80 msgr2=0x55afe1268ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe1251c80 0x55afe1268ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.092+0000 7f800119a700 1 -- 172.21.15.70:0/1480222342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afe0fcede0 con 0x55afe12512e0 2023-10-18T12:26:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f800119a700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 0x55afe1268ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0092a0 tx=0x7f7fec003db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9cc22170a8996b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f7fe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:26:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f7fe57fa700 1 -- 172.21.15.70:0/1480222342 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec026040 con 0x55afe12512e0 2023-10-18T12:26:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fec003680 con 0x55afe12512e0 2023-10-18T12:26:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f7fe57fa700 1 -- 172.21.15.70:0/1480222342 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7fec02a890 con 0x55afe12512e0 2023-10-18T12:26:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f7fe57fa700 1 -- 172.21.15.70:0/1480222342 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fec02a730 con 0x55afe12512e0 2023-10-18T12:26:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f80174efb80 10 client.25643 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55afe0ff5250 con 0x55afe12512e0 2023-10-18T12:26:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.093+0000 7f7fe57fa700 1 -- 172.21.15.70:0/1480222342 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec003680 con 0x55afe12512e0 2023-10-18T12:26:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.094+0000 7f7fe57fa700 1 -- 172.21.15.70:0/1480222342 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7fec0053a0 con 0x55afe12512e0 2023-10-18T12:26:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.094+0000 7f80174efb80 10 client.25643 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.094+0000 7f80174efb80 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.094+0000 7f80174efb80 20 client.25643 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.095+0000 7f80174efb80 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12c89c0 0x55afe12cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.095+0000 7f80174efb80 4 client.25643 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:26:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.095+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55afe119ce50 con 0x55afe12c89c0 2023-10-18T12:26:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.095+0000 7f800199b700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12c89c0 0x55afe12cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.096+0000 7f800199b700 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12c89c0 0x55afe12cade0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8442e34875cf277 server_cookie=9f5bdc1a9231dfd0 in_seq=0 out_seq=0 2023-10-18T12:26:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.096+0000 7f7fe57fa700 10 client.25643 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.096+0000 7f7fe57fa700 1 -- 172.21.15.70:0/1480222342 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55afe119ce50 con 0x55afe12c89c0 2023-10-18T12:26:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.096+0000 7f7fe57fa700 10 client.25643 handle_command_reply: tid=0 2023-10-18T12:26:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.096+0000 7f80174efb80 1 client.25643 shutdown 2023-10-18T12:26:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 msgr2=0x55afe1268ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 0x55afe1268ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0092a0 tx=0x7f7fec003db0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 shutdown_connections 2023-10-18T12:26:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55afe12c89c0 0x55afe12cade0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55afe1254db0 0x55afe12bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afe12512e0 0x55afe1268ab0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0092a0 tx=0x7f7fec003db0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f7fe57fa700 0 client.25643 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 --2- 172.21.15.70:0/1480222342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55afe1251c80 0x55afe1268ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 >> 172.21.15.70:0/1480222342 conn(0x55afe11c5720 msgr2=0x55afe12626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 shutdown_connections 2023-10-18T12:26:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 1 -- 172.21.15.70:0/1480222342 wait complete. 2023-10-18T12:26:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.097+0000 7f80174efb80 20 client.25643 trim_cache size 0 max 16384 2023-10-18T12:26:26.116 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:26:26.116 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:26:26.116 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:26:26.116 INFO:teuthology.orchestra.run.smithi070.stdout: "729d9d81-1e98-4707-81f6-cb28224ba555": { 2023-10-18T12:26:26.116 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:26:26.117 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "729d9d81-1e98-4707-81f6-cb28224ba555", 2023-10-18T12:26:26.117 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:26:26.117 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:26:26.117 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:26:26.117 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:26:26.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7fe80a14d0 msgr2=0x7f7fe80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7fe80a14d0 0x7f7fe80a3990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400ef00 tx=0x7f7ff40092e0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 msgr2=0x7f7ffc1967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc1967c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400ef30 tx=0x7f7ff400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 shutdown_connections 2023-10-18T12:26:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ffc106660 0x7f7ffc1967c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc1005c0 0x7f7ffc19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7fe80a14d0 0x7f7fe80a3990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 --2- 172.21.15.70:0/951416706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ffc100f80 0x7f7ffc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.098+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 >> 172.21.15.70:0/951416706 conn(0x7f7ffc0fc0f0 msgr2=0x7f7ffc0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.099+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 shutdown_connections 2023-10-18T12:26:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.099+0000 7f800299d700 1 -- 172.21.15.70:0/951416706 wait complete. 2023-10-18T12:26:26.127 INFO:tasks.cephfs.filesystem:scrub status for tag:729d9d81-1e98-4707-81f6-cb28224ba555 - {'path': '/', 'tag': '729d9d81-1e98-4707-81f6-cb28224ba555', 'options': 'recursive,force'} 2023-10-18T12:26:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.149+0000 7f11b0ff9700 1 -- 172.21.15.70:0/3260566317 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f11b4199ff0 con 0x7f119c0a1550 2023-10-18T12:26:26.153 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path / 2023-10-18T12:26:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f119c0a1550 msgr2=0x7f119c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f119c0a1550 0x7f119c0a3a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f11b419a240 tx=0x7f11a8005de0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 msgr2=0x7f11b419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b419ea70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11a400e9a0 tx=0x7f11a400ed60 comp rx=0 tx=0).stop 2023-10-18T12:26:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 shutdown_connections 2023-10-18T12:26:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b4108900 0x7f11b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11b41028f0 0x7f11b419ea70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f119c0a1550 0x7f119c0a3a10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 --2- 172.21.15.70:0/3260566317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b41032b0 0x7f11b419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 >> 172.21.15.70:0/3260566317 conn(0x7f11b40fe3a0 msgr2=0x7f11b40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 shutdown_connections 2023-10-18T12:26:26.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.152+0000 7f11b9fff700 1 -- 172.21.15.70:0/3260566317 wait complete. 2023-10-18T12:26:26.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:25 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1323551903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:26.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:25 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:26:26.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:25 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3738686486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:26.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-10-18T12:26:26.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:25 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1323551903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:26.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:25 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:26:26.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:25 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3738686486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:26:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.415+0000 7fa08f7ff700 1 -- 172.21.15.70:0/2214240440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881066f0 msgr2=0x7fa088106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.415+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/2214240440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881066f0 0x7fa088106ad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.415+0000 7fa08f7ff700 1 -- 172.21.15.70:0/2214240440 shutdown_connections 2023-10-18T12:26:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.415+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/2214240440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881010a0 0x7fa08810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.415+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/2214240440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881066f0 0x7fa088106ad0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.415+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/2214240440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0881006e0 0x7fa088100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.416+0000 7fa08f7ff700 1 -- 172.21.15.70:0/2214240440 >> 172.21.15.70:0/2214240440 conn(0x7fa0880fc1f0 msgr2=0x7fa0880fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.416+0000 7fa08f7ff700 1 -- 172.21.15.70:0/2214240440 shutdown_connections 2023-10-18T12:26:26.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.416+0000 7fa08f7ff700 1 -- 172.21.15.70:0/2214240440 wait complete. 2023-10-18T12:26:26.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 Processor -- start 2023-10-18T12:26:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 -- start start 2023-10-18T12:26:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 0x7fa088198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0881010a0 0x7fa0881992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881066f0 0x7fa08819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08819dca0 con 0x7fa0881066f0 2023-10-18T12:26:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08819de10 con 0x7fa0881010a0 2023-10-18T12:26:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08f7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08819dfd0 con 0x7fa0881006e0 2023-10-18T12:26:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08d59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 0x7fa088198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08dd9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881066f0 0x7fa08819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08d59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 0x7fa088198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58206/0 (socket says 172.21.15.70:58206) 2023-10-18T12:26:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08d59b700 1 -- 172.21.15.70:0/3376405854 learned_addr learned my addr 172.21.15.70:0/3376405854 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.417+0000 7fa08cd9a700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0881010a0 0x7fa0881992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08dd9c700 1 -- 172.21.15.70:0/3376405854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 msgr2=0x7fa088198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08dd9c700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 0x7fa088198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08dd9c700 1 -- 172.21.15.70:0/3376405854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0881010a0 msgr2=0x7fa0881992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08dd9c700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0881010a0 0x7fa0881992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08dd9c700 1 -- 172.21.15.70:0/3376405854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840095f0 con 0x7fa0881066f0 2023-10-18T12:26:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08d59b700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 0x7fa088198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08dd9c700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881066f0 0x7fa08819d660 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa08000efc0 tx=0x7fa08000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080018070 con 0x7fa0881066f0 2023-10-18T12:26:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08000f040 con 0x7fa0881066f0 2023-10-18T12:26:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08819e250 con 0x7fa0881066f0 2023-10-18T12:26:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08000cd40 con 0x7fa0881066f0 2023-10-18T12:26:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.418+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0881a5af0 con 0x7fa0881066f0 2023-10-18T12:26:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.420+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa088101e20 con 0x7fa0881066f0 2023-10-18T12:26:26.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.422+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa080003a00 con 0x7fa0881066f0 2023-10-18T12:26:26.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.423+0000 7fa07e7fc700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0740a14d0 0x7fa0740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.423+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa0800ccd10 con 0x7fa0881066f0 2023-10-18T12:26:26.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.423+0000 7fa08d59b700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0740a14d0 0x7fa0740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.424+0000 7fa08d59b700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0740a14d0 0x7fa0740a3990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa084006010 tx=0x7fa08400b280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.427+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa0800d2020 con 0x7fa0881066f0 2023-10-18T12:26:26.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.688+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fa088110650 con 0x7fa0740a14d0 2023-10-18T12:26:26.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.742+0000 7fa07e7fc700 1 -- 172.21.15.70:0/3376405854 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fa088110650 con 0x7fa0740a14d0 2023-10-18T12:26:26.745 INFO:teuthology.orchestra.run.smithi070.stdout:Retention added to path / 2023-10-18T12:26:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0740a14d0 msgr2=0x7fa0740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0740a14d0 0x7fa0740a3990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa084006010 tx=0x7fa08400b280 comp rx=0 tx=0).stop 2023-10-18T12:26:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881066f0 msgr2=0x7fa08819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881066f0 0x7fa08819d660 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa08000efc0 tx=0x7fa08000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 shutdown_connections 2023-10-18T12:26:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0881066f0 0x7fa08819d660 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0881006e0 0x7fa088198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0740a14d0 0x7fa0740a3990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 --2- 172.21.15.70:0/3376405854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0881010a0 0x7fa0881992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 >> 172.21.15.70:0/3376405854 conn(0x7fa0880fc1f0 msgr2=0x7fa0880fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 shutdown_connections 2023-10-18T12:26:26.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.745+0000 7fa08f7ff700 1 -- 172.21.15.70:0/3376405854 wait complete. 2023-10-18T12:26:26.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-18T12:26:26.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 -- 172.21.15.70:0/596790184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81066f0 msgr2=0x7f55a8106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/596790184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81066f0 0x7f55a8106ad0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5598009990 tx=0x7f5598009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 -- 172.21.15.70:0/596790184 shutdown_connections 2023-10-18T12:26:26.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/596790184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 0x7f55a810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/596790184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81066f0 0x7f55a8106ad0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/596790184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81006e0 0x7f55a8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 -- 172.21.15.70:0/596790184 >> 172.21.15.70:0/596790184 conn(0x7f55a80fc1f0 msgr2=0x7f55a80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:26.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 -- 172.21.15.70:0/596790184 shutdown_connections 2023-10-18T12:26:26.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.972+0000 7f55ac9ff700 1 -- 172.21.15.70:0/596790184 wait complete. 2023-10-18T12:26:26.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 Processor -- start 2023-10-18T12:26:26.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 -- start start 2023-10-18T12:26:26.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 0x7f55a819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 0x7f55a819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81066f0 0x7f55a81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8197010 con 0x7f55a81006e0 2023-10-18T12:26:26.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8197180 con 0x7f55a81010a0 2023-10-18T12:26:26.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55ac9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8197340 con 0x7f55a81066f0 2023-10-18T12:26:26.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.973+0000 7f55a659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 0x7f55a819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 0x7f55a819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55780/0 (socket says 172.21.15.70:55780) 2023-10-18T12:26:26.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 -- 172.21.15.70:0/3668116695 learned_addr learned my addr 172.21.15.70:0/3668116695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:26.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a5d9b700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 0x7f55a819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a6d9d700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81066f0 0x7f55a81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 -- 172.21.15.70:0/3668116695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81066f0 msgr2=0x7f55a81969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81066f0 0x7f55a81969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 -- 172.21.15.70:0/3668116695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 msgr2=0x7f55a819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:26.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 0x7f55a819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:26.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 -- 172.21.15.70:0/3668116695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55980095f0 con 0x7f55a81006e0 2023-10-18T12:26:26.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a6d9d700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81066f0 0x7f55a81969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:26.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a5d9b700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 0x7f55a819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:26.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55a659c700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 0x7f55a819c880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5598000c00 tx=0x7f55980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559801d070 con 0x7f55a81006e0 2023-10-18T12:26:26.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a81975c0 con 0x7f55a81006e0 2023-10-18T12:26:26.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5598022470 con 0x7f55a81006e0 2023-10-18T12:26:26.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.974+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598017650 con 0x7f55a81006e0 2023-10-18T12:26:26.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.975+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a81a3870 con 0x7f55a81006e0 2023-10-18T12:26:26.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.976+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8108ce0 con 0x7f55a81006e0 2023-10-18T12:26:26.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.977+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f55980225e0 con 0x7f55a81006e0 2023-10-18T12:26:26.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.979+0000 7f55977fe700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f55900a14d0 0x7f55900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:26.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.979+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f55980ce590 con 0x7f55a81006e0 2023-10-18T12:26:26.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.979+0000 7f55a5d9b700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f55900a14d0 0x7f55900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:26.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.980+0000 7f55a5d9b700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f55900a14d0 0x7f55900a3990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f55a8197b70 tx=0x7f559c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:26.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:26.983+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5598097860 con 0x7f55a81006e0 2023-10-18T12:26:27.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:26 smithi070 ceph-mon[98639]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 716 B/s wr, 2 op/s 2023-10-18T12:26:27.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:26 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:26:27.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:26 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:27.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:26 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:26:27.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:26 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:27.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:26 smithi070 ceph-mon[98639]: from='client.25615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:27.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:26 smithi152 ceph-mon[134767]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 716 B/s wr, 2 op/s 2023-10-18T12:26:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:26 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:26:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:26 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:26 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:26:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:26 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:27.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:26 smithi152 ceph-mon[134767]: from='client.25615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:26 smithi186 ceph-mon[135253]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 716 B/s wr, 2 op/s 2023-10-18T12:26:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:26 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:26:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:26 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:26 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:26:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:26 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:26 smithi186 ceph-mon[135253]: from='client.25615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.220+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a8197e70 con 0x7f55900a14d0 2023-10-18T12:26:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.228+0000 7f55977fe700 1 -- 172.21.15.70:0/3668116695 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f55a8197e70 con 0x7f55900a14d0 2023-10-18T12:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T12:26:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-10-18T12:26:27.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.231+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f55900a14d0 msgr2=0x7f55900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.231+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f55900a14d0 0x7f55900a3990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f55a8197b70 tx=0x7f559c009260 comp rx=0 tx=0).stop 2023-10-18T12:26:27.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.231+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 msgr2=0x7f55a819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.231+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 0x7f55a819c880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5598000c00 tx=0x7f55980057f0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 shutdown_connections 2023-10-18T12:26:27.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a81066f0 0x7f55a81969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55a81006e0 0x7f55a819c880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f55900a14d0 0x7f55900a3990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 --2- 172.21.15.70:0/3668116695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a81010a0 0x7f55a819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 >> 172.21.15.70:0/3668116695 conn(0x7f55a80fc1f0 msgr2=0x7f55a80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:27.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 shutdown_connections 2023-10-18T12:26:27.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.232+0000 7f55ac9ff700 1 -- 172.21.15.70:0/3668116695 wait complete. 2023-10-18T12:26:27.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-18T12:26:27.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.471+0000 7ff8d45ff700 1 -- 172.21.15.70:0/4206174992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1066f0 msgr2=0x7ff8cc106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.471+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/4206174992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1066f0 0x7ff8cc106ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009990 tx=0x7ff8c8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 -- 172.21.15.70:0/4206174992 shutdown_connections 2023-10-18T12:26:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/4206174992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc1010a0 0x7ff8cc10d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/4206174992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1066f0 0x7ff8cc106ad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/4206174992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 -- 172.21.15.70:0/4206174992 >> 172.21.15.70:0/4206174992 conn(0x7ff8cc0fc1b0 msgr2=0x7ff8cc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 -- 172.21.15.70:0/4206174992 shutdown_connections 2023-10-18T12:26:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.472+0000 7ff8d45ff700 1 -- 172.21.15.70:0/4206174992 wait complete. 2023-10-18T12:26:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 Processor -- start 2023-10-18T12:26:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 -- start start 2023-10-18T12:26:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc1010a0 0x7ff8cc073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:27.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc1066f0 0x7ff8cc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:27.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1a4fd0 con 0x7ff8cc1066f0 2023-10-18T12:26:27.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1a5140 con 0x7ff8cc1010a0 2023-10-18T12:26:27.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d45ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1a5300 con 0x7ff8cc1006e0 2023-10-18T12:26:27.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d2b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc1066f0 0x7ff8cc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:27.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.473+0000 7ff8d239b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:27.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58232/0 (socket says 172.21.15.70:58232) 2023-10-18T12:26:27.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 -- 172.21.15.70:0/3709118217 learned_addr learned my addr 172.21.15.70:0/3709118217 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:27.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d1b9a700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc1010a0 0x7ff8cc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:27.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 -- 172.21.15.70:0/3709118217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc1010a0 msgr2=0x7ff8cc073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc1010a0 0x7ff8cc073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 -- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc1066f0 msgr2=0x7ff8cc073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc1066f0 0x7ff8cc073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 -- 172.21.15.70:0/3709118217 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80095f0 con 0x7ff8cc1006e0 2023-10-18T12:26:27.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d239b700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc072af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8005910 tx=0x7ff8c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:27.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c801d070 con 0x7ff8cc1006e0 2023-10-18T12:26:27.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c8022470 con 0x7ff8cc1006e0 2023-10-18T12:26:27.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.474+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc1a5580 con 0x7ff8cc1006e0 2023-10-18T12:26:27.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.475+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8017650 con 0x7ff8cc1006e0 2023-10-18T12:26:27.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.475+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc1a5ab0 con 0x7ff8cc1006e0 2023-10-18T12:26:27.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.476+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc108ce0 con 0x7ff8cc1006e0 2023-10-18T12:26:27.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.477+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff8c80225e0 con 0x7ff8cc1006e0 2023-10-18T12:26:27.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.479+0000 7ff8c37fe700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff8b80a1550 0x7ff8b80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:27.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.479+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff8c80ce300 con 0x7ff8cc1006e0 2023-10-18T12:26:27.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.479+0000 7ff8d1b9a700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff8b80a1550 0x7ff8b80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:27.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.480+0000 7ff8d1b9a700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff8b80a1550 0x7ff8b80a3a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc074220 tx=0x7ff8bc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:27.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.481+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff8c80975d0 con 0x7ff8cc1006e0 2023-10-18T12:26:27.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.762+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7ff8c80979a0 con 0x7ff8cc1006e0 2023-10-18T12:26:27.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.766+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff8cc073fd0 con 0x7ff8b80a1550 2023-10-18T12:26:27.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.774+0000 7ff8c37fe700 1 -- 172.21.15.70:0/3709118217 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff8cc073fd0 con 0x7ff8b80a1550 2023-10-18T12:26:27.779 INFO:teuthology.orchestra.run.smithi070.stdout:/ 1m 6m3h 2023-10-18T12:26:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff8b80a1550 msgr2=0x7ff8b80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff8b80a1550 0x7ff8b80a3a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc074220 tx=0x7ff8bc009260 comp rx=0 tx=0).stop 2023-10-18T12:26:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 msgr2=0x7ff8cc072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc072af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8005910 tx=0x7ff8c8004730 comp rx=0 tx=0).stop 2023-10-18T12:26:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 shutdown_connections 2023-10-18T12:26:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc1066f0 0x7ff8cc073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8cc1006e0 0x7ff8cc072af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff8b80a1550 0x7ff8b80a3a10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 --2- 172.21.15.70:0/3709118217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8cc1010a0 0x7ff8cc073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 >> 172.21.15.70:0/3709118217 conn(0x7ff8cc0fc1b0 msgr2=0x7ff8cc0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 shutdown_connections 2023-10-18T12:26:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:27.776+0000 7ff8d45ff700 1 -- 172.21.15.70:0/3709118217 wait complete. 2023-10-18T12:26:27.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-10-18T12:26:27.873 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-10-18T12:26:27.884 INFO:teuthology.run_tasks:Running task workunit... 2023-10-18T12:26:27.896 INFO:tasks.workunit:Pulling workunits from ref f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:26:27.896 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-18T12:26:27.897 INFO:tasks.workunit:timeout=3h 2023-10-18T12:26:27.897 INFO:tasks.workunit:cleanup=True 2023-10-18T12:26:27.897 DEBUG:teuthology.orchestra.run.smithi070:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-18T12:26:27.947 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-10-18T12:26:27.947 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T12:26:27.948 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-18T12:26:27.948 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-18 12:24:36.655717170 +0000 2023-10-18T12:26:27.948 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-18 12:24:36.655717170 +0000 2023-10-18T12:26:27.948 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-18 12:26:10.547936184 +0000 2023-10-18T12:26:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-18 12:24:36.655717170 +0000 2023-10-18T12:26:27.949 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-18T12:26:27.949 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-18T12:26:28.010 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:27 smithi070 ceph-mon[98639]: from='client.15633 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:28.027 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T12:26:28.074 INFO:tasks.workunit.client.0.smithi070.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-18T12:26:28.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:27 smithi152 ceph-mon[134767]: from='client.15633 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:28.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:27 smithi186 ceph-mon[135253]: from='client.15633 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: pgmap v328: 129 pgs: 129 active+clean; 963 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 47 KiB/s wr, 8 op/s 2023-10-18T12:26:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: mgrmap e23: x(active, since 8m), standbys: z, y 2023-10-18T12:26:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: from='client.25666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:28 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:26:29.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: pgmap v328: 129 pgs: 129 active+clean; 963 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 47 KiB/s wr, 8 op/s 2023-10-18T12:26:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: mgrmap e23: x(active, since 8m), standbys: z, y 2023-10-18T12:26:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: from='client.25666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:28 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:26:29.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: pgmap v328: 129 pgs: 129 active+clean; 963 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 47 KiB/s wr, 8 op/s 2023-10-18T12:26:29.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: mgrmap e23: x(active, since 8m), standbys: z, y 2023-10-18T12:26:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: from='client.25666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:26:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:29.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:28 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:26:30.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:26:30.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:26:30.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:29 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:26:30.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:26:30.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:29 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:30.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:26:30.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:26:30.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:29 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:26:31.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:31 smithi152 ceph-mon[134767]: pgmap v329: 129 pgs: 129 active+clean; 963 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 45 KiB/s wr, 7 op/s 2023-10-18T12:26:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:31 smithi186 ceph-mon[135253]: pgmap v329: 129 pgs: 129 active+clean; 963 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 45 KiB/s wr, 7 op/s 2023-10-18T12:26:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:31 smithi070 ceph-mon[98639]: pgmap v329: 129 pgs: 129 active+clean; 963 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 45 KiB/s wr, 7 op/s 2023-10-18T12:26:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:33 smithi186 ceph-mon[135253]: pgmap v330: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:33.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:33 smithi070 ceph-mon[98639]: pgmap v330: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:33 smithi152 ceph-mon[134767]: pgmap v330: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:35 smithi186 ceph-mon[135253]: pgmap v331: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:35 smithi070 ceph-mon[98639]: pgmap v331: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:35 smithi152 ceph-mon[134767]: pgmap v331: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:37 smithi186 ceph-mon[135253]: pgmap v332: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:37 smithi070 ceph-mon[98639]: pgmap v332: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:37 smithi152 ceph-mon[134767]: pgmap v332: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:39.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:39 smithi186 ceph-mon[135253]: pgmap v333: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:39.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:39 smithi070 ceph-mon[98639]: pgmap v333: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:39 smithi152 ceph-mon[134767]: pgmap v333: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-18T12:26:41.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:41 smithi186 ceph-mon[135253]: pgmap v334: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 70 KiB/s wr, 30 op/s 2023-10-18T12:26:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:41 smithi070 ceph-mon[98639]: pgmap v334: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 70 KiB/s wr, 30 op/s 2023-10-18T12:26:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:41 smithi152 ceph-mon[134767]: pgmap v334: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 70 KiB/s wr, 30 op/s 2023-10-18T12:26:43.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:43 smithi186 ceph-mon[135253]: pgmap v335: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-10-18T12:26:43.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:43 smithi070 ceph-mon[98639]: pgmap v335: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-10-18T12:26:43.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:43 smithi152 ceph-mon[134767]: pgmap v335: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-10-18T12:26:45.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:45 smithi186 ceph-mon[135253]: pgmap v336: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-18T12:26:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:45 smithi070 ceph-mon[98639]: pgmap v336: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-18T12:26:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:45 smithi152 ceph-mon[134767]: pgmap v336: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-18T12:26:47.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:47 smithi186 ceph-mon[135253]: pgmap v337: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:47.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:47 smithi070 ceph-mon[98639]: pgmap v337: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:47 smithi152 ceph-mon[134767]: pgmap v337: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:49.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:49 smithi186 ceph-mon[135253]: pgmap v338: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:49 smithi070 ceph-mon[98639]: pgmap v338: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:49.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:49 smithi152 ceph-mon[134767]: pgmap v338: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:51 smithi186 ceph-mon[135253]: pgmap v339: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:51 smithi070 ceph-mon[98639]: pgmap v339: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:51.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:51 smithi152 ceph-mon[134767]: pgmap v339: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:53 smithi186 ceph-mon[135253]: pgmap v340: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:53 smithi070 ceph-mon[98639]: pgmap v340: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:53 smithi152 ceph-mon[134767]: pgmap v340: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:55.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:55 smithi186 ceph-mon[135253]: pgmap v341: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:55 smithi070 ceph-mon[98639]: pgmap v341: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:55 smithi152 ceph-mon[134767]: pgmap v341: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:56.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:26:56.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 -- 172.21.15.70:0/574482506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 msgr2=0x7f4620072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 --2- 172.21.15.70:0/574482506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f4620072f70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4610009990 tx=0x7f4610009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 -- 172.21.15.70:0/574482506 shutdown_connections 2023-10-18T12:26:56.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 --2- 172.21.15.70:0/574482506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46201115c0 0x7f46201139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 --2- 172.21.15.70:0/574482506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f4620072f70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 --2- 172.21.15.70:0/574482506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46200734b0 0x7f4620111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 -- 172.21.15.70:0/574482506 >> 172.21.15.70:0/574482506 conn(0x7f46200fc0c0 msgr2=0x7f46200fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 -- 172.21.15.70:0/574482506 shutdown_connections 2023-10-18T12:26:56.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.339+0000 7f461ffff700 1 -- 172.21.15.70:0/574482506 wait complete. 2023-10-18T12:26:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.340+0000 7f461ffff700 1 Processor -- start 2023-10-18T12:26:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.340+0000 7f461ffff700 1 -- start start 2023-10-18T12:26:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.340+0000 7f461ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f462019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.340+0000 7f461ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46200734b0 0x7f462019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46201115c0 0x7f46201a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a1fd0 con 0x7f4620072b90 2023-10-18T12:26:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a2140 con 0x7f46201115c0 2023-10-18T12:26:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a2300 con 0x7f46200734b0 2023-10-18T12:26:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f462019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f462019d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42894/0 (socket says 172.21.15.70:42894) 2023-10-18T12:26:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 -- 172.21.15.70:0/2761523940 learned_addr learned my addr 172.21.15.70:0/2761523940 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461f7fe700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46201115c0 0x7f46201a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461e7fc700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46200734b0 0x7f462019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 -- 172.21.15.70:0/2761523940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46200734b0 msgr2=0x7f462019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46200734b0 0x7f462019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 -- 172.21.15.70:0/2761523940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46201115c0 msgr2=0x7f46201a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46201115c0 0x7f46201a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 -- 172.21.15.70:0/2761523940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100095f0 con 0x7f4620072b90 2023-10-18T12:26:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.341+0000 7f461effd700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f462019d0b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4610005910 tx=0x7f4610004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.342+0000 7f4607fff700 1 -- 172.21.15.70:0/2761523940 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461001d070 con 0x7f4620072b90 2023-10-18T12:26:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.342+0000 7f4607fff700 1 -- 172.21.15.70:0/2761523940 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4610022470 con 0x7f4620072b90 2023-10-18T12:26:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.342+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46201a2580 con 0x7f4620072b90 2023-10-18T12:26:56.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.342+0000 7f4607fff700 1 -- 172.21.15.70:0/2761523940 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610017650 con 0x7f4620072b90 2023-10-18T12:26:56.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.342+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201a9e20 con 0x7f4620072b90 2023-10-18T12:26:56.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.346+0000 7f4638cf7b80 1 Processor -- start 2023-10-18T12:26:56.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.346+0000 7f4607fff700 1 -- 172.21.15.70:0/2761523940 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f46100225e0 con 0x7f4620072b90 2023-10-18T12:26:56.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 -- start start 2023-10-18T12:26:56.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e963140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e95d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e95dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e8d06e0 con 0x564e9e962d60 2023-10-18T12:26:56.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e8b6c60 con 0x564e9e963680 2023-10-18T12:26:56.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e8b4580 con 0x564e9e95d820 2023-10-18T12:26:56.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f461effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e95d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.347+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e963140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e963140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42902/0 (socket says 172.21.15.70:42902) 2023-10-18T12:26:56.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461f7fe700 1 -- 172.21.15.70:0/4203032424 learned_addr learned my addr 172.21.15.70:0/4203032424 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:56.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461e7fc700 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e95dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461effd700 1 -- 172.21.15.70:0/4203032424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 msgr2=0x564e9e95dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461effd700 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e95dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461effd700 1 -- 172.21.15.70:0/4203032424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 msgr2=0x564e9e963140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461effd700 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e963140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461effd700 1 -- 172.21.15.70:0/4203032424 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e9e86d3f0 con 0x564e9e963680 2023-10-18T12:26:56.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461e7fc700 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e95dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:56.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461f7fe700 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e963140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:56.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f461effd700 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e95d2e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46100cbab0 tx=0x7f46100d4820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d70f041cd9798ee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.348+0000 7f4607fff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f46080a1560 0x7f46080a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.349+0000 7f46057fa700 1 -- 172.21.15.70:0/4203032424 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100977b0 con 0x564e9e963680 2023-10-18T12:26:56.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.349+0000 7f4607fff700 1 -- 172.21.15.70:0/2761523940 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f46100ce520 con 0x7f4620072b90 2023-10-18T12:26:56.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.349+0000 7f46057fa700 1 -- 172.21.15.70:0/4203032424 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f461009b240 con 0x564e9e963680 2023-10-18T12:26:56.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.349+0000 7f46057fa700 1 -- 172.21.15.70:0/4203032424 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461009afb0 con 0x564e9e963680 2023-10-18T12:26:56.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.349+0000 7f461f7fe700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f46080a1560 0x7f46080a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 -- 172.21.15.70:0/4203032424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 msgr2=0x564e9e95d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f461f7fe700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f46080a1560 0x7f46080a3a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4618007610 tx=0x7f461800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e95d2e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46100cbab0 tx=0x7f46100d4820 comp rx=0 tx=0).stop 2023-10-18T12:26:56.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 -- 172.21.15.70:0/4203032424 shutdown_connections 2023-10-18T12:26:56.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e95dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e963140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/4203032424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e95d2e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 -- 172.21.15.70:0/4203032424 >> 172.21.15.70:0/4203032424 conn(0x564e9e8d1720 msgr2=0x564e9e957cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 -- 172.21.15.70:0/4203032424 shutdown_connections 2023-10-18T12:26:56.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 -- 172.21.15.70:0/4203032424 wait complete. 2023-10-18T12:26:56.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.350+0000 7f4638cf7b80 1 -- start start 2023-10-18T12:26:56.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 Processor -- start 2023-10-18T12:26:56.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e969620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e962d60 0x564e9e966910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 0x564e9e966f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e8afe70 con 0x564e9e963680 2023-10-18T12:26:56.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e84c710 con 0x564e9e962d60 2023-10-18T12:26:56.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e84c3b0 con 0x564e9e95d820 2023-10-18T12:26:56.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.351+0000 7f4638cf7b80 10 client.0 _ll_register_callbacks cb 0x564e9e796a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 0x564e9e966f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 0x564e9e966f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42918/0 (socket says 172.21.15.70:42918) 2023-10-18T12:26:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 -- 172.21.15.70:0/3957826007 learned_addr learned my addr 172.21.15.70:0/3957826007 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461e7fc700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e962d60 0x564e9e966910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461effd700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e969620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 -- 172.21.15.70:0/3957826007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 msgr2=0x564e9e969620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e969620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 -- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e962d60 msgr2=0x564e9e966910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e962d60 0x564e9e966910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 -- 172.21.15.70:0/3957826007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46180112b0 con 0x564e9e963680 2023-10-18T12:26:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461e7fc700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e962d60 0x564e9e966910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461effd700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e969620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f461f7fe700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 0x564e9e966f90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4618016dd0 tx=0x7f4618004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced98b829d497fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.352+0000 7f4602ff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4602ff5700 1 -- 172.21.15.70:0/3957826007 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800e040 con 0x564e9e963680 2023-10-18T12:26:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4602ff5700 1 -- 172.21.15.70:0/3957826007 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f46180092d0 con 0x564e9e963680 2023-10-18T12:26:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e9e8b4ef0 con 0x564e9e963680 2023-10-18T12:26:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4602ff5700 1 -- 172.21.15.70:0/3957826007 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4618011dc0 con 0x564e9e963680 2023-10-18T12:26:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4638cf7b80 10 client.15657 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f46201a9e20 con 0x564e9e963680 2023-10-18T12:26:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4602ff5700 1 -- 172.21.15.70:0/3957826007 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e9e8b4ef0 con 0x564e9e963680 2023-10-18T12:26:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4602ff5700 1 -- 172.21.15.70:0/3957826007 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4618028020 con 0x564e9e963680 2023-10-18T12:26:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4638cf7b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.353+0000 7f4638cf7b80 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f4638cf7b80 20 client.15657 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e976e40 0x564e9e979260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f4638cf7b80 4 client.15657 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e9e92c8b0 con 0x564e9e976e40 2023-10-18T12:26:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f461effd700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e976e40 0x564e9e979260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f461effd700 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e976e40 0x564e9e979260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad59d1b9126e6d0 server_cookie=4ef92575d2318db7 in_seq=0 out_seq=0 2023-10-18T12:26:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.354+0000 7f4602ff5700 10 client.15657 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.355+0000 7f4602ff5700 1 -- 172.21.15.70:0/3957826007 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564e9e92c8b0 con 0x564e9e976e40 2023-10-18T12:26:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.355+0000 7f4602ff5700 10 client.15657 handle_command_reply: tid=0 2023-10-18T12:26:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.355+0000 7f4638cf7b80 1 client.15657 shutdown 2023-10-18T12:26:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.355+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 msgr2=0x564e9e966f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.355+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 0x564e9e966f90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4618016dd0 tx=0x7f4618004ad0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 shutdown_connections 2023-10-18T12:26:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e976e40 0x564e9e979260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e963680 0x564e9e966f90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e969620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/3957826007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e962d60 0x564e9e966910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 >> 172.21.15.70:0/3957826007 conn(0x564e9e8d1720 msgr2=0x564e9e95c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4602ff5700 0 client.15657 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 shutdown_connections 2023-10-18T12:26:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 1 -- 172.21.15.70:0/3957826007 wait complete. 2023-10-18T12:26:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.356+0000 7f4638cf7b80 20 client.15657 trim_cache size 0 max 16384 2023-10-18T12:26:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.374+0000 7f4638cf7b80 1 Processor -- start 2023-10-18T12:26:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.374+0000 7f4638cf7b80 1 -- start start 2023-10-18T12:26:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 0x564e9e966f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e962d60 0x564e9e964a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e730920 con 0x564e9e95d820 2023-10-18T12:26:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e958bc0 con 0x564e9e963680 2023-10-18T12:26:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e96e2f0 con 0x564e9e962d60 2023-10-18T12:26:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 0x564e9e966f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 0x564e9e966f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42934/0 (socket says 172.21.15.70:42934) 2023-10-18T12:26:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 -- 172.21.15.70:0/357071246 learned_addr learned my addr 172.21.15.70:0/357071246 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461f7fe700 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461e7fc700 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e962d60 0x564e9e964a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 -- 172.21.15.70:0/357071246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e962d60 msgr2=0x564e9e964a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e962d60 0x564e9e964a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 -- 172.21.15.70:0/357071246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 msgr2=0x564e9e96ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461effd700 1 -- 172.21.15.70:0/357071246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e9e6dade0 con 0x564e9e95d820 2023-10-18T12:26:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.375+0000 7f461f7fe700 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.376+0000 7f461effd700 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 0x564e9e966f00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4610097ee0 tx=0x7f46100d40e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e330dc33acd1bc90 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.376+0000 7f4605ffb700 1 -- 172.21.15.70:0/357071246 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100ce040 con 0x564e9e95d820 2023-10-18T12:26:56.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.376+0000 7f4605ffb700 1 -- 172.21.15.70:0/357071246 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46100d4b70 con 0x564e9e95d820 2023-10-18T12:26:56.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.376+0000 7f4605ffb700 1 -- 172.21.15.70:0/357071246 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100b3ea0 con 0x564e9e95d820 2023-10-18T12:26:56.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 -- 172.21.15.70:0/357071246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 msgr2=0x564e9e966f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 0x564e9e966f00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4610097ee0 tx=0x7f46100d40e0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 -- 172.21.15.70:0/357071246 shutdown_connections 2023-10-18T12:26:56.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e95d820 0x564e9e966f00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/357071246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e962d60 0x564e9e964a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 -- 172.21.15.70:0/357071246 >> 172.21.15.70:0/357071246 conn(0x564e9e8d1720 msgr2=0x564e9e957a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 -- 172.21.15.70:0/357071246 shutdown_connections 2023-10-18T12:26:56.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 -- 172.21.15.70:0/357071246 wait complete. 2023-10-18T12:26:56.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 -- start start 2023-10-18T12:26:56.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.377+0000 7f4638cf7b80 1 Processor -- start 2023-10-18T12:26:56.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e96bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e96c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e9c9d10 con 0x564e9e962d60 2023-10-18T12:26:56.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e9c9e80 con 0x564e9e963680 2023-10-18T12:26:56.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9e9ca040 con 0x564e9e95d820 2023-10-18T12:26:56.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f4638cf7b80 10 client.0 _ll_register_callbacks cb 0x564e9e796a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:56.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f461e7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e96c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f461e7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e96c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42950/0 (socket says 172.21.15.70:42950) 2023-10-18T12:26:56.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f461e7fc700 1 -- 172.21.15.70:0/792853818 learned_addr learned my addr 172.21.15.70:0/792853818 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:56.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f461effd700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e96bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.378+0000 7f461f7fe700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f461e7fc700 1 -- 172.21.15.70:0/792853818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 msgr2=0x564e9e96bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f461e7fc700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e96bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f461e7fc700 1 -- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 msgr2=0x564e9e96c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:56.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f461e7fc700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f461e7fc700 1 -- 172.21.15.70:0/792853818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201a9e20 con 0x564e9e962d60 2023-10-18T12:26:56.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f461e7fc700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e96c2d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f462019e600 tx=0x7f4614003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dbb31192247427 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f4602ff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:56.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f4602ff5700 1 -- 172.21.15.70:0/792853818 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614021070 con 0x564e9e962d60 2023-10-18T12:26:56.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e9e8a8e50 con 0x564e9e962d60 2023-10-18T12:26:56.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f4602ff5700 1 -- 172.21.15.70:0/792853818 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4614004830 con 0x564e9e962d60 2023-10-18T12:26:56.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.379+0000 7f4602ff5700 1 -- 172.21.15.70:0/792853818 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4614017050 con 0x564e9e962d60 2023-10-18T12:26:56.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 10 client.15663 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:56.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564e9e86d3f0 con 0x564e9e962d60 2023-10-18T12:26:56.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4602ff5700 1 -- 172.21.15.70:0/792853818 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e9e8a8e50 con 0x564e9e962d60 2023-10-18T12:26:56.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4602ff5700 1 -- 172.21.15.70:0/792853818 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4614024020 con 0x564e9e962d60 2023-10-18T12:26:56.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:56.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:56.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 20 client.15663 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:56.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e9cb950 0x564e9e9cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 4 client.15663 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:26:56.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e9e8a8b70 con 0x564e9e9cb950 2023-10-18T12:26:56.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.380+0000 7f461effd700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e9cb950 0x564e9e9cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4602ff5700 10 client.15663 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:56.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f461effd700 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e9cb950 0x564e9e9cdd70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76742ee2b195785e server_cookie=dad191a3fe1e7659 in_seq=0 out_seq=0 2023-10-18T12:26:56.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4602ff5700 1 -- 172.21.15.70:0/792853818 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564e9e8a8b70 con 0x564e9e9cb950 2023-10-18T12:26:56.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4602ff5700 10 client.15663 handle_command_reply: tid=0 2023-10-18T12:26:56.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 client.15663 shutdown 2023-10-18T12:26:56.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 msgr2=0x564e9e96c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e96c2d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f462019e600 tx=0x7f4614003be0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 shutdown_connections 2023-10-18T12:26:56.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564e9e9cb950 0x564e9e9cdd70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e9e963680 0x564e9e96c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9e95d820 0x564e9e96bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4602ff5700 0 client.15663 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:56.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 --2- 172.21.15.70:0/792853818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564e9e962d60 0x564e9e96c2d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 >> 172.21.15.70:0/792853818 conn(0x564e9e8d1720 msgr2=0x564e9e959580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.381+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 shutdown_connections 2023-10-18T12:26:56.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f4638cf7b80 1 -- 172.21.15.70:0/792853818 wait complete. 2023-10-18T12:26:56.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f4638cf7b80 20 client.15663 trim_cache size 0 max 16384 2023-10-18T12:26:56.401 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:26:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:26:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:26:56.402 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:26:56.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f46080a1560 msgr2=0x7f46080a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f46080a1560 0x7f46080a3a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4618007610 tx=0x7f461800b040 comp rx=0 tx=0).stop 2023-10-18T12:26:56.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 msgr2=0x7f462019d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f462019d0b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4610005910 tx=0x7f4610004730 comp rx=0 tx=0).stop 2023-10-18T12:26:56.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 shutdown_connections 2023-10-18T12:26:56.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f46080a1560 0x7f46080a3a20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46201115c0 0x7f46201a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4620072b90 0x7f462019d0b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 --2- 172.21.15.70:0/2761523940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46200734b0 0x7f462019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 >> 172.21.15.70:0/2761523940 conn(0x7f46200fc0c0 msgr2=0x7f46200fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.382+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 shutdown_connections 2023-10-18T12:26:56.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.383+0000 7f461ffff700 1 -- 172.21.15.70:0/2761523940 wait complete. 2023-10-18T12:26:56.406 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:26:56.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:26:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.608+0000 7fbd3119d700 1 -- 172.21.15.70:0/3245934789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c067830 msgr2=0x7fbd2c067cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.608+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3245934789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c067830 0x7fbd2c067cb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009990 tx=0x7fbd20009ca0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 -- 172.21.15.70:0/3245934789 shutdown_connections 2023-10-18T12:26:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3245934789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c0681f0 0x7fbd2c10d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3245934789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c106510 0x7fbd2c1068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3245934789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c067830 0x7fbd2c067cb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 -- 172.21.15.70:0/3245934789 >> 172.21.15.70:0/3245934789 conn(0x7fbd2c074ee0 msgr2=0x7fbd2c0752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 -- 172.21.15.70:0/3245934789 shutdown_connections 2023-10-18T12:26:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.609+0000 7fbd3119d700 1 -- 172.21.15.70:0/3245934789 wait complete. 2023-10-18T12:26:56.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 Processor -- start 2023-10-18T12:26:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 -- start start 2023-10-18T12:26:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 0x7fbd2c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0681f0 0x7fbd2c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 0x7fbd2c196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c196d90 con 0x7fbd2c106510 2023-10-18T12:26:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c196f00 con 0x7fbd2c0681f0 2023-10-18T12:26:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.610+0000 7fbd3119d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c1970c0 con 0x7fbd2c067830 2023-10-18T12:26:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 0x7fbd2c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd3099c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 0x7fbd2c196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0681f0 0x7fbd2c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 0x7fbd2c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:59932/0 (socket says 172.21.15.70:59932) 2023-10-18T12:26:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0681f0 0x7fbd2c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43450/0 (socket says 172.21.15.70:43450) 2023-10-18T12:26:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 -- 172.21.15.70:0/3864603487 learned_addr learned my addr 172.21.15.70:0/3864603487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd3099c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 0x7fbd2c196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42966/0 (socket says 172.21.15.70:42966) 2023-10-18T12:26:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 -- 172.21.15.70:0/3864603487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0681f0 msgr2=0x7fbd2c19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0681f0 0x7fbd2c19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 -- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 msgr2=0x7fbd2c196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 0x7fbd2c196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd2bfff700 1 -- 172.21.15.70:0/3864603487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c009520 con 0x7fbd2c067830 2023-10-18T12:26:56.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.611+0000 7fbd3099c700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 0x7fbd2c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:56.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.612+0000 7fbd2bfff700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 0x7fbd2c19c5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00eb30 tx=0x7fbd1c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.612+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c009880 con 0x7fbd2c067830 2023-10-18T12:26:56.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.612+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd200095f0 con 0x7fbd2c067830 2023-10-18T12:26:56.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.612+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd1c0044e0 con 0x7fbd2c067830 2023-10-18T12:26:56.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.612+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c003ce0 con 0x7fbd2c067830 2023-10-18T12:26:56.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.612+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c197590 con 0x7fbd2c067830 2023-10-18T12:26:56.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.613+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c108a70 con 0x7fbd2c067830 2023-10-18T12:26:56.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.615+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7fbd1c004020 con 0x7fbd2c067830 2023-10-18T12:26:56.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.616+0000 7fbd297fa700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbd140a1560 0x7fbd140a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:56.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.617+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fbd1c014070 con 0x7fbd2c067830 2023-10-18T12:26:56.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.617+0000 7fbd2b7fe700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbd140a1560 0x7fbd140a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:56.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.617+0000 7fbd2b7fe700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbd140a1560 0x7fbd140a3a20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c197df0 tx=0x7fbd2000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:56.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.619+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbd1c096840 con 0x7fbd2c067830 2023-10-18T12:26:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.872+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd2c101bc0 con 0x7fbd2c067830 2023-10-18T12:26:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.873+0000 7fbd297fa700 1 -- 172.21.15.70:0/3864603487 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fbd1c099e60 con 0x7fbd2c067830 2023-10-18T12:26:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:26:56.877 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:26:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbd140a1560 msgr2=0x7fbd140a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbd140a1560 0x7fbd140a3a20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c197df0 tx=0x7fbd2000b260 comp rx=0 tx=0).stop 2023-10-18T12:26:56.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 msgr2=0x7fbd2c19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:56.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 0x7fbd2c19c5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00eb30 tx=0x7fbd1c00ee40 comp rx=0 tx=0).stop 2023-10-18T12:26:56.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 shutdown_connections 2023-10-18T12:26:56.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd2c106510 0x7fbd2c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd2c067830 0x7fbd2c19c5b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbd140a1560 0x7fbd140a3a20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 --2- 172.21.15.70:0/3864603487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c0681f0 0x7fbd2c19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:56.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 >> 172.21.15.70:0/3864603487 conn(0x7fbd2c074ee0 msgr2=0x7fbd2c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:56.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 shutdown_connections 2023-10-18T12:26:56.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:56.876+0000 7fbd3119d700 1 -- 172.21.15.70:0/3864603487 wait complete. 2023-10-18T12:26:56.882 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:26:56.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:26:57.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/3435692742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac410abb0 msgr2=0x7f1ac410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/3435692742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac410abb0 0x7f1ac410d070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009f90 tx=0x7f1abc00b330 comp rx=0 tx=0).stop 2023-10-18T12:26:57.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/3435692742 shutdown_connections 2023-10-18T12:26:57.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/3435692742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac410abb0 0x7f1ac410d070 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/3435692742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac4101790 0x7f1ac4101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/3435692742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac41020b0 0x7f1ac410a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/3435692742 >> 172.21.15.70:0/3435692742 conn(0x7f1ac40fafb0 msgr2=0x7f1ac40fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:57.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/3435692742 shutdown_connections 2023-10-18T12:26:57.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.103+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/3435692742 wait complete. 2023-10-18T12:26:57.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.104+0000 7f1ac9d9d700 1 Processor -- start 2023-10-18T12:26:57.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.104+0000 7f1ac9d9d700 1 -- start start 2023-10-18T12:26:57.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac9d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 0x7f1ac4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac9d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac41020b0 0x7f1ac41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac9d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac410abb0 0x7f1ac419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac9d9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419dd60 con 0x7f1ac41020b0 2023-10-18T12:26:57.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac9d9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419ded0 con 0x7f1ac4101790 2023-10-18T12:26:57.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac9d9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419e090 con 0x7f1ac410abb0 2023-10-18T12:26:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 0x7f1ac4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 0x7f1ac4198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43472/0 (socket says 172.21.15.70:43472) 2023-10-18T12:26:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/509392978 learned_addr learned my addr 172.21.15.70:0/509392978 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac41020b0 0x7f1ac41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac959c700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac410abb0 0x7f1ac419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/509392978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac410abb0 msgr2=0x7f1ac419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac410abb0 0x7f1ac419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac41020b0 msgr2=0x7f1ac41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac41020b0 0x7f1ac41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.105+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/509392978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80095f0 con 0x7f1ac4101790 2023-10-18T12:26:57.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.106+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 0x7f1ac4198e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800ec10 tx=0x7f1ab800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:57.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.106+0000 7f1ac1ffb700 1 -- 172.21.15.70:0/509392978 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab800cc90 con 0x7f1ac4101790 2023-10-18T12:26:57.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.106+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc009c40 con 0x7f1ac4101790 2023-10-18T12:26:57.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.106+0000 7f1ac1ffb700 1 -- 172.21.15.70:0/509392978 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab80044e0 con 0x7f1ac4101790 2023-10-18T12:26:57.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.106+0000 7f1ac1ffb700 1 -- 172.21.15.70:0/509392978 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8010430 con 0x7f1ac4101790 2023-10-18T12:26:57.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.106+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac419e560 con 0x7f1ac4101790 2023-10-18T12:26:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.111+0000 7f1ac1ffb700 1 -- 172.21.15.70:0/509392978 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f1ab8003fb0 con 0x7f1ac4101790 2023-10-18T12:26:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 Processor -- start 2023-10-18T12:26:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 -- start start 2023-10-18T12:26:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff5d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff57c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0feca6e0 con 0x560c0ff5d680 2023-10-18T12:26:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0feb0c60 con 0x560c0ff5cd60 2023-10-18T12:26:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.112+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0feae580 con 0x560c0ff57820 2023-10-18T12:26:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff5d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff5d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43486/0 (socket says 172.21.15.70:43486) 2023-10-18T12:26:57.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 -- 172.21.15.70:0/3839908925 learned_addr learned my addr 172.21.15.70:0/3839908925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:57.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff57c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac1ffb700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ab40a1560 0x7f1ab40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac1ffb700 1 -- 172.21.15.70:0/509392978 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1ab8014070 con 0x7f1ac4101790 2023-10-18T12:26:57.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 -- 172.21.15.70:0/3839908925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 msgr2=0x560c0ff57c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff57c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 -- 172.21.15.70:0/3839908925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 msgr2=0x560c0ff572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 -- 172.21.15.70:0/3839908925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c0fe673f0 con 0x560c0ff5cd60 2023-10-18T12:26:57.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.113+0000 7f1ac959c700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ab40a1560 0x7f1ab40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.114+0000 7f1ac959c700 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff5d140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009ac0 tx=0x7f1abc009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9babd8bd6f980a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:57.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.114+0000 7f1aaeffd700 1 -- 172.21.15.70:0/3839908925 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00e040 con 0x560c0ff5cd60 2023-10-18T12:26:57.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.114+0000 7f1aaeffd700 1 -- 172.21.15.70:0/3839908925 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1abc003e90 con 0x560c0ff5cd60 2023-10-18T12:26:57.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.114+0000 7f1aaeffd700 1 -- 172.21.15.70:0/3839908925 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc01e740 con 0x560c0ff5cd60 2023-10-18T12:26:57.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.114+0000 7f1ac959c700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ab40a1560 0x7f1ab40a3a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1abc029000 tx=0x7f1abc026000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:57.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 -- 172.21.15.70:0/3839908925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 msgr2=0x560c0ff5d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff5d140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009ac0 tx=0x7f1abc009af0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 -- 172.21.15.70:0/3839908925 shutdown_connections 2023-10-18T12:26:57.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff57c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff5d140 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/3839908925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 -- 172.21.15.70:0/3839908925 >> 172.21.15.70:0/3839908925 conn(0x560c0fecb720 msgr2=0x560c0ff51cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:57.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 -- 172.21.15.70:0/3839908925 shutdown_connections 2023-10-18T12:26:57.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 -- 172.21.15.70:0/3839908925 wait complete. 2023-10-18T12:26:57.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.115+0000 7f1ade95cb80 1 -- start start 2023-10-18T12:26:57.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 Processor -- start 2023-10-18T12:26:57.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff63610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0fea9e70 con 0x560c0ff5d680 2023-10-18T12:26:57.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0fe46710 con 0x560c0ff5cd60 2023-10-18T12:26:57.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0fe463b0 con 0x560c0ff57820 2023-10-18T12:26:57.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.116+0000 7f1ade95cb80 10 client.0 _ll_register_callbacks cb 0x560c0fd90a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:57.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43500/0 (socket says 172.21.15.70:43500) 2023-10-18T12:26:57.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff60ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42996/0 (socket says 172.21.15.70:42996) 2023-10-18T12:26:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac3fff700 1 -- 172.21.15.70:0/4069640291 learned_addr learned my addr 172.21.15.70:0/4069640291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff63610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 -- 172.21.15.70:0/4069640291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 msgr2=0x560c0ff63610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff63610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 -- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 msgr2=0x560c0ff608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 -- 172.21.15.70:0/4069640291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac410dd30 con 0x560c0ff5d680 2023-10-18T12:26:57.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:57.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1ac959c700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff60ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009fd0 tx=0x7f1abc007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263872f8ef27517b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:57.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.117+0000 7f1aa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:57.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1aa3fff700 1 -- 172.21.15.70:0/4069640291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc007720 con 0x560c0ff5d680 2023-10-18T12:26:57.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c0feaeef0 con 0x560c0ff5d680 2023-10-18T12:26:57.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1aa3fff700 1 -- 172.21.15.70:0/4069640291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1abc00bbf0 con 0x560c0ff5d680 2023-10-18T12:26:57.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1aa3fff700 1 -- 172.21.15.70:0/4069640291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1abc01ea50 con 0x560c0ff5d680 2023-10-18T12:26:57.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 10 client.15681 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:57.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560c0fcd4de0 con 0x560c0ff5d680 2023-10-18T12:26:57.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1aa3fff700 1 -- 172.21.15.70:0/4069640291 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c0feaeef0 con 0x560c0ff5d680 2023-10-18T12:26:57.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1aa3fff700 1 -- 172.21.15.70:0/4069640291 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1abc03a610 con 0x560c0ff5d680 2023-10-18T12:26:57.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 10 client.15681 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:57.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 20 client.15681 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ff70bb0 0x560c0ff72fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 4 client.15681 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.118+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c0ff268b0 con 0x560c0ff70bb0 2023-10-18T12:26:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.119+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ff70bb0 0x560c0ff72fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.119+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ff70bb0 0x560c0ff72fd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50eb3fa7c597cc0 server_cookie=eea830948d7dcf5d in_seq=0 out_seq=0 2023-10-18T12:26:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.119+0000 7f1aa3fff700 10 client.15681 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1aa3fff700 1 -- 172.21.15.70:0/4069640291 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c0ff268b0 con 0x560c0ff70bb0 2023-10-18T12:26:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1aa3fff700 10 client.15681 handle_command_reply: tid=0 2023-10-18T12:26:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 client.15681 shutdown 2023-10-18T12:26:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 msgr2=0x560c0ff60ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff60ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009fd0 tx=0x7f1abc007520 comp rx=0 tx=0).stop 2023-10-18T12:26:57.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 shutdown_connections 2023-10-18T12:26:57.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ff70bb0 0x560c0ff72fd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ff60ef0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff63610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4069640291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5cd60 0x560c0ff608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1aa3fff700 0 client.15681 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.120+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 >> 172.21.15.70:0/4069640291 conn(0x560c0fecb720 msgr2=0x560c0ff562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.121+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 shutdown_connections 2023-10-18T12:26:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.121+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4069640291 wait complete. 2023-10-18T12:26:57.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.121+0000 7f1ade95cb80 20 client.15681 trim_cache size 0 max 16384 2023-10-18T12:26:57.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 Processor -- start 2023-10-18T12:26:57.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 -- start start 2023-10-18T12:26:57.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 0x560c0ff63140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5d680 0x560c0ff5a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0fd2a920 con 0x560c0ff5cd60 2023-10-18T12:26:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0ff61f00 con 0x560c0ff5d680 2023-10-18T12:26:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.135+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0ff614f0 con 0x560c0ff57820 2023-10-18T12:26:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 0x560c0ff63140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 0x560c0ff63140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43010/0 (socket says 172.21.15.70:43010) 2023-10-18T12:26:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 -- 172.21.15.70:0/4267662552 learned_addr learned my addr 172.21.15.70:0/4267662552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac959c700 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5d680 0x560c0ff5a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 -- 172.21.15.70:0/4267662552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 msgr2=0x560c0ff68060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff68060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 -- 172.21.15.70:0/4267662552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5d680 msgr2=0x560c0ff5a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5d680 0x560c0ff5a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 -- 172.21.15.70:0/4267662552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c0fcbdc60 con 0x560c0ff5cd60 2023-10-18T12:26:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac959c700 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5d680 0x560c0ff5a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.136+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 0x560c0ff63140 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00127a0 tx=0x7f1ab0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02e86c09c6b2d3c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1aaf7fe700 1 -- 172.21.15.70:0/4267662552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0020070 con 0x560c0ff5cd60 2023-10-18T12:26:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1aaf7fe700 1 -- 172.21.15.70:0/4267662552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab0018040 con 0x560c0ff5cd60 2023-10-18T12:26:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1aaf7fe700 1 -- 172.21.15.70:0/4267662552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0017050 con 0x560c0ff5cd60 2023-10-18T12:26:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4267662552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 msgr2=0x560c0ff63140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 0x560c0ff63140 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00127a0 tx=0x7f1ab0009260 comp rx=0 tx=0).stop 2023-10-18T12:26:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4267662552 shutdown_connections 2023-10-18T12:26:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff5d680 0x560c0ff5a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff57820 0x560c0ff68060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/4267662552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5cd60 0x560c0ff63140 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4267662552 >> 172.21.15.70:0/4267662552 conn(0x560c0fecb720 msgr2=0x560c0ff56a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.137+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4267662552 shutdown_connections 2023-10-18T12:26:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.138+0000 7f1ade95cb80 1 -- 172.21.15.70:0/4267662552 wait complete. 2023-10-18T12:26:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.138+0000 7f1ade95cb80 1 -- start start 2023-10-18T12:26:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.138+0000 7f1ade95cb80 1 Processor -- start 2023-10-18T12:26:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 0x560c0ff67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 0x560c0ff679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ffc39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0ffc3f20 con 0x560c0ff5d680 2023-10-18T12:26:57.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0ffc4060 con 0x560c0ff57820 2023-10-18T12:26:57.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c0ffc41f0 con 0x560c0ff5cd60 2023-10-18T12:26:57.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ade95cb80 10 client.0 _ll_register_callbacks cb 0x560c0fd90a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:57.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 0x560c0ff67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac959c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ffc39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 0x560c0ff679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 0x560c0ff67470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43510/0 (socket says 172.21.15.70:43510) 2023-10-18T12:26:57.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 0x560c0ff679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:59986/0 (socket says 172.21.15.70:59986) 2023-10-18T12:26:57.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/2913752589 learned_addr learned my addr 172.21.15.70:0/2913752589 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:57.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/2913752589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 msgr2=0x560c0ff679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 0x560c0ff679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/2913752589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 msgr2=0x560c0ffc39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:57.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ffc39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac8d9b700 1 -- 172.21.15.70:0/2913752589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c0fcd4de0 con 0x560c0ff57820 2023-10-18T12:26:57.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 0x560c0ff679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:57.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.139+0000 7f1ac959c700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ffc39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:57.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1ac8d9b700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 0x560c0ff67470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8096af0 tx=0x7f1ab8069aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2114edb945b307b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:57.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1aa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:26:57.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1aa3fff700 1 -- 172.21.15.70:0/2913752589 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8099f50 con 0x560c0ff57820 2023-10-18T12:26:57.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1abc003e90 con 0x560c0ff57820 2023-10-18T12:26:57.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1aa3fff700 1 -- 172.21.15.70:0/2913752589 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1ab8065e60 con 0x560c0ff57820 2023-10-18T12:26:57.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1aa3fff700 1 -- 172.21.15.70:0/2913752589 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab809acf0 con 0x560c0ff57820 2023-10-18T12:26:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1ade95cb80 10 client.25703 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f1ab0012450 con 0x560c0ff57820 2023-10-18T12:26:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.140+0000 7f1aa3fff700 1 -- 172.21.15.70:0/2913752589 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1abc003e90 con 0x560c0ff57820 2023-10-18T12:26:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1aa3fff700 1 -- 172.21.15.70:0/2913752589 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ab806c0b0 con 0x560c0ff57820 2023-10-18T12:26:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ade95cb80 10 client.25703 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ade95cb80 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ade95cb80 20 client.25703 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ffc59f0 0x560c0ffc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ade95cb80 4 client.25703 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:26:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560c0fea2e50 con 0x560c0ffc59f0 2023-10-18T12:26:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.141+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ffc59f0 0x560c0ffc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:57.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1aa3fff700 10 client.25703 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:57.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ac3fff700 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ffc59f0 0x560c0ffc7e10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3515bd917943fb server_cookie=ffa345b371e0d8be in_seq=0 out_seq=0 2023-10-18T12:26:57.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1aa3fff700 1 -- 172.21.15.70:0/2913752589 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560c0fea2e50 con 0x560c0ffc59f0 2023-10-18T12:26:57.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1aa3fff700 10 client.25703 handle_command_reply: tid=0 2023-10-18T12:26:57.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 client.25703 shutdown 2023-10-18T12:26:57.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 msgr2=0x560c0ff67470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 0x560c0ff67470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8096af0 tx=0x7f1ab8069aa0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 shutdown_connections 2023-10-18T12:26:57.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c0ffc59f0 0x560c0ffc7e10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c0ff5d680 0x560c0ffc39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c0ff57820 0x560c0ff67470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1aa3fff700 0 client.25703 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:57.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 --2- 172.21.15.70:0/2913752589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c0ff5cd60 0x560c0ff679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 >> 172.21.15.70:0/2913752589 conn(0x560c0fecb720 msgr2=0x560c0ff5a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:57.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 shutdown_connections 2023-10-18T12:26:57.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.142+0000 7f1ade95cb80 1 -- 172.21.15.70:0/2913752589 wait complete. 2023-10-18T12:26:57.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ade95cb80 20 client.25703 trim_cache size 0 max 16384 2023-10-18T12:26:57.161 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:26:57.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ab40a1560 msgr2=0x7f1ab40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ab40a1560 0x7f1ab40a3a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1abc029000 tx=0x7f1abc026000 comp rx=0 tx=0).stop 2023-10-18T12:26:57.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 msgr2=0x7f1ac4198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:57.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 0x7f1ac4198e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800ec10 tx=0x7f1ab800ef20 comp rx=0 tx=0).stop 2023-10-18T12:26:57.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 shutdown_connections 2023-10-18T12:26:57.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac410abb0 0x7f1ac419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1ab40a1560 0x7f1ab40a3a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101790 0x7f1ac4198e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 --2- 172.21.15.70:0/509392978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac41020b0 0x7f1ac41993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:57.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 >> 172.21.15.70:0/509392978 conn(0x7f1ac40fafb0 msgr2=0x7f1ac41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:57.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 shutdown_connections 2023-10-18T12:26:57.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:57.143+0000 7f1ac9d9d700 1 -- 172.21.15.70:0/509392978 wait complete. 2023-10-18T12:26:57.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:57 smithi186 ceph-mon[135253]: pgmap v342: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:57.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:57 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3864603487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:57 smithi070 ceph-mon[98639]: pgmap v342: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:57 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3864603487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:57 smithi152 ceph-mon[134767]: pgmap v342: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:26:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:57 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3864603487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:26:58.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:26:58.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:26:58.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/3475638929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c810abb0 msgr2=0x7fa4c810d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/3475638929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c810abb0 0x7fa4c810d070 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009f90 tx=0x7fa4c000b330 comp rx=0 tx=0).stop 2023-10-18T12:26:58.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/3475638929 shutdown_connections 2023-10-18T12:26:58.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/3475638929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c810abb0 0x7fa4c810d070 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/3475638929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4c8101790 0x7fa4c8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/3475638929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c81020b0 0x7fa4c810a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/3475638929 >> 172.21.15.70:0/3475638929 conn(0x7fa4c80fafb0 msgr2=0x7fa4c80fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/3475638929 shutdown_connections 2023-10-18T12:26:58.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.378+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/3475638929 wait complete. 2023-10-18T12:26:58.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.379+0000 7fa4cc9fe700 1 Processor -- start 2023-10-18T12:26:58.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.379+0000 7fa4cc9fe700 1 -- start start 2023-10-18T12:26:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4cc9fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 0x7fa4c8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4cc9fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c81020b0 0x7fa4c81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4cc9fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4c810abb0 0x7fa4c819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4cc9fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c819dd60 con 0x7fa4c8101790 2023-10-18T12:26:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4cc9fe700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c819ded0 con 0x7fa4c81020b0 2023-10-18T12:26:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4cc9fe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c819e090 con 0x7fa4c810abb0 2023-10-18T12:26:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4c6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 0x7fa4c8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4c6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 0x7fa4c8198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42662/0 (socket says 172.21.15.70:42662) 2023-10-18T12:26:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4c6d9d700 1 -- 172.21.15.70:0/958991862 learned_addr learned my addr 172.21.15.70:0/958991862 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4c759e700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4c810abb0 0x7fa4c819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.380+0000 7fa4c659c700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c81020b0 0x7fa4c81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4c6d9d700 1 -- 172.21.15.70:0/958991862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4c810abb0 msgr2=0x7fa4c819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4c6d9d700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4c810abb0 0x7fa4c819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4c6d9d700 1 -- 172.21.15.70:0/958991862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c81020b0 msgr2=0x7fa4c81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4c6d9d700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c81020b0 0x7fa4c81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4c6d9d700 1 -- 172.21.15.70:0/958991862 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c0009c40 con 0x7fa4c8101790 2023-10-18T12:26:58.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4c6d9d700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 0x7fa4c8198e90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800e9a0 tx=0x7fa4b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4affff700 1 -- 172.21.15.70:0/958991862 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8009790 con 0x7fa4c8101790 2023-10-18T12:26:58.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4affff700 1 -- 172.21.15.70:0/958991862 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b8010430 con 0x7fa4c8101790 2023-10-18T12:26:58.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4affff700 1 -- 172.21.15.70:0/958991862 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8018680 con 0x7fa4c8101790 2023-10-18T12:26:58.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c819e370 con 0x7fa4c8101790 2023-10-18T12:26:58.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.381+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c81a5bb0 con 0x7fa4c8101790 2023-10-18T12:26:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.385+0000 7fa4affff700 1 -- 172.21.15.70:0/958991862 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7fa4b80105a0 con 0x7fa4c8101790 2023-10-18T12:26:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 Processor -- start 2023-10-18T12:26:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 -- start start 2023-10-18T12:26:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 0x55e7445dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445d68a0 0x55e7445d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e744549760 con 0x55e7445d68a0 2023-10-18T12:26:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74452fce0 con 0x55e7445dc700 2023-10-18T12:26:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.386+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74452d600 con 0x55e7445dbde0 2023-10-18T12:26:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 0x55e7445dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c6d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 0x55e7445dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:56682/0 (socket says 172.21.15.70:56682) 2023-10-18T12:26:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 -- 172.21.15.70:0/1500992331 learned_addr learned my addr 172.21.15.70:0/1500992331 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c659c700 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445d68a0 0x55e7445d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 -- 172.21.15.70:0/1500992331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 msgr2=0x55e7445d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445d6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 -- 172.21.15.70:0/1500992331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445d68a0 msgr2=0x55e7445d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445d68a0 0x55e7445d6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4c759e700 1 -- 172.21.15.70:0/1500992331 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7444e6470 con 0x55e7445dbde0 2023-10-18T12:26:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.387+0000 7fa4affff700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4b00a1560 0x7fa4b00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.388+0000 7fa4c759e700 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 0x55e7445dc1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00094b0 tx=0x7fa4c00094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f22f99345d4fd81 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.388+0000 7fa4affff700 1 -- 172.21.15.70:0/958991862 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa4b8014070 con 0x7fa4c8101790 2023-10-18T12:26:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.388+0000 7fa4c6d9d700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4b00a1560 0x7fa4b00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.388+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/1500992331 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000e040 con 0x55e7445dbde0 2023-10-18T12:26:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.388+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/1500992331 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4c0007620 con 0x55e7445dbde0 2023-10-18T12:26:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.388+0000 7fa4ad7fa700 1 -- 172.21.15.70:0/1500992331 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0007c60 con 0x55e7445dbde0 2023-10-18T12:26:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/1500992331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 msgr2=0x55e7445dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 0x55e7445dc1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00094b0 tx=0x7fa4c00094e0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4c6d9d700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4b00a1560 0x7fa4b00a3a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55e7445dcb60 tx=0x7fa4b80d0e10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/1500992331 shutdown_connections 2023-10-18T12:26:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445d68a0 0x55e7445d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dbde0 0x55e7445dc1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/1500992331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/1500992331 >> 172.21.15.70:0/1500992331 conn(0x55e74454a7a0 msgr2=0x55e7445d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/1500992331 shutdown_connections 2023-10-18T12:26:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/1500992331 wait complete. 2023-10-18T12:26:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.389+0000 7fa4e0af9b80 1 -- start start 2023-10-18T12:26:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 Processor -- start 2023-10-18T12:26:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e744528ef0 con 0x55e7445dbde0 2023-10-18T12:26:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7444c5790 con 0x55e7445dc700 2023-10-18T12:26:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7444c5430 con 0x55e7445d68a0 2023-10-18T12:26:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4e0af9b80 10 client.0 _ll_register_callbacks cb 0x55e74440fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4c659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4c759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4c659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445df9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42688/0 (socket says 172.21.15.70:42688) 2023-10-18T12:26:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4c659c700 1 -- 172.21.15.70:0/3314531777 learned_addr learned my addr 172.21.15.70:0/3314531777 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.390+0000 7fa4c759e700 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4c659c700 1 -- 172.21.15.70:0/3314531777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 msgr2=0x55e7445e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4c659c700 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4c659c700 1 -- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 msgr2=0x55e7445dfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4c659c700 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445dfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4c659c700 1 -- 172.21.15.70:0/3314531777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c810dd30 con 0x55e7445dbde0 2023-10-18T12:26:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4c659c700 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445df9e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc012840 tx=0x7fa4bc012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f38a019204bc5f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4aaff5700 1 -- 172.21.15.70:0/3314531777 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc017070 con 0x55e7445dbde0 2023-10-18T12:26:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e74452df70 con 0x55e7445dbde0 2023-10-18T12:26:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.391+0000 7fa4aaff5700 1 -- 172.21.15.70:0/3314531777 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa4bc00f540 con 0x55e7445dbde0 2023-10-18T12:26:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4aaff5700 1 -- 172.21.15.70:0/3314531777 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4bc0122e0 con 0x55e7445dbde0 2023-10-18T12:26:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 10 client.15699 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e744353e60 con 0x55e7445dbde0 2023-10-18T12:26:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4aaff5700 1 -- 172.21.15.70:0/3314531777 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e74452df70 con 0x55e7445dbde0 2023-10-18T12:26:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4aaff5700 1 -- 172.21.15.70:0/3314531777 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa4bc0108a0 con 0x55e7445dbde0 2023-10-18T12:26:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 20 client.15699 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7445f01c0 0x55e7445f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 4 client.15699 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.392+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7445a5930 con 0x55e7445f01c0 2023-10-18T12:26:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.393+0000 7fa4c759e700 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7445f01c0 0x55e7445f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.393+0000 7fa4c759e700 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7445f01c0 0x55e7445f25e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78a7c71cbb8c65a server_cookie=9ca6aeba939a9f49 in_seq=0 out_seq=0 2023-10-18T12:26:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.393+0000 7fa4aaff5700 10 client.15699 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4aaff5700 1 -- 172.21.15.70:0/3314531777 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7445a5930 con 0x55e7445f01c0 2023-10-18T12:26:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4aaff5700 10 client.15699 handle_command_reply: tid=0 2023-10-18T12:26:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 client.15699 shutdown 2023-10-18T12:26:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 msgr2=0x55e7445df9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445df9e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc012840 tx=0x7fa4bc012b50 comp rx=0 tx=0).stop 2023-10-18T12:26:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 shutdown_connections 2023-10-18T12:26:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7445f01c0 0x55e7445f25e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e7445dfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4aaff5700 0 client.15699 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/3314531777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445df9e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 >> 172.21.15.70:0/3314531777 conn(0x55e74454a7a0 msgr2=0x55e7445d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 shutdown_connections 2023-10-18T12:26:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.394+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/3314531777 wait complete. 2023-10-18T12:26:58.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.395+0000 7fa4e0af9b80 20 client.15699 trim_cache size 0 max 16384 2023-10-18T12:26:58.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.415+0000 7fa4e0af9b80 1 Processor -- start 2023-10-18T12:26:58.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.415+0000 7fa4e0af9b80 1 -- start start 2023-10-18T12:26:58.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445d68a0 0x55e7445e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dc700 0x55e7445de2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7443a99a0 con 0x55e7445dbde0 2023-10-18T12:26:58.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74452df70 con 0x55e7445d68a0 2023-10-18T12:26:58.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7445d2070 con 0x55e7445dc700 2023-10-18T12:26:58.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445d68a0 0x55e7445e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42698/0 (socket says 172.21.15.70:42698) 2023-10-18T12:26:58.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 -- 172.21.15.70:0/421069723 learned_addr learned my addr 172.21.15.70:0/421069723 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c759e700 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dc700 0x55e7445de2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 -- 172.21.15.70:0/421069723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dc700 msgr2=0x55e7445de2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dc700 0x55e7445de2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 -- 172.21.15.70:0/421069723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445d68a0 msgr2=0x55e7445e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445d68a0 0x55e7445e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.416+0000 7fa4c659c700 1 -- 172.21.15.70:0/421069723 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7445df540 con 0x55e7445dbde0 2023-10-18T12:26:58.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.417+0000 7fa4c659c700 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445d5350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00a010 tx=0x7fa4bc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8622ae0087e1f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.417+0000 7fa4adffb700 1 -- 172.21.15.70:0/421069723 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc012ea0 con 0x55e7445dbde0 2023-10-18T12:26:58.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.417+0000 7fa4adffb700 1 -- 172.21.15.70:0/421069723 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4bc0050d0 con 0x55e7445dbde0 2023-10-18T12:26:58.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.417+0000 7fa4adffb700 1 -- 172.21.15.70:0/421069723 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc017040 con 0x55e7445dbde0 2023-10-18T12:26:58.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/421069723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 msgr2=0x55e7445d5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445d5350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00a010 tx=0x7fa4bc005a00 comp rx=0 tx=0).stop 2023-10-18T12:26:58.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/421069723 shutdown_connections 2023-10-18T12:26:58.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445dc700 0x55e7445de2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445d68a0 0x55e7445e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/421069723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445d5350 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/421069723 >> 172.21.15.70:0/421069723 conn(0x55e74454a7a0 msgr2=0x55e7445d5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/421069723 shutdown_connections 2023-10-18T12:26:58.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/421069723 wait complete. 2023-10-18T12:26:58.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.418+0000 7fa4e0af9b80 1 -- start start 2023-10-18T12:26:58.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 Processor -- start 2023-10-18T12:26:58.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e74463e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74463e540 con 0x55e7445dbde0 2023-10-18T12:26:58.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74463e6b0 con 0x55e7445dc700 2023-10-18T12:26:58.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74463e820 con 0x55e7445d68a0 2023-10-18T12:26:58.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4e0af9b80 10 client.0 _ll_register_callbacks cb 0x55e74440fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:58.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4c659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.419+0000 7fa4c659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42700/0 (socket says 172.21.15.70:42700) 2023-10-18T12:26:58.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 -- 172.21.15.70:0/960295483 learned_addr learned my addr 172.21.15.70:0/960295483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c759e700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 -- 172.21.15.70:0/960295483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 msgr2=0x55e7445e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c759e700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e74463e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 -- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 msgr2=0x55e74463e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e74463e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 -- 172.21.15.70:0/960295483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e744353e60 con 0x55e7445dbde0 2023-10-18T12:26:58.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c759e700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e74463e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:26:58.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4c659c700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445e6590 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc012b90 tx=0x7fa4bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958d82d5a4dd7729 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:58.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4aaff5700 1 -- 172.21.15.70:0/960295483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc017070 con 0x55e7445dbde0 2023-10-18T12:26:58.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4c0007620 con 0x55e7445dbde0 2023-10-18T12:26:58.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.420+0000 7fa4aaff5700 1 -- 172.21.15.70:0/960295483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa4bc000e40 con 0x55e7445dbde0 2023-10-18T12:26:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4aaff5700 1 -- 172.21.15.70:0/960295483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4bc012ea0 con 0x55e7445dbde0 2023-10-18T12:26:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 10 client.15705 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e74433cce0 con 0x55e7445dbde0 2023-10-18T12:26:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4aaff5700 1 -- 172.21.15.70:0/960295483 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4bc03f020 con 0x55e7445dbde0 2023-10-18T12:26:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4aaff5700 1 -- 172.21.15.70:0/960295483 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa4bc03c070 con 0x55e7445dbde0 2023-10-18T12:26:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:58.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:58.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 20 client.15705 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:58.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e744645e60 0x55e744648280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 4 client.15705 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:26:58.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.421+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e744521ed0 con 0x55e744645e60 2023-10-18T12:26:58.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.422+0000 7fa4c759e700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e744645e60 0x55e744648280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.422+0000 7fa4aaff5700 10 client.15705 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.422+0000 7fa4c759e700 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e744645e60 0x55e744648280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f940f0f248c0dd server_cookie=173d6a60a530bc88 in_seq=0 out_seq=0 2023-10-18T12:26:58.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.423+0000 7fa4aaff5700 1 -- 172.21.15.70:0/960295483 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e744521ed0 con 0x55e744645e60 2023-10-18T12:26:58.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.423+0000 7fa4aaff5700 10 client.15705 handle_command_reply: tid=0 2023-10-18T12:26:58.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.423+0000 7fa4e0af9b80 1 client.15705 shutdown 2023-10-18T12:26:58.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 msgr2=0x55e7445e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445e6590 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc012b90 tx=0x7fa4bc012000 comp rx=0 tx=0).stop 2023-10-18T12:26:58.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 shutdown_connections 2023-10-18T12:26:58.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e744645e60 0x55e744648280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e7445dc700 0x55e74463e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7445d68a0 0x55e7445e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4aaff5700 0 client.15705 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 --2- 172.21.15.70:0/960295483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e7445dbde0 0x55e7445e6590 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 >> 172.21.15.70:0/960295483 conn(0x55e74454a7a0 msgr2=0x55e7445d9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 shutdown_connections 2023-10-18T12:26:58.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 1 -- 172.21.15.70:0/960295483 wait complete. 2023-10-18T12:26:58.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.424+0000 7fa4e0af9b80 20 client.15705 trim_cache size 0 max 16384 2023-10-18T12:26:58.440 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:26:58.441 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:26:58.441 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "0cf2f440-d625-488e-9c76-f6438301c3d3", 2023-10-18T12:26:58.441 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:26:58.441 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:26:58.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4b00a1560 msgr2=0x7fa4b00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4b00a1560 0x7fa4b00a3a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55e7445dcb60 tx=0x7fa4b80d0e10 comp rx=0 tx=0).stop 2023-10-18T12:26:58.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 msgr2=0x7fa4c8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 0x7fa4c8198e90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800e9a0 tx=0x7fa4b800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 shutdown_connections 2023-10-18T12:26:58.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4c810abb0 0x7fa4c819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c8101790 0x7fa4c8198e90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4b00a1560 0x7fa4b00a3a20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 --2- 172.21.15.70:0/958991862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4c81020b0 0x7fa4c81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 >> 172.21.15.70:0/958991862 conn(0x7fa4c80fafb0 msgr2=0x7fa4c81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 shutdown_connections 2023-10-18T12:26:58.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.425+0000 7fa4cc9fe700 1 -- 172.21.15.70:0/958991862 wait complete. 2023-10-18T12:26:58.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:26:58.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 -- 172.21.15.70:0/2548709964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588010cf20 msgr2=0x7f588010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 --2- 172.21.15.70:0/2548709964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588010cf20 0x7f588010f310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f587800b0a0 tx=0x7f587800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 -- 172.21.15.70:0/2548709964 shutdown_connections 2023-10-18T12:26:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 --2- 172.21.15.70:0/2548709964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588010cf20 0x7f588010f310 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 --2- 172.21.15.70:0/2548709964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ff370 0x7f58800ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 --2- 172.21.15.70:0/2548709964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58800ffd20 0x7f588010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 -- 172.21.15.70:0/2548709964 >> 172.21.15.70:0/2548709964 conn(0x7f58800fafb0 msgr2=0x7f58800fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 -- 172.21.15.70:0/2548709964 shutdown_connections 2023-10-18T12:26:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.646+0000 7f5884d9d700 1 -- 172.21.15.70:0/2548709964 wait complete. 2023-10-18T12:26:58.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.647+0000 7f5884d9d700 1 Processor -- start 2023-10-18T12:26:58.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 -- start start 2023-10-18T12:26:58.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58800ff370 0x7f5880198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ffd20 0x7f58801993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f588010cf20 0x7f588019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019dd80 con 0x7f588010cf20 2023-10-18T12:26:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019def0 con 0x7f58800ffd20 2023-10-18T12:26:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.648+0000 7f5884d9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019e0b0 con 0x7f58800ff370 2023-10-18T12:26:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58800ff370 0x7f5880198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f588010cf20 0x7f588019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58800ff370 0x7f5880198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:56714/0 (socket says 172.21.15.70:56714) 2023-10-18T12:26:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587f7fe700 1 -- 172.21.15.70:0/3048599563 learned_addr learned my addr 172.21.15.70:0/3048599563 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587effd700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ffd20 0x7f58801993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 -- 172.21.15.70:0/3048599563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58800ff370 msgr2=0x7f5880198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58800ff370 0x7f5880198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 -- 172.21.15.70:0/3048599563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ffd20 msgr2=0x7f58801993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ffd20 0x7f58801993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 -- 172.21.15.70:0/3048599563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878009d00 con 0x7f588010cf20 2023-10-18T12:26:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587effd700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ffd20 0x7f58801993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:26:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.649+0000 7f587ffff700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f588010cf20 0x7f588019d740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f587800b070 tx=0x7f5878007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.650+0000 7f587cff9700 1 -- 172.21.15.70:0/3048599563 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800e040 con 0x7f588010cf20 2023-10-18T12:26:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.650+0000 7f587cff9700 1 -- 172.21.15.70:0/3048599563 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587801d470 con 0x7f588010cf20 2023-10-18T12:26:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.650+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588019e330 con 0x7f588010cf20 2023-10-18T12:26:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.650+0000 7f587cff9700 1 -- 172.21.15.70:0/3048599563 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780052a0 con 0x7f588010cf20 2023-10-18T12:26:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.650+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801a5bd0 con 0x7f588010cf20 2023-10-18T12:26:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.653+0000 7f587cff9700 1 -- 172.21.15.70:0/3048599563 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f587801d5e0 con 0x7f588010cf20 2023-10-18T12:26:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.654+0000 7f5899908b80 1 Processor -- start 2023-10-18T12:26:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 -- start start 2023-10-18T12:26:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f587cff9700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58680a1560 0x7f58680a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a956140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a9502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a950c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a8c36e0 con 0x55fd5a955d60 2023-10-18T12:26:58.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a8a9c60 con 0x55fd5a956680 2023-10-18T12:26:58.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f5899908b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a8a7580 con 0x55fd5a950820 2023-10-18T12:26:58.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f587cff9700 1 -- 172.21.15.70:0/3048599563 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f58780ce550 con 0x7f588010cf20 2023-10-18T12:26:58.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.655+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a956140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a956140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42716/0 (socket says 172.21.15.70:42716) 2023-10-18T12:26:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 -- 172.21.15.70:0/577563425 learned_addr learned my addr 172.21.15.70:0/577563425 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587effd700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58680a1560 0x7f58680a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587ffff700 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a950c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587effd700 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a9502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 -- 172.21.15.70:0/577563425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 msgr2=0x55fd5a950c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a950c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 -- 172.21.15.70:0/577563425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 msgr2=0x55fd5a9502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a9502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 -- 172.21.15.70:0/577563425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd5a8603f0 con 0x55fd5a955d60 2023-10-18T12:26:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587ffff700 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a950c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:26:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.656+0000 7f587f7fe700 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a956140 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5870009e50 tx=0x7f5870011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3f182bc79d4a76 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.657+0000 7f587effd700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58680a1560 0x7f58680a3a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55fd5a956ae0 tx=0x7f5874009750 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.657+0000 7f5865ffb700 1 -- 172.21.15.70:0/577563425 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870004a30 con 0x55fd5a955d60 2023-10-18T12:26:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.657+0000 7f5865ffb700 1 -- 172.21.15.70:0/577563425 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58700054e0 con 0x55fd5a955d60 2023-10-18T12:26:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.657+0000 7f5865ffb700 1 -- 172.21.15.70:0/577563425 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870019930 con 0x55fd5a955d60 2023-10-18T12:26:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.657+0000 7f5899908b80 1 -- 172.21.15.70:0/577563425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 msgr2=0x55fd5a956140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.657+0000 7f5899908b80 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a956140 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5870009e50 tx=0x7f5870011720 comp rx=0 tx=0).stop 2023-10-18T12:26:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 -- 172.21.15.70:0/577563425 shutdown_connections 2023-10-18T12:26:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a950c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a956140 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 --2- 172.21.15.70:0/577563425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a9502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 -- 172.21.15.70:0/577563425 >> 172.21.15.70:0/577563425 conn(0x55fd5a8c4720 msgr2=0x55fd5a94acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 -- 172.21.15.70:0/577563425 shutdown_connections 2023-10-18T12:26:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 -- 172.21.15.70:0/577563425 wait complete. 2023-10-18T12:26:58.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 -- start start 2023-10-18T12:26:58.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.658+0000 7f5899908b80 1 Processor -- start 2023-10-18T12:26:58.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a950820 0x55fd5a95c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a955d60 0x55fd5a959ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 0x55fd5a95a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a8a2e70 con 0x55fd5a956680 2023-10-18T12:26:58.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a83f710 con 0x55fd5a950820 2023-10-18T12:26:58.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a83f3b0 con 0x55fd5a955d60 2023-10-18T12:26:58.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f5899908b80 10 client.0 _ll_register_callbacks cb 0x55fd5a789a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:58.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f587ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 0x55fd5a95a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f587effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a955d60 0x55fd5a959ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f587ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 0x55fd5a95a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42722/0 (socket says 172.21.15.70:42722) 2023-10-18T12:26:58.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f587ffff700 1 -- 172.21.15.70:0/2746929415 learned_addr learned my addr 172.21.15.70:0/2746929415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.659+0000 7f587f7fe700 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a950820 0x55fd5a95c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f587ffff700 1 -- 172.21.15.70:0/2746929415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a955d60 msgr2=0x55fd5a959ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f587ffff700 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a955d60 0x55fd5a959ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f587ffff700 1 -- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a950820 msgr2=0x55fd5a95c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f587ffff700 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a950820 0x55fd5a95c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f587ffff700 1 -- 172.21.15.70:0/2746929415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5870009a80 con 0x55fd5a956680 2023-10-18T12:26:58.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f587ffff700 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 0x55fd5a95a200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5878009fd0 tx=0x7f587809ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6918f497a661ee57 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f585effd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:58.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f585effd700 1 -- 172.21.15.70:0/2746929415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780ce070 con 0x55fd5a956680 2023-10-18T12:26:58.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f585effd700 1 -- 172.21.15.70:0/2746929415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f587809b320 con 0x55fd5a956680 2023-10-18T12:26:58.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd5a8a7ef0 con 0x55fd5a956680 2023-10-18T12:26:58.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f585effd700 1 -- 172.21.15.70:0/2746929415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f587809f470 con 0x55fd5a956680 2023-10-18T12:26:58.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f5899908b80 10 client.15717 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:58.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.660+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fd5a6cdde0 con 0x55fd5a956680 2023-10-18T12:26:58.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f585effd700 1 -- 172.21.15.70:0/2746929415 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd5a8a7ef0 con 0x55fd5a956680 2023-10-18T12:26:58.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f585effd700 1 -- 172.21.15.70:0/2746929415 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f587809b580 con 0x55fd5a956680 2023-10-18T12:26:58.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f5899908b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:58.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f5899908b80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:58.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f5899908b80 20 client.15717 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:58.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f5899908b80 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a969d50 0x55fd5a96c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f5899908b80 4 client.15717 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:26:58.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd5a91f8b0 con 0x55fd5a969d50 2023-10-18T12:26:58.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.661+0000 7f587f7fe700 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a969d50 0x55fd5a96c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.662+0000 7f587f7fe700 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a969d50 0x55fd5a96c170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c06b9a4eb18a2f server_cookie=63ca979b8426140e in_seq=0 out_seq=0 2023-10-18T12:26:58.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.662+0000 7f585effd700 10 client.15717 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f585effd700 1 -- 172.21.15.70:0/2746929415 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd5a91f8b0 con 0x55fd5a969d50 2023-10-18T12:26:58.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f585effd700 10 client.15717 handle_command_reply: tid=0 2023-10-18T12:26:58.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 client.15717 shutdown 2023-10-18T12:26:58.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 msgr2=0x55fd5a95a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 0x55fd5a95a200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5878009fd0 tx=0x7f587809ad00 comp rx=0 tx=0).stop 2023-10-18T12:26:58.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 shutdown_connections 2023-10-18T12:26:58.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a969d50 0x55fd5a96c170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a956680 0x55fd5a95a200 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a950820 0x55fd5a95c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f585effd700 0 client.15717 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 --2- 172.21.15.70:0/2746929415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a955d60 0x55fd5a959ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 >> 172.21.15.70:0/2746929415 conn(0x55fd5a8c4720 msgr2=0x55fd5a94f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 shutdown_connections 2023-10-18T12:26:58.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 1 -- 172.21.15.70:0/2746929415 wait complete. 2023-10-18T12:26:58.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.663+0000 7f5899908b80 20 client.15717 trim_cache size 0 max 16384 2023-10-18T12:26:58.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.681+0000 7f5899908b80 1 Processor -- start 2023-10-18T12:26:58.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.681+0000 7f5899908b80 1 -- start start 2023-10-18T12:26:58.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.681+0000 7f5899908b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a961060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.681+0000 7f5899908b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a95c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f5899908b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a95c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f5899908b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a945d40 con 0x55fd5a955d60 2023-10-18T12:26:58.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f5899908b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a723920 con 0x55fd5a956680 2023-10-18T12:26:58.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f5899908b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a95ce10 con 0x55fd5a950820 2023-10-18T12:26:58.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a95c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a961060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a961060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:56762/0 (socket says 172.21.15.70:56762) 2023-10-18T12:26:58.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a95c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42732/0 (socket says 172.21.15.70:42732) 2023-10-18T12:26:58.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 -- 172.21.15.70:0/861727589 learned_addr learned my addr 172.21.15.70:0/861727589 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587ffff700 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a95c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 -- 172.21.15.70:0/861727589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 msgr2=0x55fd5a95c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a95c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 -- 172.21.15.70:0/861727589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 msgr2=0x55fd5a95c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a95c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587f7fe700 1 -- 172.21.15.70:0/861727589 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd5a6b6c60 con 0x55fd5a950820 2023-10-18T12:26:58.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.682+0000 7f587ffff700 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a95c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:26:58.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.683+0000 7f587f7fe700 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a961060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f587001b040 tx=0x7f5870011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df5e7762e869a13a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.683+0000 7f58667fc700 1 -- 172.21.15.70:0/861727589 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870029040 con 0x55fd5a950820 2023-10-18T12:26:58.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.683+0000 7f58667fc700 1 -- 172.21.15.70:0/861727589 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587002a440 con 0x55fd5a950820 2023-10-18T12:26:58.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.683+0000 7f58667fc700 1 -- 172.21.15.70:0/861727589 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587002ad90 con 0x55fd5a950820 2023-10-18T12:26:58.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 -- 172.21.15.70:0/861727589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 msgr2=0x55fd5a961060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a961060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f587001b040 tx=0x7f5870011720 comp rx=0 tx=0).stop 2023-10-18T12:26:58.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 -- 172.21.15.70:0/861727589 shutdown_connections 2023-10-18T12:26:58.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a956680 0x55fd5a95c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a950820 0x55fd5a961060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 --2- 172.21.15.70:0/861727589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a955d60 0x55fd5a95c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 -- 172.21.15.70:0/861727589 >> 172.21.15.70:0/861727589 conn(0x55fd5a8c4720 msgr2=0x55fd5a94c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 -- 172.21.15.70:0/861727589 shutdown_connections 2023-10-18T12:26:58.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 -- 172.21.15.70:0/861727589 wait complete. 2023-10-18T12:26:58.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.684+0000 7f5899908b80 1 -- start start 2023-10-18T12:26:58.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 Processor -- start 2023-10-18T12:26:58.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 0x55fd5a95f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a955d60 0x55fd5a95fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a956680 0x55fd5a960310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a9bdb90 con 0x55fd5a950820 2023-10-18T12:26:58.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a9bdcd0 con 0x55fd5a955d60 2023-10-18T12:26:58.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd5a9bde10 con 0x55fd5a956680 2023-10-18T12:26:58.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f5899908b80 10 client.0 _ll_register_callbacks cb 0x55fd5a789a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:26:58.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.685+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 0x55fd5a95f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 0x55fd5a95f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42748/0 (socket says 172.21.15.70:42748) 2023-10-18T12:26:58.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 -- 172.21.15.70:0/4192359806 learned_addr learned my addr 172.21.15.70:0/4192359806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:26:58.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587ffff700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a956680 0x55fd5a960310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587effd700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a955d60 0x55fd5a95fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 -- 172.21.15.70:0/4192359806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a956680 msgr2=0x55fd5a960310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a956680 0x55fd5a960310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 -- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a955d60 msgr2=0x55fd5a95fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:26:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a955d60 0x55fd5a95fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 -- 172.21.15.70:0/4192359806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd5a8603f0 con 0x55fd5a950820 2023-10-18T12:26:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f587f7fe700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 0x55fd5a95f890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5870009340 tx=0x7f587002a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d4799017a33a52 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:26:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.686+0000 7f585effd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:26:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f585effd700 1 -- 172.21.15.70:0/4192359806 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870029040 con 0x55fd5a950820 2023-10-18T12:26:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f585effd700 1 -- 172.21.15.70:0/4192359806 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5870004f40 con 0x55fd5a950820 2023-10-18T12:26:58.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58700054e0 con 0x55fd5a950820 2023-10-18T12:26:58.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f585effd700 1 -- 172.21.15.70:0/4192359806 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5870005100 con 0x55fd5a950820 2023-10-18T12:26:58.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 10 client.15723 fetch_fsmap learned FSMap version 20 2023-10-18T12:26:58.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fd5a6f4250 con 0x55fd5a950820 2023-10-18T12:26:58.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f585effd700 1 -- 172.21.15.70:0/4192359806 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58700054e0 con 0x55fd5a950820 2023-10-18T12:26:58.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f585effd700 1 -- 172.21.15.70:0/4192359806 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5870011440 con 0x55fd5a950820 2023-10-18T12:26:58.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:26:58.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:26:58.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 20 client.15723 populate_metadata read hostname 'smithi070' 2023-10-18T12:26:58.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.687+0000 7f5899908b80 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a9bf1e0 0x55fd5a9c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:26:58.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.688+0000 7f5899908b80 4 client.15723 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:26:58.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.688+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fd5a89be50 con 0x55fd5a9bf1e0 2023-10-18T12:26:58.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.688+0000 7f587effd700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a9bf1e0 0x55fd5a9c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:26:58.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f585effd700 10 client.15723 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f587effd700 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a9bf1e0 0x55fd5a9c1600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca307f3b4338e27 server_cookie=21f1d10a0ddf3102 in_seq=0 out_seq=0 2023-10-18T12:26:58.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f585effd700 1 -- 172.21.15.70:0/4192359806 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fd5a89be50 con 0x55fd5a9bf1e0 2023-10-18T12:26:58.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f585effd700 10 client.15723 handle_command_reply: tid=0 2023-10-18T12:26:58.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 client.15723 shutdown 2023-10-18T12:26:58.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 msgr2=0x55fd5a95f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 0x55fd5a95f890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5870009340 tx=0x7f587002a350 comp rx=0 tx=0).stop 2023-10-18T12:26:58.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 shutdown_connections 2023-10-18T12:26:58.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd5a9bf1e0 0x55fd5a9c1600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd5a956680 0x55fd5a960310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd5a950820 0x55fd5a95f890 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 --2- 172.21.15.70:0/4192359806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd5a955d60 0x55fd5a95fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f585effd700 0 client.15723 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:26:58.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 >> 172.21.15.70:0/4192359806 conn(0x55fd5a8c4720 msgr2=0x55fd5a944700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 shutdown_connections 2023-10-18T12:26:58.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.689+0000 7f5899908b80 1 -- 172.21.15.70:0/4192359806 wait complete. 2023-10-18T12:26:58.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5899908b80 20 client.15723 trim_cache size 0 max 16384 2023-10-18T12:26:58.710 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:26:58.710 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:26:58.711 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:26:58.711 INFO:teuthology.orchestra.run.smithi070.stdout: "0cf2f440-d625-488e-9c76-f6438301c3d3": { 2023-10-18T12:26:58.711 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:26:58.711 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "0cf2f440-d625-488e-9c76-f6438301c3d3", 2023-10-18T12:26:58.711 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:26:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:26:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:26:58.712 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:26:58.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58680a1560 msgr2=0x7f58680a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58680a1560 0x7f58680a3a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55fd5a956ae0 tx=0x7f5874009750 comp rx=0 tx=0).stop 2023-10-18T12:26:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f588010cf20 msgr2=0x7f588019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:26:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f588010cf20 0x7f588019d740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f587800b070 tx=0x7f5878007860 comp rx=0 tx=0).stop 2023-10-18T12:26:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 shutdown_connections 2023-10-18T12:26:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f588010cf20 0x7f588019d740 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58800ff370 0x7f5880198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f58680a1560 0x7f58680a3a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 --2- 172.21.15.70:0/3048599563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800ffd20 0x7f58801993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:26:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.690+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 >> 172.21.15.70:0/3048599563 conn(0x7f58800fafb0 msgr2=0x7f58800fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:26:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.691+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 shutdown_connections 2023-10-18T12:26:58.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:26:58.691+0000 7f5884d9d700 1 -- 172.21.15.70:0/3048599563 wait complete. 2023-10-18T12:26:58.716 INFO:tasks.cephfs.filesystem:scrub status for tag:0cf2f440-d625-488e-9c76-f6438301c3d3 - {'path': '/', 'tag': '0cf2f440-d625-488e-9c76-f6438301c3d3', 'options': 'recursive,force'} 2023-10-18T12:26:59.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:59 smithi186 ceph-mon[135253]: pgmap v343: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:59.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:59 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:26:59.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:59 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:59.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:59 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:26:59.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:26:59 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:59.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:59 smithi070 ceph-mon[98639]: pgmap v343: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:59.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:59 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:26:59.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:59 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:59.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:59 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:26:59.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:26:59 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:59 smithi152 ceph-mon[134767]: pgmap v343: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:26:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:59 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:26:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:59 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:26:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:59 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:26:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:26:59 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:01.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:27:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:27:00.755+0000 7f32e80f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:27:01.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:27:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:27:00.755+0000 7f32e80f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:27:01.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:27:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:27:00.755+0000 7f32e80f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:27:01.106 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:27:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:27:00.755+0000 7f32e80f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:27:01.106 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:27:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:27:00.755+0000 7f32e80f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:27:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:01 smithi186 ceph-mon[135253]: pgmap v344: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:01.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:01 smithi070 ceph-mon[98639]: pgmap v344: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:01 smithi152 ceph-mon[134767]: pgmap v344: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:03 smithi070 ceph-mon[98639]: pgmap v345: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 853 B/s wr, 3 op/s 2023-10-18T12:27:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:03 smithi070 ceph-mon[98639]: mgrmap e24: x(active, since 8m), standbys: z, y 2023-10-18T12:27:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:03 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:27:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:03 smithi152 ceph-mon[134767]: pgmap v345: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 853 B/s wr, 3 op/s 2023-10-18T12:27:03.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:03 smithi152 ceph-mon[134767]: mgrmap e24: x(active, since 8m), standbys: z, y 2023-10-18T12:27:03.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:03 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:27:03.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:03 smithi186 ceph-mon[135253]: pgmap v345: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 853 B/s wr, 3 op/s 2023-10-18T12:27:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:03 smithi186 ceph-mon[135253]: mgrmap e24: x(active, since 8m), standbys: z, y 2023-10-18T12:27:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:03 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:27:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:05 smithi070 ceph-mon[98639]: pgmap v346: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 3 op/s 2023-10-18T12:27:05.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:05 smithi152 ceph-mon[134767]: pgmap v346: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 3 op/s 2023-10-18T12:27:05.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:05 smithi186 ceph-mon[135253]: pgmap v346: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 3 op/s 2023-10-18T12:27:07.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:07 smithi070 ceph-mon[98639]: pgmap v347: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 3 op/s 2023-10-18T12:27:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:07 smithi152 ceph-mon[134767]: pgmap v347: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 3 op/s 2023-10-18T12:27:07.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:07 smithi186 ceph-mon[135253]: pgmap v347: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 3 op/s 2023-10-18T12:27:08.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:08 smithi070 ceph-mon[98639]: pgmap v348: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-18T12:27:08.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:08 smithi152 ceph-mon[134767]: pgmap v348: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-18T12:27:08.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:08 smithi186 ceph-mon[135253]: pgmap v348: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-18T12:27:11.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:11 smithi186 ceph-mon[135253]: pgmap v349: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-18T12:27:11.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:11 smithi070 ceph-mon[98639]: pgmap v349: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-18T12:27:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:11 smithi152 ceph-mon[134767]: pgmap v349: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-18T12:27:13.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:13 smithi186 ceph-mon[135253]: pgmap v350: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-18T12:27:13.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:13 smithi070 ceph-mon[98639]: pgmap v350: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-18T12:27:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:13 smithi152 ceph-mon[134767]: pgmap v350: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-18T12:27:15.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:15 smithi186 ceph-mon[135253]: pgmap v351: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-18T12:27:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:15 smithi070 ceph-mon[98639]: pgmap v351: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-18T12:27:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:15 smithi152 ceph-mon[134767]: pgmap v351: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-18T12:27:17.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:17 smithi186 ceph-mon[135253]: pgmap v352: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-10-18T12:27:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:17 smithi070 ceph-mon[98639]: pgmap v352: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-10-18T12:27:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:17 smithi152 ceph-mon[134767]: pgmap v352: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-10-18T12:27:19.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:19 smithi186 ceph-mon[135253]: pgmap v353: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-10-18T12:27:19.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:19 smithi070 ceph-mon[98639]: pgmap v353: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-10-18T12:27:19.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:19 smithi152 ceph-mon[134767]: pgmap v353: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-10-18T12:27:21.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:21 smithi186 ceph-mon[135253]: pgmap v354: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-18T12:27:21.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:21 smithi070 ceph-mon[98639]: pgmap v354: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-18T12:27:21.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:21 smithi152 ceph-mon[134767]: pgmap v354: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-18T12:27:23.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:23 smithi186 ceph-mon[135253]: pgmap v355: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-18T12:27:23.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:23 smithi070 ceph-mon[98639]: pgmap v355: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-18T12:27:23.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:23 smithi152 ceph-mon[134767]: pgmap v355: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-18T12:27:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:25 smithi186 ceph-mon[135253]: pgmap v356: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:25.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:25 smithi070 ceph-mon[98639]: pgmap v356: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:25.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:25 smithi152 ceph-mon[134767]: pgmap v356: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:27.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:27 smithi186 ceph-mon[135253]: pgmap v357: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:27.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:27 smithi070 ceph-mon[98639]: pgmap v357: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:27.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:27 smithi152 ceph-mon[134767]: pgmap v357: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:28.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:27:28.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 -- 172.21.15.70:0/3944799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4072c20 msgr2=0x7f3cc4073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/3944799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4072c20 0x7f3cc4073000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009990 tx=0x7f3cb4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 -- 172.21.15.70:0/3944799167 shutdown_connections 2023-10-18T12:27:28.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/3944799167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc410cf50 0x7f3cc410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/3944799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4072c20 0x7f3cc4073000 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/3944799167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cc4073540 0x7f3cc410ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 -- 172.21.15.70:0/3944799167 >> 172.21.15.70:0/3944799167 conn(0x7f3cc40fc200 msgr2=0x7f3cc40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:28.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 -- 172.21.15.70:0/3944799167 shutdown_connections 2023-10-18T12:27:28.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.924+0000 7f3cc99ff700 1 -- 172.21.15.70:0/3944799167 wait complete. 2023-10-18T12:27:28.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.925+0000 7f3cc99ff700 1 Processor -- start 2023-10-18T12:27:28.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.925+0000 7f3cc99ff700 1 -- start start 2023-10-18T12:27:28.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cc4072c20 0x7f3cc4198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 0x7f3cc41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc410cf50 0x7f3cc419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419dcb0 con 0x7f3cc4073540 2023-10-18T12:27:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419de20 con 0x7f3cc4072c20 2023-10-18T12:27:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419dfe0 con 0x7f3cc410cf50 2023-10-18T12:27:28.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 0x7f3cc41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 0x7f3cc41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50454/0 (socket says 172.21.15.70:50454) 2023-10-18T12:27:28.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc410cf50 0x7f3cc419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 -- 172.21.15.70:0/615246406 learned_addr learned my addr 172.21.15.70:0/615246406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:28.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cc4072c20 0x7f3cc4198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 -- 172.21.15.70:0/615246406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc410cf50 msgr2=0x7f3cc419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc410cf50 0x7f3cc419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 -- 172.21.15.70:0/615246406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cc4072c20 msgr2=0x7f3cc4198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cc4072c20 0x7f3cc4198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.926+0000 7f3cc27fc700 1 -- 172.21.15.70:0/615246406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40095f0 con 0x7f3cc4073540 2023-10-18T12:27:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.927+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 0x7f3cc41992d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e9a0 tx=0x7f3cb800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.927+0000 7f3cc89fd700 1 -- 172.21.15.70:0/615246406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8009790 con 0x7f3cc4073540 2023-10-18T12:27:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.927+0000 7f3cc89fd700 1 -- 172.21.15.70:0/615246406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb8010430 con 0x7f3cc4073540 2023-10-18T12:27:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.927+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc419e2c0 con 0x7f3cc4073540 2023-10-18T12:27:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.927+0000 7f3cc89fd700 1 -- 172.21.15.70:0/615246406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8018680 con 0x7f3cc4073540 2023-10-18T12:27:28.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.927+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc41a5b00 con 0x7f3cc4073540 2023-10-18T12:27:28.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.930+0000 7f3cc89fd700 1 -- 172.21.15.70:0/615246406 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3cb80105a0 con 0x7f3cc4073540 2023-10-18T12:27:28.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.931+0000 7f3cdd3d6b80 1 Processor -- start 2023-10-18T12:27:28.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 -- start start 2023-10-18T12:27:28.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cc89fd700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cac0a15f0 0x7f3cac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cc89fd700 1 -- 172.21.15.70:0/615246406 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3cb8014070 con 0x7f3cc4073540 2023-10-18T12:27:28.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d506e0 con 0x5640c3de2be0 2023-10-18T12:27:28.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d36c60 con 0x5640c3de3500 2023-10-18T12:27:28.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d34580 con 0x5640c3ddd5d0 2023-10-18T12:27:28.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cac0a15f0 0x7f3cac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.932+0000 7f3cc2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50456/0 (socket says 172.21.15.70:50456) 2023-10-18T12:27:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/1298350577 learned_addr learned my addr 172.21.15.70:0/1298350577 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/1298350577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3ddd5d0 msgr2=0x5640c3ddda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/1298350577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 msgr2=0x5640c3ddd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/1298350577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c3ced3f0 con 0x5640c3de2be0 2023-10-18T12:27:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:27:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de2fc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005ec0 tx=0x7f3cb4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81028ab197b8c5a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cac0a15f0 0x7f3cac0a3ab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00f660 tx=0x7f3cbc009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.933+0000 7f3ca9ffb700 1 -- 172.21.15.70:0/1298350577 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb401d070 con 0x5640c3de2be0 2023-10-18T12:27:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.934+0000 7f3ca9ffb700 1 -- 172.21.15.70:0/1298350577 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb401e040 con 0x5640c3de2be0 2023-10-18T12:27:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.934+0000 7f3ca9ffb700 1 -- 172.21.15.70:0/1298350577 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4004920 con 0x5640c3de2be0 2023-10-18T12:27:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.934+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1298350577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 msgr2=0x5640c3de2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.934+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de2fc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005ec0 tx=0x7f3cb4005ef0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1298350577 shutdown_connections 2023-10-18T12:27:28.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de2fc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1298350577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1298350577 >> 172.21.15.70:0/1298350577 conn(0x5640c3d51720 msgr2=0x5640c3dd7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1298350577 shutdown_connections 2023-10-18T12:27:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1298350577 wait complete. 2023-10-18T12:27:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.935+0000 7f3cdd3d6b80 1 -- start start 2023-10-18T12:27:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 Processor -- start 2023-10-18T12:27:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3de93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de3500 0x5640c3de6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d2fe70 con 0x5640c3de2be0 2023-10-18T12:27:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ccc710 con 0x5640c3ddd5d0 2023-10-18T12:27:28.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ccc3b0 con 0x5640c3de3500 2023-10-18T12:27:28.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cdd3d6b80 10 client.0 _ll_register_callbacks cb 0x5640c3c16a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:28.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cc27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cc2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3de93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cc27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50466/0 (socket says 172.21.15.70:50466) 2023-10-18T12:27:28.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cc27fc700 1 -- 172.21.15.70:0/1170339193 learned_addr learned my addr 172.21.15.70:0/1170339193 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:28.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.936+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de3500 0x5640c3de6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cc27fc700 1 -- 172.21.15.70:0/1170339193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de3500 msgr2=0x5640c3de6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de3500 0x5640c3de6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cc27fc700 1 -- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 msgr2=0x5640c3de93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3de93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cc27fc700 1 -- 172.21.15.70:0/1170339193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc4110080 con 0x5640c3de2be0 2023-10-18T12:27:28.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de66c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80caa70 tx=0x7f3cb809a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73be78feecfb35ae server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:28.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3ca2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:27:28.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/1170339193 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80a2630 con 0x5640c3de2be0 2023-10-18T12:27:28.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/1170339193 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3cb8010880 con 0x5640c3de2be0 2023-10-18T12:27:28.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640c3d34ef0 con 0x5640c3de2be0 2023-10-18T12:27:28.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.937+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/1170339193 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb80cd240 con 0x5640c3de2be0 2023-10-18T12:27:28.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 10 client.15741 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:28.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5640c3b5ade0 con 0x5640c3de2be0 2023-10-18T12:27:28.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/1170339193 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640c3d34ef0 con 0x5640c3de2be0 2023-10-18T12:27:28.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/1170339193 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3cb8099200 con 0x5640c3de2be0 2023-10-18T12:27:28.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:28.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:28.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 20 client.15741 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:28.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3df6e50 0x5640c3df9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 4 client.15741 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:27:28.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.938+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640c3dac8b0 con 0x5640c3df6e50 2023-10-18T12:27:28.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.939+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3df6e50 0x5640c3df9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.939+0000 7f3ca2ffd700 10 client.15741 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:28.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.939+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3df6e50 0x5640c3df9270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b66445b008c1a20 server_cookie=c1b75652d02e828a in_seq=0 out_seq=0 2023-10-18T12:27:28.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/1170339193 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5640c3dac8b0 con 0x5640c3df6e50 2023-10-18T12:27:28.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3ca2ffd700 10 client.15741 handle_command_reply: tid=0 2023-10-18T12:27:28.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 client.15741 shutdown 2023-10-18T12:27:28.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 msgr2=0x5640c3de66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de66c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80caa70 tx=0x7f3cb809a2a0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 shutdown_connections 2023-10-18T12:27:28.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3df6e50 0x5640c3df9270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de3500 0x5640c3de6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3de93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/1170339193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de2be0 0x5640c3de66c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3ca2ffd700 0 client.15741 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:28.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 >> 172.21.15.70:0/1170339193 conn(0x5640c3d51720 msgr2=0x5640c3dd7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:28.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 shutdown_connections 2023-10-18T12:27:28.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.940+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/1170339193 wait complete. 2023-10-18T12:27:28.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.941+0000 7f3cdd3d6b80 20 client.15741 trim_cache size 0 max 16384 2023-10-18T12:27:28.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 Processor -- start 2023-10-18T12:27:28.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 -- start start 2023-10-18T12:27:28.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 0x5640c3de4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3bb0920 con 0x5640c3de3500 2023-10-18T12:27:28.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3dac8e0 con 0x5640c3ddd5d0 2023-10-18T12:27:28.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3dde030 con 0x5640c3de2be0 2023-10-18T12:27:28.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 0x5640c3de4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 0x5640c3de4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50470/0 (socket says 172.21.15.70:50470) 2023-10-18T12:27:28.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cc37fe700 1 -- 172.21.15.70:0/3206609946 learned_addr learned my addr 172.21.15.70:0/3206609946 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:28.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.958+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc37fe700 1 -- 172.21.15.70:0/3206609946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 msgr2=0x5640c3dede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc37fe700 1 -- 172.21.15.70:0/3206609946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 msgr2=0x5640c3ddd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc37fe700 1 -- 172.21.15.70:0/3206609946 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c3b43c60 con 0x5640c3de3500 2023-10-18T12:27:28.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:28.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:27:28.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 0x5640c3de4ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc018600 tx=0x7f3cbc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ebbc58c18969ac server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:28.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3caa7fc700 1 -- 172.21.15.70:0/3206609946 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00e040 con 0x5640c3de3500 2023-10-18T12:27:28.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3caa7fc700 1 -- 172.21.15.70:0/3206609946 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cbc00f040 con 0x5640c3de3500 2023-10-18T12:27:28.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.959+0000 7f3caa7fc700 1 -- 172.21.15.70:0/3206609946 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc015860 con 0x5640c3de3500 2023-10-18T12:27:28.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3206609946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 msgr2=0x5640c3de4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 0x5640c3de4ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc018600 tx=0x7f3cbc014000 comp rx=0 tx=0).stop 2023-10-18T12:27:28.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3206609946 shutdown_connections 2023-10-18T12:27:28.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3de3500 0x5640c3de4ff0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3ddd5d0 0x5640c3ddd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3206609946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3206609946 >> 172.21.15.70:0/3206609946 conn(0x5640c3d51720 msgr2=0x5640c3de08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:28.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3206609946 shutdown_connections 2023-10-18T12:27:28.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.960+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3206609946 wait complete. 2023-10-18T12:27:28.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 -- start start 2023-10-18T12:27:28.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 Processor -- start 2023-10-18T12:27:28.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 0x5640c3dee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dde9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ddf5f0 con 0x5640c3ddd5d0 2023-10-18T12:27:28.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ddf760 con 0x5640c3de3500 2023-10-18T12:27:28.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3e4aca0 con 0x5640c3de2be0 2023-10-18T12:27:28.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.961+0000 7f3cdd3d6b80 10 client.0 _ll_register_callbacks cb 0x5640c3c16a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:28.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 0x5640c3dee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 0x5640c3dee040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50478/0 (socket says 172.21.15.70:50478) 2023-10-18T12:27:28.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/3914118785 learned_addr learned my addr 172.21.15.70:0/3914118785 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:28.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc37fe700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dde9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/3914118785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 msgr2=0x5640c3dde9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dde9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 msgr2=0x5640c3ddef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:28.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 -- 172.21.15.70:0/3914118785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640c3b5ade0 con 0x5640c3ddd5d0 2023-10-18T12:27:28.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3cc2ffd700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 0x5640c3dee040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009990 tx=0x7f3cb4005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5302af94625acd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:28.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.962+0000 7f3ca2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:27:28.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/3914118785 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4032070 con 0x5640c3ddd5d0 2023-10-18T12:27:28.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/3914118785 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3cb4034520 con 0x5640c3ddd5d0 2023-10-18T12:27:28.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cb401e040 con 0x5640c3ddd5d0 2023-10-18T12:27:28.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/3914118785 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb401d040 con 0x5640c3ddd5d0 2023-10-18T12:27:28.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 10 client.15747 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:28.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f3cbc01e040 con 0x5640c3ddd5d0 2023-10-18T12:27:28.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/3914118785 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb401e040 con 0x5640c3ddd5d0 2023-10-18T12:27:28.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/3914118785 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3cb400bb30 con 0x5640c3ddd5d0 2023-10-18T12:27:28.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:28.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:28.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 20 client.15747 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:28.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3e4bfe0 0x5640c3e4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:28.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 4 client.15747 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:27:28.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.963+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5640c3d28e50 con 0x5640c3e4bfe0 2023-10-18T12:27:28.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.964+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3e4bfe0 0x5640c3e4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:28.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.964+0000 7f3ca2ffd700 10 client.15747 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:28.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.964+0000 7f3cc27fc700 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3e4bfe0 0x5640c3e4e400 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd3cfdb229aec64 server_cookie=bd56f49cba51dbca in_seq=0 out_seq=0 2023-10-18T12:27:28.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.964+0000 7f3ca2ffd700 1 -- 172.21.15.70:0/3914118785 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5640c3d28e50 con 0x5640c3e4bfe0 2023-10-18T12:27:28.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3ca2ffd700 10 client.15747 handle_command_reply: tid=0 2023-10-18T12:27:28.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 client.15747 shutdown 2023-10-18T12:27:28.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 msgr2=0x5640c3dee040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 0x5640c3dee040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009990 tx=0x7f3cb4005440 comp rx=0 tx=0).stop 2023-10-18T12:27:28.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 shutdown_connections 2023-10-18T12:27:28.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5640c3e4bfe0 0x5640c3e4e400 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5640c3de3500 0x5640c3ddef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5640c3ddd5d0 0x5640c3dee040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3ca2ffd700 0 client.15747 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:28.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 --2- 172.21.15.70:0/3914118785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5640c3de2be0 0x5640c3dde9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 >> 172.21.15.70:0/3914118785 conn(0x5640c3d51720 msgr2=0x5640c3de4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:28.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 shutdown_connections 2023-10-18T12:27:28.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 1 -- 172.21.15.70:0/3914118785 wait complete. 2023-10-18T12:27:28.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.965+0000 7f3cdd3d6b80 20 client.15747 trim_cache size 0 max 16384 2023-10-18T12:27:28.983 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:27:28.984 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:27:28.984 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:27:28.984 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:27:28.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cac0a15f0 msgr2=0x7f3cac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cac0a15f0 0x7f3cac0a3ab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00f660 tx=0x7f3cbc009920 comp rx=0 tx=0).stop 2023-10-18T12:27:28.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 msgr2=0x7f3cc41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:28.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 0x7f3cc41992d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e9a0 tx=0x7f3cb800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 shutdown_connections 2023-10-18T12:27:28.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc410cf50 0x7f3cc419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cac0a15f0 0x7f3cac0a3ab0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cc4072c20 0x7f3cc4198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 --2- 172.21.15.70:0/615246406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cc4073540 0x7f3cc41992d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:28.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 >> 172.21.15.70:0/615246406 conn(0x7f3cc40fc200 msgr2=0x7f3cc40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:28.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 shutdown_connections 2023-10-18T12:27:28.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:28.966+0000 7f3cc99ff700 1 -- 172.21.15.70:0/615246406 wait complete. 2023-10-18T12:27:28.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:27:28.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:27:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 -- 172.21.15.70:0/1988598958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 msgr2=0x7fedb4106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 --2- 172.21.15.70:0/1988598958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 0x7fedb4106aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009990 tx=0x7fedb0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 -- 172.21.15.70:0/1988598958 shutdown_connections 2023-10-18T12:27:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 --2- 172.21.15.70:0/1988598958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb410d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 --2- 172.21.15.70:0/1988598958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 0x7fedb4106aa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 --2- 172.21.15.70:0/1988598958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 0x7fedb4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 -- 172.21.15.70:0/1988598958 >> 172.21.15.70:0/1988598958 conn(0x7fedb40fc1c0 msgr2=0x7fedb40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 -- 172.21.15.70:0/1988598958 shutdown_connections 2023-10-18T12:27:29.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.200+0000 7fedbbbff700 1 -- 172.21.15.70:0/1988598958 wait complete. 2023-10-18T12:27:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 Processor -- start 2023-10-18T12:27:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 -- start start 2023-10-18T12:27:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 0x7fedb4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 0x7fedb4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41a4e60 con 0x7fedb41066c0 2023-10-18T12:27:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41a4fd0 con 0x7fedb41006b0 2023-10-18T12:27:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.201+0000 7fedbbbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41a5190 con 0x7fedb4101070 2023-10-18T12:27:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb999b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 0x7fedb4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb4073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50268/0 (socket says 172.21.15.70:50268) 2023-10-18T12:27:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 -- 172.21.15.70:0/2976750167 learned_addr learned my addr 172.21.15.70:0/2976750167 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb999b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 0x7fedb4072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39692/0 (socket says 172.21.15.70:39692) 2023-10-18T12:27:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedba19c700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 0x7fedb4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 -- 172.21.15.70:0/2976750167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 msgr2=0x7fedb4072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 0x7fedb4072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 -- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 msgr2=0x7fedb4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 0x7fedb4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.202+0000 7fedb919a700 1 -- 172.21.15.70:0/2976750167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00095f0 con 0x7fedb4101070 2023-10-18T12:27:29.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.203+0000 7fedb919a700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb4073030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feda400e9a0 tx=0x7feda400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.205+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4009790 con 0x7fedb4101070 2023-10-18T12:27:29.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.205+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda4010430 con 0x7fedb4101070 2023-10-18T12:27:29.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.205+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4018680 con 0x7fedb4101070 2023-10-18T12:27:29.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.205+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb41a5470 con 0x7fedb4101070 2023-10-18T12:27:29.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.205+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb41a5930 con 0x7fedb4101070 2023-10-18T12:27:29.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.206+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb4108cb0 con 0x7fedb4101070 2023-10-18T12:27:29.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.208+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7feda40105a0 con 0x7fedb4101070 2023-10-18T12:27:29.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.209+0000 7fedaaffd700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda00a1670 0x7feda00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.209+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7feda4014070 con 0x7fedb4101070 2023-10-18T12:27:29.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.210+0000 7fedb999b700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda00a1670 0x7feda00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.210+0000 7fedb999b700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda00a1670 0x7feda00a3b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fedb000b2e0 tx=0x7fedb0005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.212+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feda40965a0 con 0x7fedb4101070 2023-10-18T12:27:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:29 smithi070 ceph-mon[98639]: pgmap v358: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:27:29.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.463+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedb4101df0 con 0x7fedb4101070 2023-10-18T12:27:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.464+0000 7fedaaffd700 1 -- 172.21.15.70:0/2976750167 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7feda4099bc0 con 0x7fedb4101070 2023-10-18T12:27:29.467 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:27:29.467 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:27:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:29 smithi186 ceph-mon[135253]: pgmap v358: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:27:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda00a1670 msgr2=0x7feda00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda00a1670 0x7feda00a3b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fedb000b2e0 tx=0x7fedb0005960 comp rx=0 tx=0).stop 2023-10-18T12:27:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 msgr2=0x7fedb4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb4073030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feda400e9a0 tx=0x7feda400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 shutdown_connections 2023-10-18T12:27:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb41066c0 0x7fedb4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda00a1670 0x7feda00a3b30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb41006b0 0x7fedb4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 --2- 172.21.15.70:0/2976750167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb4101070 0x7fedb4073030 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 >> 172.21.15.70:0/2976750167 conn(0x7fedb40fc1c0 msgr2=0x7fedb40fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 shutdown_connections 2023-10-18T12:27:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.466+0000 7fedbbbff700 1 -- 172.21.15.70:0/2976750167 wait complete. 2023-10-18T12:27:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:27:29.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:27:29.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:29 smithi152 ceph-mon[134767]: pgmap v358: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-18T12:27:29.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.691+0000 7fda9bdff700 1 -- 172.21.15.70:0/1925673438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda941010e0 msgr2=0x7fda9410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.691+0000 7fda9bdff700 1 --2- 172.21.15.70:0/1925673438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda941010e0 0x7fda9410d5f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00b0a0 tx=0x7fda8c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 -- 172.21.15.70:0/1925673438 shutdown_connections 2023-10-18T12:27:29.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 --2- 172.21.15.70:0/1925673438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda941010e0 0x7fda9410d5f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 --2- 172.21.15.70:0/1925673438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda94106730 0x7fda94106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 --2- 172.21.15.70:0/1925673438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda94100720 0x7fda94100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 -- 172.21.15.70:0/1925673438 >> 172.21.15.70:0/1925673438 conn(0x7fda940fc1b0 msgr2=0x7fda940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 -- 172.21.15.70:0/1925673438 shutdown_connections 2023-10-18T12:27:29.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.692+0000 7fda9bdff700 1 -- 172.21.15.70:0/1925673438 wait complete. 2023-10-18T12:27:29.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 Processor -- start 2023-10-18T12:27:29.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 -- start start 2023-10-18T12:27:29.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda94100720 0x7fda94198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda941010e0 0x7fda941992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 0x7fda9419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9419dc90 con 0x7fda94106730 2023-10-18T12:27:29.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9419de00 con 0x7fda94100720 2023-10-18T12:27:29.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9419dfc0 con 0x7fda941010e0 2023-10-18T12:27:29.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.693+0000 7fda9a39c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 0x7fda9419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9939a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda941010e0 0x7fda941992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda94100720 0x7fda94198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 0x7fda9419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50524/0 (socket says 172.21.15.70:50524) 2023-10-18T12:27:29.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 -- 172.21.15.70:0/4260463313 learned_addr learned my addr 172.21.15.70:0/4260463313 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:29.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 -- 172.21.15.70:0/4260463313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda941010e0 msgr2=0x7fda941992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda941010e0 0x7fda941992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 -- 172.21.15.70:0/4260463313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda94100720 msgr2=0x7fda94198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda94100720 0x7fda94198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 -- 172.21.15.70:0/4260463313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c009d00 con 0x7fda94106730 2023-10-18T12:27:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.694+0000 7fda9a39c700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 0x7fda9419d650 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fda8c007b60 tx=0x7fda8c005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.695+0000 7fda8affd700 1 -- 172.21.15.70:0/4260463313 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00e040 con 0x7fda94106730 2023-10-18T12:27:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.695+0000 7fda8affd700 1 -- 172.21.15.70:0/4260463313 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda8c012dc0 con 0x7fda94106730 2023-10-18T12:27:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.695+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9419e240 con 0x7fda94106730 2023-10-18T12:27:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.695+0000 7fda8affd700 1 -- 172.21.15.70:0/4260463313 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c01b450 con 0x7fda94106730 2023-10-18T12:27:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.695+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda941a5ae0 con 0x7fda94106730 2023-10-18T12:27:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.699+0000 7fdaaf766b80 1 Processor -- start 2023-10-18T12:27:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.699+0000 7fda8affd700 1 -- 172.21.15.70:0/4260463313 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fda8c01d680 con 0x7fda94106730 2023-10-18T12:27:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.699+0000 7fdaaf766b80 1 -- start start 2023-10-18T12:27:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1462fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa13d06e0 con 0x560aa145d5d0 2023-10-18T12:27:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa13b6c60 con 0x560aa1462be0 2023-10-18T12:27:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa13b4580 con 0x560aa1463500 2023-10-18T12:27:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fda9939a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50302/0 (socket says 172.21.15.70:50302) 2023-10-18T12:27:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fda9939a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50540/0 (socket says 172.21.15.70:50540) 2023-10-18T12:27:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fda99b9b700 1 -- 172.21.15.70:0/822511090 learned_addr learned my addr 172.21.15.70:0/822511090 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.700+0000 7fda9939a700 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1462fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda99b9b700 1 -- 172.21.15.70:0/822511090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 msgr2=0x560aa1462fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda99b9b700 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1462fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda99b9b700 1 -- 172.21.15.70:0/822511090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 msgr2=0x560aa145da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda99b9b700 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda99b9b700 1 -- 172.21.15.70:0/822511090 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa136d3f0 con 0x560aa1463500 2023-10-18T12:27:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda8affd700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda800a15f0 0x7fda800a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda99b9b700 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145d090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda90007c70 tx=0x7fda90007f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621e09febbcdcbf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda8affd700 1 -- 172.21.15.70:0/4260463313 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fda8c0cf180 con 0x7fda94106730 2023-10-18T12:27:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda9939a700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda800a15f0 0x7fda800a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda7ffff700 1 -- 172.21.15.70:0/822511090 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90010be0 con 0x560aa1463500 2023-10-18T12:27:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda7ffff700 1 -- 172.21.15.70:0/822511090 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda90015070 con 0x560aa1463500 2023-10-18T12:27:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.701+0000 7fda7ffff700 1 -- 172.21.15.70:0/822511090 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90019630 con 0x560aa1463500 2023-10-18T12:27:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 -- 172.21.15.70:0/822511090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 msgr2=0x560aa145d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fda9939a700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda800a15f0 0x7fda800a3ab0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda84005d90 tx=0x7fda84005d20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145d090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda90007c70 tx=0x7fda90007f80 comp rx=0 tx=0).stop 2023-10-18T12:27:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 -- 172.21.15.70:0/822511090 shutdown_connections 2023-10-18T12:27:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1462fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/822511090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145d090 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 -- 172.21.15.70:0/822511090 >> 172.21.15.70:0/822511090 conn(0x560aa13d1720 msgr2=0x560aa1457b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 -- 172.21.15.70:0/822511090 shutdown_connections 2023-10-18T12:27:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.702+0000 7fdaaf766b80 1 -- 172.21.15.70:0/822511090 wait complete. 2023-10-18T12:27:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.703+0000 7fdaaf766b80 1 -- start start 2023-10-18T12:27:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.703+0000 7fdaaf766b80 1 Processor -- start 2023-10-18T12:27:29.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa14694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1462be0 0x560aa1466740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1463500 0x560aa1466d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa13afe70 con 0x560aa145d5d0 2023-10-18T12:27:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa134c710 con 0x560aa1463500 2023-10-18T12:27:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa134c3b0 con 0x560aa1462be0 2023-10-18T12:27:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fdaaf766b80 10 client.0 _ll_register_callbacks cb 0x560aa1296a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa14694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa14694c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50554/0 (socket says 172.21.15.70:50554) 2023-10-18T12:27:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fda9939a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1462be0 0x560aa1466740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.704+0000 7fda99b9b700 1 -- 172.21.15.70:0/643670586 learned_addr learned my addr 172.21.15.70:0/643670586 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda9a39c700 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1463500 0x560aa1466d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda99b9b700 1 -- 172.21.15.70:0/643670586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1462be0 msgr2=0x560aa1466740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda99b9b700 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1462be0 0x560aa1466740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda99b9b700 1 -- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1463500 msgr2=0x560aa1466d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda99b9b700 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1463500 0x560aa1466d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda99b9b700 1 -- 172.21.15.70:0/643670586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda900078a0 con 0x560aa145d5d0 2023-10-18T12:27:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda99b9b700 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa14694c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fda900106b0 tx=0x7fda90005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97e7e0e39155bb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:27:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda7d7fa700 1 -- 172.21.15.70:0/643670586 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90010b50 con 0x560aa145d5d0 2023-10-18T12:27:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fda7d7fa700 1 -- 172.21.15.70:0/643670586 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fda90010cb0 con 0x560aa145d5d0 2023-10-18T12:27:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.705+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560aa13b4ef0 con 0x560aa145d5d0 2023-10-18T12:27:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fda7d7fa700 1 -- 172.21.15.70:0/643670586 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda90010e70 con 0x560aa145d5d0 2023-10-18T12:27:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 10 client.15765 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560aa11dade0 con 0x560aa145d5d0 2023-10-18T12:27:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fda7d7fa700 1 -- 172.21.15.70:0/643670586 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560aa13b4ef0 con 0x560aa145d5d0 2023-10-18T12:27:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fda7d7fa700 1 -- 172.21.15.70:0/643670586 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda90004750 con 0x560aa145d5d0 2023-10-18T12:27:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 20 client.15765 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa1476dc0 0x560aa14791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 4 client.15765 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:27:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.706+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560aa142c8b0 con 0x560aa1476dc0 2023-10-18T12:27:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.707+0000 7fda9939a700 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa1476dc0 0x560aa14791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.707+0000 7fda7d7fa700 10 client.15765 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.707+0000 7fda9939a700 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa1476dc0 0x560aa14791e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26d2841ef2b33e2 server_cookie=b93971ecefea00b1 in_seq=0 out_seq=0 2023-10-18T12:27:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fda7d7fa700 1 -- 172.21.15.70:0/643670586 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560aa142c8b0 con 0x560aa1476dc0 2023-10-18T12:27:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fda7d7fa700 10 client.15765 handle_command_reply: tid=0 2023-10-18T12:27:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 client.15765 shutdown 2023-10-18T12:27:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 msgr2=0x560aa14694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa14694c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fda900106b0 tx=0x7fda90005990 comp rx=0 tx=0).stop 2023-10-18T12:27:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 shutdown_connections 2023-10-18T12:27:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa1476dc0 0x560aa14791e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1463500 0x560aa1466d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fda7d7fa700 0 client.15765 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa14694c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/643670586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1462be0 0x560aa1466740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 >> 172.21.15.70:0/643670586 conn(0x560aa13d1720 msgr2=0x560aa1457ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.708+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 shutdown_connections 2023-10-18T12:27:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.709+0000 7fdaaf766b80 1 -- 172.21.15.70:0/643670586 wait complete. 2023-10-18T12:27:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.709+0000 7fdaaf766b80 20 client.15765 trim_cache size 0 max 16384 2023-10-18T12:27:29.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.722+0000 7fdaaf766b80 1 Processor -- start 2023-10-18T12:27:29.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.722+0000 7fdaaf766b80 1 -- start start 2023-10-18T12:27:29.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa146deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1468ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa1465fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa1458130 con 0x560aa145d5d0 2023-10-18T12:27:29.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa1230920 con 0x560aa1462be0 2023-10-18T12:27:29.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa146e420 con 0x560aa1463500 2023-10-18T12:27:29.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa146deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda9939a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1468ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa146deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50560/0 (socket says 172.21.15.70:50560) 2023-10-18T12:27:29.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 -- 172.21.15.70:0/3793294767 learned_addr learned my addr 172.21.15.70:0/3793294767 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:29.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda9a39c700 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa1465fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 -- 172.21.15.70:0/3793294767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 msgr2=0x560aa1465fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa1465fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 -- 172.21.15.70:0/3793294767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 msgr2=0x560aa1468ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1468ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda99b9b700 1 -- 172.21.15.70:0/3793294767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa11c3c60 con 0x560aa145d5d0 2023-10-18T12:27:29.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.723+0000 7fda9939a700 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1468ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:29.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.724+0000 7fda99b9b700 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa146deb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda90007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44104f8405a13f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.724+0000 7fda88bff700 1 -- 172.21.15.70:0/3793294767 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900109e0 con 0x560aa145d5d0 2023-10-18T12:27:29.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.724+0000 7fda88bff700 1 -- 172.21.15.70:0/3793294767 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda90011440 con 0x560aa145d5d0 2023-10-18T12:27:29.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.724+0000 7fda88bff700 1 -- 172.21.15.70:0/3793294767 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900050b0 con 0x560aa145d5d0 2023-10-18T12:27:29.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 -- 172.21.15.70:0/3793294767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 msgr2=0x560aa146deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa146deb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda90007520 comp rx=0 tx=0).stop 2023-10-18T12:27:29.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 -- 172.21.15.70:0/3793294767 shutdown_connections 2023-10-18T12:27:29.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa1465fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa146deb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/3793294767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa1468ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 -- 172.21.15.70:0/3793294767 >> 172.21.15.70:0/3793294767 conn(0x560aa13d1720 msgr2=0x560aa1465030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 -- 172.21.15.70:0/3793294767 shutdown_connections 2023-10-18T12:27:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 -- 172.21.15.70:0/3793294767 wait complete. 2023-10-18T12:27:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.725+0000 7fdaaf766b80 1 -- start start 2023-10-18T12:27:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 Processor -- start 2023-10-18T12:27:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa145e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa145f0f0 con 0x560aa145d5d0 2023-10-18T12:27:29.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa145f260 con 0x560aa1462be0 2023-10-18T12:27:29.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa145f420 con 0x560aa1463500 2023-10-18T12:27:29.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fdaaf766b80 10 client.0 _ll_register_callbacks cb 0x560aa1296a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:29.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fda99b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50568/0 (socket says 172.21.15.70:50568) 2023-10-18T12:27:29.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fda99b9b700 1 -- 172.21.15.70:0/950240530 learned_addr learned my addr 172.21.15.70:0/950240530 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:29.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.726+0000 7fda9939a700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa145e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda9a39c700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda99b9b700 1 -- 172.21.15.70:0/950240530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 msgr2=0x560aa145ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda99b9b700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda99b9b700 1 -- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 msgr2=0x560aa145e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:29.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda99b9b700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa145e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda99b9b700 1 -- 172.21.15.70:0/950240530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8401a040 con 0x560aa145d5d0 2023-10-18T12:27:29.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda9939a700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa145e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:29.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda99b9b700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145fdb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda90005860 tx=0x7fda90005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e163d83141c401 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:29.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:27:29.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda7d7fa700 1 -- 172.21.15.70:0/950240530 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900117c0 con 0x560aa145d5d0 2023-10-18T12:27:29.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda90015070 con 0x560aa145d5d0 2023-10-18T12:27:29.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.727+0000 7fda7d7fa700 1 -- 172.21.15.70:0/950240530 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fda90000e40 con 0x560aa145d5d0 2023-10-18T12:27:29.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fda7d7fa700 1 -- 172.21.15.70:0/950240530 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda90011920 con 0x560aa145d5d0 2023-10-18T12:27:29.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 10 client.15771 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:29.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560aa11dade0 con 0x560aa145d5d0 2023-10-18T12:27:29.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fda7d7fa700 1 -- 172.21.15.70:0/950240530 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda90015070 con 0x560aa145d5d0 2023-10-18T12:27:29.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fda7d7fa700 1 -- 172.21.15.70:0/950240530 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda900052a0 con 0x560aa145d5d0 2023-10-18T12:27:29.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:29.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:29.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 20 client.15771 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:29.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa14cbc40 0x560aa14ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:29.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 4 client.15771 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:27:29.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.728+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560aa13a8e50 con 0x560aa14cbc40 2023-10-18T12:27:29.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.729+0000 7fda9939a700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa14cbc40 0x560aa14ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:29.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.729+0000 7fda7d7fa700 10 client.15771 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:29.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.729+0000 7fda9939a700 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa14cbc40 0x560aa14ce030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357f3a3f9199e7d server_cookie=397b60fd404c800a in_seq=0 out_seq=0 2023-10-18T12:27:29.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fda7d7fa700 1 -- 172.21.15.70:0/950240530 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560aa13a8e50 con 0x560aa14cbc40 2023-10-18T12:27:29.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fda7d7fa700 10 client.15771 handle_command_reply: tid=0 2023-10-18T12:27:29.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 client.15771 shutdown 2023-10-18T12:27:29.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 msgr2=0x560aa145fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145fdb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda90005860 tx=0x7fda90005fd0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 shutdown_connections 2023-10-18T12:27:29.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560aa14cbc40 0x560aa14ce030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560aa1463500 0x560aa145ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560aa145d5d0 0x560aa145fdb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fda7d7fa700 0 client.15771 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 --2- 172.21.15.70:0/950240530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560aa1462be0 0x560aa145e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 >> 172.21.15.70:0/950240530 conn(0x560aa13d1720 msgr2=0x560aa14514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 shutdown_connections 2023-10-18T12:27:29.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 1 -- 172.21.15.70:0/950240530 wait complete. 2023-10-18T12:27:29.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.730+0000 7fdaaf766b80 20 client.15771 trim_cache size 0 max 16384 2023-10-18T12:27:29.748 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:27:29.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda800a15f0 msgr2=0x7fda800a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda800a15f0 0x7fda800a3ab0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda84005d90 tx=0x7fda84005d20 comp rx=0 tx=0).stop 2023-10-18T12:27:29.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 msgr2=0x7fda9419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:29.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 0x7fda9419d650 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fda8c007b60 tx=0x7fda8c005a60 comp rx=0 tx=0).stop 2023-10-18T12:27:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 shutdown_connections 2023-10-18T12:27:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda94106730 0x7fda9419d650 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda800a15f0 0x7fda800a3ab0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda94100720 0x7fda94198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 --2- 172.21.15.70:0/4260463313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda941010e0 0x7fda941992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 >> 172.21.15.70:0/4260463313 conn(0x7fda940fc1b0 msgr2=0x7fda9410be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:29.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 shutdown_connections 2023-10-18T12:27:29.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:29.731+0000 7fda9bdff700 1 -- 172.21.15.70:0/4260463313 wait complete. 2023-10-18T12:27:30.330 INFO:tasks.workunit.client.0.smithi070.stderr:Updating files: 63% (7642/12063) Updating files: 64% (7721/12063) Updating files: 65% (7841/12063) Updating files: 66% (7962/12063) Updating files: 67% (8083/12063) Updating files: 68% (8203/12063) Updating files: 69% (8324/12063) Updating files: 70% (8445/12063) Updating files: 71% (8565/12063) Updating files: 72% (8686/12063) Updating files: 73% (8806/12063) Updating files: 74% (8927/12063) Updating files: 75% (9048/12063) Updating files: 76% (9168/12063) Updating files: 77% (9289/12063) Updating files: 78% (9410/12063) Updating files: 79% (9530/12063) Updating files: 80% (9651/12063) Updating files: 81% (9772/12063) Updating files: 82% (9892/12063) Updating files: 83% (10013/12063) Updating files: 84% (10133/12063) Updating files: 85% (10254/12063) Updating files: 86% (10375/12063) Updating files: 87% (10495/12063) Updating files: 88% (10616/12063) Updating files: 89% (10737/12063) Updating files: 90% (10857/12063) Updating files: 91% (10978/12063) Updating files: 92% (11098/12063) Updating files: 93% (11219/12063) Updating files: 94% (11340/12063) Updating files: 95% (11460/12063) Updating files: 96% (11581/12063) Updating files: 97% (11702/12063) Updating files: 98% (11822/12063) Updating files: 99% (11943/12063) Updating files: 100% (12063/12063) Updating files: 100% (12063/12063), done. 2023-10-18T12:27:30.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:30 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2976750167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:27:30.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:30 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:27:30.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:30 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2976750167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:27:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:30 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:27:30.629 INFO:tasks.workunit.client.0.smithi070.stderr:Note: switching to 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'. 2023-10-18T12:27:30.629 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.629 INFO:tasks.workunit.client.0.smithi070.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-18T12:27:30.629 INFO:tasks.workunit.client.0.smithi070.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-18T12:27:30.630 INFO:tasks.workunit.client.0.smithi070.stderr:state without impacting any branches by switching back to a branch. 2023-10-18T12:27:30.630 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.630 INFO:tasks.workunit.client.0.smithi070.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-18T12:27:30.630 INFO:tasks.workunit.client.0.smithi070.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-18T12:27:30.631 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.631 INFO:tasks.workunit.client.0.smithi070.stderr: git switch -c 2023-10-18T12:27:30.631 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.631 INFO:tasks.workunit.client.0.smithi070.stderr:Or undo this operation with: 2023-10-18T12:27:30.631 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.632 INFO:tasks.workunit.client.0.smithi070.stderr: git switch - 2023-10-18T12:27:30.632 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.632 INFO:tasks.workunit.client.0.smithi070.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-18T12:27:30.632 INFO:tasks.workunit.client.0.smithi070.stderr: 2023-10-18T12:27:30.632 INFO:tasks.workunit.client.0.smithi070.stderr:HEAD is now at f7420d7f8e5 Merge PR #48130 into wip-batrick-testing-20231016.203825 2023-10-18T12:27:30.635 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-18T12:27:30.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:30 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2976750167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:27:30.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:30 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:27:30.685 INFO:tasks.workunit.client.0.smithi070.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-18T12:27:30.690 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-18T12:27:30.690 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-18T12:27:30.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:27:30.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:27:30.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1044962656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08100720 msgr2=0x7f2b08100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:30.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1044962656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08100720 0x7f2b08100ba0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009990 tx=0x7f2afc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1044962656 shutdown_connections 2023-10-18T12:27:30.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1044962656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 0x7f2b0810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1044962656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08106730 0x7f2b08106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1044962656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08100720 0x7f2b08100ba0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1044962656 >> 172.21.15.70:0/1044962656 conn(0x7f2b080fc1b0 msgr2=0x7f2b080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:30.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1044962656 shutdown_connections 2023-10-18T12:27:30.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.962+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1044962656 wait complete. 2023-10-18T12:27:30.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.963+0000 7f2b0cfff700 1 Processor -- start 2023-10-18T12:27:30.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 -- start start 2023-10-18T12:27:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08100720 0x7f2b0819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 0x7f2b0819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08106730 0x7f2b08198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08199250 con 0x7f2b08106730 2023-10-18T12:27:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b081993c0 con 0x7f2b081010e0 2023-10-18T12:27:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08199580 con 0x7f2b08100720 2023-10-18T12:27:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08100720 0x7f2b0819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08100720 0x7f2b0819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50336/0 (socket says 172.21.15.70:50336) 2023-10-18T12:27:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b0659c700 1 -- 172.21.15.70:0/1745054773 learned_addr learned my addr 172.21.15.70:0/1745054773 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 0x7f2b0819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.964+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08106730 0x7f2b08198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1745054773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08100720 msgr2=0x7f2b0819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:30.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08100720 0x7f2b0819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08106730 msgr2=0x7f2b08198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:30.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08106730 0x7f2b08198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1745054773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0095f0 con 0x7f2b081010e0 2023-10-18T12:27:30.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08106730 0x7f2b08198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:30.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 0x7f2b0819efb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2afc000c00 tx=0x7f2afc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:30.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2af77fe700 1 -- 172.21.15.70:0/1745054773 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc01d070 con 0x7f2b081010e0 2023-10-18T12:27:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2af77fe700 1 -- 172.21.15.70:0/1745054773 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2afc021410 con 0x7f2b081010e0 2023-10-18T12:27:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2af77fe700 1 -- 172.21.15.70:0/1745054773 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc017650 con 0x7f2b081010e0 2023-10-18T12:27:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.965+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b08199800 con 0x7f2b081010e0 2023-10-18T12:27:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.966+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b081a5a60 con 0x7f2b081010e0 2023-10-18T12:27:30.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 Processor -- start 2023-10-18T12:27:30.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 -- start start 2023-10-18T12:27:30.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2af77fe700 1 -- 172.21.15.70:0/1745054773 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2afc021580 con 0x7f2b081010e0 2023-10-18T12:27:30.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d93040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d8d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761d00760 con 0x55d761d93580 2023-10-18T12:27:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761ce6ce0 con 0x55d761d8d650 2023-10-18T12:27:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.970+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761ce4600 con 0x55d761d92c60 2023-10-18T12:27:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d8d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d93040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d93040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50350/0 (socket says 172.21.15.70:50350) 2023-10-18T12:27:30.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 -- 172.21.15.70:0/872533213 learned_addr learned my addr 172.21.15.70:0/872533213 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:30.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 -- 172.21.15.70:0/872533213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 msgr2=0x55d761d8dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d8dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 -- 172.21.15.70:0/872533213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 msgr2=0x55d761d8d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d8d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.971+0000 7f2b06d9d700 1 -- 172.21.15.70:0/872533213 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761c9d470 con 0x55d761d92c60 2023-10-18T12:27:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d93040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b00003eb0 tx=0x7f2b00009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2247a91897e3bd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:30.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2af4ff9700 1 -- 172.21.15.70:0/872533213 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000092d0 con 0x55d761d92c60 2023-10-18T12:27:30.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2af4ff9700 1 -- 172.21.15.70:0/872533213 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b0000f040 con 0x55d761d92c60 2023-10-18T12:27:30.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2af4ff9700 1 -- 172.21.15.70:0/872533213 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000e8b0 con 0x55d761d92c60 2023-10-18T12:27:30.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2af77fe700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2af00a15f0 0x7f2af00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2af77fe700 1 -- 172.21.15.70:0/1745054773 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2afc0ce680 con 0x7f2b081010e0 2023-10-18T12:27:30.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.972+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2af00a15f0 0x7f2af00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 -- 172.21.15.70:0/872533213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 msgr2=0x55d761d93040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:30.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d93040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b00003eb0 tx=0x7f2b00009920 comp rx=0 tx=0).stop 2023-10-18T12:27:30.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 -- 172.21.15.70:0/872533213 shutdown_connections 2023-10-18T12:27:30.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d93040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/872533213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d8d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 -- 172.21.15.70:0/872533213 >> 172.21.15.70:0/872533213 conn(0x55d761d017a0 msgr2=0x55d761d87be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:30.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 -- 172.21.15.70:0/872533213 shutdown_connections 2023-10-18T12:27:30.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b2085ab80 1 -- 172.21.15.70:0/872533213 wait complete. 2023-10-18T12:27:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.973+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2af00a15f0 0x7f2af00a3ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d761d88af0 tx=0x7f2af8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.974+0000 7f2b2085ab80 1 -- start start 2023-10-18T12:27:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.974+0000 7f2b2085ab80 1 Processor -- start 2023-10-18T12:27:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.974+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.974+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 0x55d761d96820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.974+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d93580 0x55d761d96f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761cdfef0 con 0x55d761d92c60 2023-10-18T12:27:30.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761c7c790 con 0x55d761d8d650 2023-10-18T12:27:30.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761c7c430 con 0x55d761d93580 2023-10-18T12:27:30.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b2085ab80 10 client.0 _ll_register_callbacks cb 0x55d761bc6b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:30.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 0x55d761d96820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 0x55d761d96820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50604/0 (socket says 172.21.15.70:50604) 2023-10-18T12:27:30.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1576864298 learned_addr learned my addr 172.21.15.70:0/1576864298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:30.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39784/0 (socket says 172.21.15.70:39784) 2023-10-18T12:27:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d93580 0x55d761d96f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1576864298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d93580 msgr2=0x55d761d96f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:30.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d93580 0x55d761d96f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 msgr2=0x55d761d995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 -- 172.21.15.70:0/1576864298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b00003b90 con 0x55d761d92c60 2023-10-18T12:27:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 0x55d761d96820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2afc012040 tx=0x7f2afc021b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33fb1a3e854110e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.975+0000 7f2aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:27:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2aedffb700 1 -- 172.21.15.70:0/1576864298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0976d0 con 0x55d761d92c60 2023-10-18T12:27:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2aedffb700 1 -- 172.21.15.70:0/1576864298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2afc09ba90 con 0x55d761d92c60 2023-10-18T12:27:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d761ce4f70 con 0x55d761d92c60 2023-10-18T12:27:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2aedffb700 1 -- 172.21.15.70:0/1576864298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2afc0b1050 con 0x55d761d92c60 2023-10-18T12:27:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2b2085ab80 10 client.15783 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d761b0ae60 con 0x55d761d92c60 2023-10-18T12:27:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2aedffb700 1 -- 172.21.15.70:0/1576864298 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d761ce4f70 con 0x55d761d92c60 2023-10-18T12:27:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2aedffb700 1 -- 172.21.15.70:0/1576864298 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2afc0ce2f0 con 0x55d761d92c60 2023-10-18T12:27:30.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2b2085ab80 10 client.15783 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:30.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.976+0000 7f2b2085ab80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:30.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2b2085ab80 20 client.15783 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:30.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761daf3a0 0x55d761db17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:30.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2b2085ab80 4 client.15783 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:27:30.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d761d5c930 con 0x55d761daf3a0 2023-10-18T12:27:30.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761daf3a0 0x55d761db17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:30.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761daf3a0 0x55d761db17c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220fdc40764b10bc server_cookie=501cba7adbbc3154 in_seq=0 out_seq=0 2023-10-18T12:27:30.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.977+0000 7f2aedffb700 10 client.15783 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:30.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2aedffb700 1 -- 172.21.15.70:0/1576864298 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d761d5c930 con 0x55d761daf3a0 2023-10-18T12:27:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2aedffb700 10 client.15783 handle_command_reply: tid=0 2023-10-18T12:27:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 client.15783 shutdown 2023-10-18T12:27:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 msgr2=0x55d761d96820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 0x55d761d96820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2afc012040 tx=0x7f2afc021b40 comp rx=0 tx=0).stop 2023-10-18T12:27:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 shutdown_connections 2023-10-18T12:27:30.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761daf3a0 0x55d761db17c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d93580 0x55d761d96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761d995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2aedffb700 0 client.15783 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:30.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1576864298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d92c60 0x55d761d96820 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.978+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 >> 172.21.15.70:0/1576864298 conn(0x55d761d017a0 msgr2=0x55d761d8c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.979+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 shutdown_connections 2023-10-18T12:27:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.979+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1576864298 wait complete. 2023-10-18T12:27:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.979+0000 7f2b2085ab80 20 client.15783 trim_cache size 0 max 16384 2023-10-18T12:27:31.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 Processor -- start 2023-10-18T12:27:31.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 -- start start 2023-10-18T12:27:31.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761da6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d990d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761b609a0 con 0x55d761d93580 2023-10-18T12:27:31.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.997+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761d5c960 con 0x55d761d8d650 2023-10-18T12:27:31.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761d800e0 con 0x55d761d92c60 2023-10-18T12:27:31.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761da6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761da6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39798/0 (socket says 172.21.15.70:39798) 2023-10-18T12:27:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 -- 172.21.15.70:0/1055847236 learned_addr learned my addr 172.21.15.70:0/1055847236 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 -- 172.21.15.70:0/1055847236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 msgr2=0x55d761d92040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d92040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 -- 172.21.15.70:0/1055847236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 msgr2=0x55d761d990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b0659c700 1 -- 172.21.15.70:0/1055847236 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761af3ce0 con 0x55d761d8d650 2023-10-18T12:27:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.998+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.999+0000 7f2b0659c700 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761da6820 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2af80096b0 tx=0x7f2af8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaeeb7e7911f5bf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.999+0000 7f2af57fa700 1 -- 172.21.15.70:0/1055847236 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8024040 con 0x55d761d8d650 2023-10-18T12:27:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.999+0000 7f2af57fa700 1 -- 172.21.15.70:0/1055847236 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af8029dc0 con 0x55d761d8d650 2023-10-18T12:27:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:30.999+0000 7f2af57fa700 1 -- 172.21.15.70:0/1055847236 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8017070 con 0x55d761d8d650 2023-10-18T12:27:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1055847236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 msgr2=0x55d761da6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761da6820 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2af80096b0 tx=0x7f2af8009f90 comp rx=0 tx=0).stop 2023-10-18T12:27:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1055847236 shutdown_connections 2023-10-18T12:27:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d93580 0x55d761d990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d8d650 0x55d761da6820 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/1055847236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761d92040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1055847236 >> 172.21.15.70:0/1055847236 conn(0x55d761d017a0 msgr2=0x55d761d91930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1055847236 shutdown_connections 2023-10-18T12:27:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.000+0000 7f2b2085ab80 1 -- 172.21.15.70:0/1055847236 wait complete. 2023-10-18T12:27:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 -- start start 2023-10-18T12:27:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 Processor -- start 2023-10-18T12:27:31.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d8d650 0x55d761dfc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761dfc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 0x55d761da5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761da5e80 con 0x55d761d8d650 2023-10-18T12:27:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761da5ff0 con 0x55d761d93580 2023-10-18T12:27:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761da61b0 con 0x55d761d92c60 2023-10-18T12:27:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.001+0000 7f2b2085ab80 10 client.0 _ll_register_callbacks cb 0x55d761bc6b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 0x55d761da5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b0659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d8d650 0x55d761dfc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 0x55d761da5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39808/0 (socket says 172.21.15.70:39808) 2023-10-18T12:27:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 -- 172.21.15.70:0/886660258 learned_addr learned my addr 172.21.15.70:0/886660258 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b05d9b700 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761dfc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 -- 172.21.15.70:0/886660258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 msgr2=0x55d761dfc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761dfc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 -- 172.21.15.70:0/886660258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d8d650 msgr2=0x55d761dfc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d8d650 0x55d761dfc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 -- 172.21.15.70:0/886660258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d761c9d470 con 0x55d761d93580 2023-10-18T12:27:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2b06d9d700 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 0x55d761da5860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b00004600 tx=0x7f2b00007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b470f849f7b869ab server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.002+0000 7f2aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:27:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2aedffb700 1 -- 172.21.15.70:0/886660258 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00007b00 con 0x55d761d93580 2023-10-18T12:27:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b0000f040 con 0x55d761d93580 2023-10-18T12:27:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2aedffb700 1 -- 172.21.15.70:0/886660258 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2b00007c60 con 0x55d761d93580 2023-10-18T12:27:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2aedffb700 1 -- 172.21.15.70:0/886660258 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b0000e4f0 con 0x55d761d93580 2023-10-18T12:27:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 10 client.25802 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f2af8012cd0 con 0x55d761d93580 2023-10-18T12:27:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2aedffb700 1 -- 172.21.15.70:0/886660258 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b0000f040 con 0x55d761d93580 2023-10-18T12:27:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2aedffb700 1 -- 172.21.15.70:0/886660258 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2b000287e0 con 0x55d761d93580 2023-10-18T12:27:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 10 client.25802 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 20 client.25802 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761e05510 0x55d761e07930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 4 client.25802 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:27:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.003+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d761cd8ed0 con 0x55d761e05510 2023-10-18T12:27:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.004+0000 7f2b0659c700 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761e05510 0x55d761e07930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.004+0000 7f2aedffb700 10 client.25802 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.004+0000 7f2b0659c700 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761e05510 0x55d761e07930 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a109a8f67e56e8d server_cookie=e189910d17014e3d in_seq=0 out_seq=0 2023-10-18T12:27:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2aedffb700 1 -- 172.21.15.70:0/886660258 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d761cd8ed0 con 0x55d761e05510 2023-10-18T12:27:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2aedffb700 10 client.25802 handle_command_reply: tid=0 2023-10-18T12:27:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2b2085ab80 1 client.25802 shutdown 2023-10-18T12:27:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 msgr2=0x55d761da5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 0x55d761da5860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b00004600 tx=0x7f2b00007500 comp rx=0 tx=0).stop 2023-10-18T12:27:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 shutdown_connections 2023-10-18T12:27:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.005+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d761e05510 0x55d761e07930 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d761d93580 0x55d761da5860 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d761d8d650 0x55d761dfc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 1 --2- 172.21.15.70:0/886660258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d761d92c60 0x55d761dfc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 >> 172.21.15.70:0/886660258 conn(0x55d761d017a0 msgr2=0x55d761d950f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2aedffb700 0 client.25802 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 shutdown_connections 2023-10-18T12:27:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 1 -- 172.21.15.70:0/886660258 wait complete. 2023-10-18T12:27:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.006+0000 7f2b2085ab80 20 client.25802 trim_cache size 0 max 16384 2023-10-18T12:27:31.021 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:27:31.022 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:27:31.022 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "f05a2276-5930-4523-a432-0a2eddee43e0", 2023-10-18T12:27:31.022 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:27:31.022 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:27:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2af00a15f0 msgr2=0x7f2af00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2af00a15f0 0x7f2af00a3ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d761d88af0 tx=0x7f2af8009260 comp rx=0 tx=0).stop 2023-10-18T12:27:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 msgr2=0x7f2b0819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 0x7f2b0819efb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2afc000c00 tx=0x7f2afc004730 comp rx=0 tx=0).stop 2023-10-18T12:27:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 shutdown_connections 2023-10-18T12:27:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b08106730 0x7f2b08198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b08100720 0x7f2b0819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2af00a15f0 0x7f2af00a3ab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 --2- 172.21.15.70:0/1745054773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b081010e0 0x7f2b0819efb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 >> 172.21.15.70:0/1745054773 conn(0x7f2b080fc1b0 msgr2=0x7f2b080fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 shutdown_connections 2023-10-18T12:27:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.007+0000 7f2b0cfff700 1 -- 172.21.15.70:0/1745054773 wait complete. 2023-10-18T12:27:31.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:27:31.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 -- 172.21.15.70:0/263666278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8067d40 msgr2=0x7fdfa8068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 --2- 172.21.15.70:0/263666278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8067d40 0x7fdfa8068120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009990 tx=0x7fdf98009ca0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 -- 172.21.15.70:0/263666278 shutdown_connections 2023-10-18T12:27:31.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 --2- 172.21.15.70:0/263666278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa8105b30 0x7fdfa810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 --2- 172.21.15.70:0/263666278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8067d40 0x7fdfa8068120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 --2- 172.21.15.70:0/263666278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8105170 0x7fdfa81055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 -- 172.21.15.70:0/263666278 >> 172.21.15.70:0/263666278 conn(0x7fdfa80767f0 msgr2=0x7fdfa8076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 -- 172.21.15.70:0/263666278 shutdown_connections 2023-10-18T12:27:31.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.241+0000 7fdfadfff700 1 -- 172.21.15.70:0/263666278 wait complete. 2023-10-18T12:27:31.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.242+0000 7fdfadfff700 1 Processor -- start 2023-10-18T12:27:31.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.242+0000 7fdfadfff700 1 -- start start 2023-10-18T12:27:31.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.242+0000 7fdfadfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 0x7fdfa80ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.242+0000 7fdfadfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa8105170 0x7fdfa80ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfadfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8105b30 0x7fdfa8100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfadfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81a4f70 con 0x7fdfa8067d40 2023-10-18T12:27:31.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfadfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81a50e0 con 0x7fdfa8105b30 2023-10-18T12:27:31.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfadfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81a52a0 con 0x7fdfa8105170 2023-10-18T12:27:31.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 0x7fdfa80ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 0x7fdfa80ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50632/0 (socket says 172.21.15.70:50632) 2023-10-18T12:27:31.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 -- 172.21.15.70:0/3142358151 learned_addr learned my addr 172.21.15.70:0/3142358151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8105b30 0x7fdfa8100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa8105170 0x7fdfa80ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 -- 172.21.15.70:0/3142358151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa8105170 msgr2=0x7fdfa80ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa8105170 0x7fdfa80ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 -- 172.21.15.70:0/3142358151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8105b30 msgr2=0x7fdfa8100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8105b30 0x7fdfa8100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 -- 172.21.15.70:0/3142358151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf980095f0 con 0x7fdfa8067d40 2023-10-18T12:27:31.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8105b30 0x7fdfa8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:31.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.243+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 0x7fdfa80ff6b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005910 tx=0x7fdf98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.244+0000 7fdfa4ff9700 1 -- 172.21.15.70:0/3142358151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9801d070 con 0x7fdfa8067d40 2023-10-18T12:27:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.244+0000 7fdfa4ff9700 1 -- 172.21.15.70:0/3142358151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf98022470 con 0x7fdfa8067d40 2023-10-18T12:27:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.244+0000 7fdfa4ff9700 1 -- 172.21.15.70:0/3142358151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98017650 con 0x7fdfa8067d40 2023-10-18T12:27:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.244+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa81a5520 con 0x7fdfa8067d40 2023-10-18T12:27:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.244+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa81a5940 con 0x7fdfa8067d40 2023-10-18T12:27:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.248+0000 7fdfc1978b80 1 Processor -- start 2023-10-18T12:27:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 -- start start 2023-10-18T12:27:31.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfa4ff9700 1 -- 172.21.15.70:0/3142358151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdf980225e0 con 0x7fdfa8067d40 2023-10-18T12:27:31.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 0x55d1f962efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 0x55d1f9629090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9629a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f959c6e0 con 0x55d1f962ebe0 2023-10-18T12:27:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9582c60 con 0x55d1f96295d0 2023-10-18T12:27:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9580580 con 0x55d1f962f500 2023-10-18T12:27:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfa7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 0x55d1f962efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.249+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 0x55d1f9629090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 0x55d1f9629090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50420/0 (socket says 172.21.15.70:50420) 2023-10-18T12:27:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa77fe700 1 -- 172.21.15.70:0/320827686 learned_addr learned my addr 172.21.15.70:0/320827686 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 0x55d1f962efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50636/0 (socket says 172.21.15.70:50636) 2023-10-18T12:27:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9629a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/320827686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 msgr2=0x55d1f9629090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 0x55d1f9629090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/320827686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 msgr2=0x55d1f962efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 0x55d1f962efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/320827686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1f95393f0 con 0x55d1f96295d0 2023-10-18T12:27:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 0x55d1f9629090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 0x55d1f962efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.250+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9629a50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009c80 tx=0x7fdf9c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d2116b6da4bc43 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.251+0000 7fdf8dffb700 1 -- 172.21.15.70:0/320827686 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c011cb0 con 0x55d1f96295d0 2023-10-18T12:27:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.251+0000 7fdfa4ff9700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a1670 0x7fdf900a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.251+0000 7fdf8dffb700 1 -- 172.21.15.70:0/320827686 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf9c011e10 con 0x55d1f96295d0 2023-10-18T12:27:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.251+0000 7fdf8dffb700 1 -- 172.21.15.70:0/320827686 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c005280 con 0x55d1f96295d0 2023-10-18T12:27:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.251+0000 7fdfa4ff9700 1 -- 172.21.15.70:0/3142358151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdf980ce6f0 con 0x7fdfa8067d40 2023-10-18T12:27:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.251+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a1670 0x7fdf900a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 -- 172.21.15.70:0/320827686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 msgr2=0x55d1f9629a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9629a50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009c80 tx=0x7fdf9c009f90 comp rx=0 tx=0).stop 2023-10-18T12:27:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a1670 0x7fdf900a3b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009fd0 tx=0x7fdfa000b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 -- 172.21.15.70:0/320827686 shutdown_connections 2023-10-18T12:27:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9629a50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962ebe0 0x55d1f962efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/320827686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962f500 0x55d1f9629090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 -- 172.21.15.70:0/320827686 >> 172.21.15.70:0/320827686 conn(0x55d1f959d720 msgr2=0x55d1f9623b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 -- 172.21.15.70:0/320827686 shutdown_connections 2023-10-18T12:27:31.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 -- 172.21.15.70:0/320827686 wait complete. 2023-10-18T12:27:31.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.252+0000 7fdfc1978b80 1 -- start start 2023-10-18T12:27:31.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 Processor -- start 2023-10-18T12:27:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9635480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f9632810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9632f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f957be70 con 0x55d1f962f500 2023-10-18T12:27:31.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9518710 con 0x55d1f962ebe0 2023-10-18T12:27:31.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f95183b0 con 0x55d1f96295d0 2023-10-18T12:27:31.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfc1978b80 10 client.0 _ll_register_callbacks cb 0x55d1f9462a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfa6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f9632810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfa6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f9632810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39848/0 (socket says 172.21.15.70:39848) 2023-10-18T12:27:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/1679978572 learned_addr learned my addr 172.21.15.70:0/1679978572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.253+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9632f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9635480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/1679978572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 msgr2=0x55d1f9635480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9635480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/1679978572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 msgr2=0x55d1f9632f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9632f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/1679978572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa81a5940 con 0x55d1f962ebe0 2023-10-18T12:27:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9635480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f9632810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c011dc0 tx=0x7fdf9c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d4b19c66613471 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdf86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:27:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.254+0000 7fdf86ffd700 1 -- 172.21.15.70:0/1679978572 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c004260 con 0x55d1f962ebe0 2023-10-18T12:27:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1f9580ef0 con 0x55d1f962ebe0 2023-10-18T12:27:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdf86ffd700 1 -- 172.21.15.70:0/1679978572 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdf9c0043c0 con 0x55d1f962ebe0 2023-10-18T12:27:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdf86ffd700 1 -- 172.21.15.70:0/1679978572 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf9c004580 con 0x55d1f962ebe0 2023-10-18T12:27:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdfc1978b80 10 client.25814 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fdf9c0098e0 con 0x55d1f962ebe0 2023-10-18T12:27:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdf86ffd700 1 -- 172.21.15.70:0/1679978572 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1f9580ef0 con 0x55d1f962ebe0 2023-10-18T12:27:31.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.255+0000 7fdf86ffd700 1 -- 172.21.15.70:0/1679978572 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf9c005260 con 0x55d1f962ebe0 2023-10-18T12:27:31.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfc1978b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:31.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfc1978b80 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:31.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfc1978b80 20 client.25814 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:31.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9642dd0 0x55d1f96451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfc1978b80 4 client.25814 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:27:31.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1f95f88b0 con 0x55d1f9642dd0 2023-10-18T12:27:31.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9642dd0 0x55d1f96451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdf86ffd700 10 client.25814 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:31.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.256+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9642dd0 0x55d1f96451f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8bab58370f4a19 server_cookie=29b8b6b914e62a92 in_seq=0 out_seq=0 2023-10-18T12:27:31.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdf86ffd700 1 -- 172.21.15.70:0/1679978572 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1f95f88b0 con 0x55d1f9642dd0 2023-10-18T12:27:31.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdf86ffd700 10 client.25814 handle_command_reply: tid=0 2023-10-18T12:27:31.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 client.25814 shutdown 2023-10-18T12:27:31.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 msgr2=0x55d1f9632810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f9632810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c011dc0 tx=0x7fdf9c004750 comp rx=0 tx=0).stop 2023-10-18T12:27:31.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 shutdown_connections 2023-10-18T12:27:31.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9642dd0 0x55d1f96451f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9632f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9635480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/1679978572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f9632810 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdf86ffd700 0 client.25814 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 >> 172.21.15.70:0/1679978572 conn(0x55d1f959d720 msgr2=0x55d1f9628080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 shutdown_connections 2023-10-18T12:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.257+0000 7fdfc1978b80 1 -- 172.21.15.70:0/1679978572 wait complete. 2023-10-18T12:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.258+0000 7fdfc1978b80 20 client.25814 trim_cache size 0 max 16384 2023-10-18T12:27:31.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 Processor -- start 2023-10-18T12:27:31.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 -- start start 2023-10-18T12:27:31.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9639ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962ebe0 0x55d1f9623810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9634fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f93fc920 con 0x55d1f962f500 2023-10-18T12:27:31.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9633d70 con 0x55d1f96295d0 2023-10-18T12:27:31.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9624c20 con 0x55d1f962ebe0 2023-10-18T12:27:31.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9639ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9639ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39856/0 (socket says 172.21.15.70:39856) 2023-10-18T12:27:31.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfa77fe700 1 -- 172.21.15.70:0/904534082 learned_addr learned my addr 172.21.15.70:0/904534082 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.279+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9634fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962ebe0 0x55d1f9623810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa77fe700 1 -- 172.21.15.70:0/904534082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962ebe0 msgr2=0x55d1f9623810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962ebe0 0x55d1f9623810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa77fe700 1 -- 172.21.15.70:0/904534082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 msgr2=0x55d1f9634fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9634fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa77fe700 1 -- 172.21.15.70:0/904534082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1f93a6de0 con 0x55d1f96295d0 2023-10-18T12:27:31.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9639ee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf98097f90 tx=0x7fdf9801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57947661497f7d1e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdf8e7fc700 1 -- 172.21.15.70:0/904534082 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98097040 con 0x55d1f96295d0 2023-10-18T12:27:31.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdf8e7fc700 1 -- 172.21.15.70:0/904534082 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf98097860 con 0x55d1f96295d0 2023-10-18T12:27:31.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.280+0000 7fdf8e7fc700 1 -- 172.21.15.70:0/904534082 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9809bc20 con 0x55d1f96295d0 2023-10-18T12:27:31.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 -- 172.21.15.70:0/904534082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 msgr2=0x55d1f9639ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9639ee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf98097f90 tx=0x7fdf9801a040 comp rx=0 tx=0).stop 2023-10-18T12:27:31.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 -- 172.21.15.70:0/904534082 shutdown_connections 2023-10-18T12:27:31.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f9634fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f96295d0 0x55d1f9639ee0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/904534082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f962ebe0 0x55d1f9623810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 -- 172.21.15.70:0/904534082 >> 172.21.15.70:0/904534082 conn(0x55d1f959d720 msgr2=0x55d1f96289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 -- 172.21.15.70:0/904534082 shutdown_connections 2023-10-18T12:27:31.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.281+0000 7fdfc1978b80 1 -- 172.21.15.70:0/904534082 wait complete. 2023-10-18T12:27:31.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.282+0000 7fdfc1978b80 1 -- start start 2023-10-18T12:27:31.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.282+0000 7fdfc1978b80 1 Processor -- start 2023-10-18T12:27:31.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9639b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f963a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f96381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9638880 con 0x55d1f962f500 2023-10-18T12:27:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f96389f0 con 0x55d1f962ebe0 2023-10-18T12:27:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1f9638bb0 con 0x55d1f96295d0 2023-10-18T12:27:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfc1978b80 10 client.0 _ll_register_callbacks cb 0x55d1f9462a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:27:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9639b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9639b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50438/0 (socket says 172.21.15.70:50438) 2023-10-18T12:27:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa77fe700 1 -- 172.21.15.70:0/4197875187 learned_addr learned my addr 172.21.15.70:0/4197875187 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:27:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f963a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa7fff700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f96381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/4197875187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 msgr2=0x55d1f9639b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9639b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/4197875187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 msgr2=0x55d1f96381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:27:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f96381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa6ffd700 1 -- 172.21.15.70:0/4197875187 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa81a5520 con 0x55d1f962ebe0 2023-10-18T12:27:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.283+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9639b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:27:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfa6ffd700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f963a0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0056c0 tx=0x7fdf9c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c7489c7dc45ccf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:27:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdf86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:27:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdf86ffd700 1 -- 172.21.15.70:0/4197875187 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c004470 con 0x55d1f962ebe0 2023-10-18T12:27:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf9c011e10 con 0x55d1f962ebe0 2023-10-18T12:27:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdf86ffd700 1 -- 172.21.15.70:0/4197875187 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdf9c022700 con 0x55d1f962ebe0 2023-10-18T12:27:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdf86ffd700 1 -- 172.21.15.70:0/4197875187 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf9c0045d0 con 0x55d1f962ebe0 2023-10-18T12:27:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfc1978b80 10 client.25820 fetch_fsmap learned FSMap version 20 2023-10-18T12:27:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d1f938fc60 con 0x55d1f962ebe0 2023-10-18T12:27:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdf86ffd700 1 -- 172.21.15.70:0/4197875187 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf9c011e10 con 0x55d1f962ebe0 2023-10-18T12:27:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdf86ffd700 1 -- 172.21.15.70:0/4197875187 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf9c003ba0 con 0x55d1f962ebe0 2023-10-18T12:27:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfc1978b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:27:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfc1978b80 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:27:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.284+0000 7fdfc1978b80 20 client.25820 populate_metadata read hostname 'smithi070' 2023-10-18T12:27:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.285+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9697b80 0x55d1f9699f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:27:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.285+0000 7fdfc1978b80 4 client.25820 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:27:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.285+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1f9574e50 con 0x55d1f9697b80 2023-10-18T12:27:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.285+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9697b80 0x55d1f9699f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:27:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.285+0000 7fdfa77fe700 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9697b80 0x55d1f9699f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944f63447ec122f9 server_cookie=36d4432077b5c46f in_seq=0 out_seq=0 2023-10-18T12:27:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.285+0000 7fdf86ffd700 10 client.25820 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdf86ffd700 1 -- 172.21.15.70:0/4197875187 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d1f9574e50 con 0x55d1f9697b80 2023-10-18T12:27:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdf86ffd700 10 client.25820 handle_command_reply: tid=0 2023-10-18T12:27:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 client.25820 shutdown 2023-10-18T12:27:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 msgr2=0x55d1f963a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f963a0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0056c0 tx=0x7fdf9c011810 comp rx=0 tx=0).stop 2023-10-18T12:27:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 shutdown_connections 2023-10-18T12:27:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d1f9697b80 0x55d1f9699f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d1f962f500 0x55d1f96381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdf86ffd700 0 client.25820 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:27:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d1f96295d0 0x55d1f9639b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 --2- 172.21.15.70:0/4197875187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d1f962ebe0 0x55d1f963a0a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 >> 172.21.15.70:0/4197875187 conn(0x55d1f959d720 msgr2=0x55d1f9623810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 shutdown_connections 2023-10-18T12:27:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 1 -- 172.21.15.70:0/4197875187 wait complete. 2023-10-18T12:27:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.286+0000 7fdfc1978b80 20 client.25820 trim_cache size 0 max 16384 2023-10-18T12:27:31.304 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:27:31.304 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:27:31.304 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:27:31.304 INFO:teuthology.orchestra.run.smithi070.stdout: "f05a2276-5930-4523-a432-0a2eddee43e0": { 2023-10-18T12:27:31.304 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:27:31.305 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "f05a2276-5930-4523-a432-0a2eddee43e0", 2023-10-18T12:27:31.305 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:27:31.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:27:31.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:27:31.305 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:27:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a1670 msgr2=0x7fdf900a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a1670 0x7fdf900a3b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009fd0 tx=0x7fdfa000b040 comp rx=0 tx=0).stop 2023-10-18T12:27:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 msgr2=0x7fdfa80ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:27:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 0x7fdfa80ff6b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005910 tx=0x7fdf98004730 comp rx=0 tx=0).stop 2023-10-18T12:27:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 shutdown_connections 2023-10-18T12:27:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdf900a1670 0x7fdf900a3b30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfa8105b30 0x7fdfa8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfa8067d40 0x7fdfa80ff6b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 --2- 172.21.15.70:0/3142358151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa8105170 0x7fdfa80ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:27:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 >> 172.21.15.70:0/3142358151 conn(0x7fdfa80767f0 msgr2=0x7fdfa80fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:27:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 shutdown_connections 2023-10-18T12:27:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:27:31.287+0000 7fdfadfff700 1 -- 172.21.15.70:0/3142358151 wait complete. 2023-10-18T12:27:31.309 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-18T12:27:31.317 INFO:tasks.cephfs.filesystem:scrub status for tag:f05a2276-5930-4523-a432-0a2eddee43e0 - {'path': '/', 'tag': 'f05a2276-5930-4523-a432-0a2eddee43e0', 'options': 'recursive,force'} 2023-10-18T12:27:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: pgmap v359: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:27:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:27:31.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:31 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:31.388 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-18T12:27:31.453 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-18T12:27:31.457 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-18T12:27:31.457 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-18T12:27:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: pgmap v359: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:27:31.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:31.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:27:31.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:31 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:31.522 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-18T12:27:31.529 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:27:31.530 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-18T12:27:31.614 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-10-18T12:27:31.615 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-10-18T12:27:31.615 DEBUG:teuthology.orchestra.run.smithi070:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7420d7f8e55ceea786dcbf536c338ce920c21eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-10-18T12:27:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: pgmap v359: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-18T12:27:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:31.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:27:31.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:31.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:27:31.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:31 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:27:31.714 INFO:tasks.workunit.client.0.smithi070.stderr:+ dbench 1 2023-10-18T12:27:31.731 INFO:tasks.workunit.client.0.smithi070.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-10-18T12:27:31.732 INFO:tasks.workunit.client.0.smithi070.stdout: 2023-10-18T12:27:31.732 INFO:tasks.workunit.client.0.smithi070.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-10-18T12:27:31.733 INFO:tasks.workunit.client.0.smithi070.stdout:0 of 1 processes prepared for launch 0 sec 2023-10-18T12:27:31.834 INFO:tasks.workunit.client.0.smithi070.stdout:1 of 1 processes prepared for launch 0 sec 2023-10-18T12:27:31.834 INFO:tasks.workunit.client.0.smithi070.stdout:releasing clients 2023-10-18T12:27:32.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:32 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:27:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:32 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:27:32.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:32 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:32.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:32 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:27:32.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:32 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:27:32.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:32 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:32.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:32 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:27:32.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:32 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:27:32.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:32 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:27:32.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 3199 34.28 MB/sec warmup 1 sec latency 6.012 ms 2023-10-18T12:27:33.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:33 smithi186 ceph-mon[135253]: pgmap v360: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-18T12:27:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:33 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:27:33.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:33 smithi070 ceph-mon[98639]: pgmap v360: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-18T12:27:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:33 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:27:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:33 smithi152 ceph-mon[134767]: pgmap v360: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-18T12:27:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:33 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:27:33.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 5976 23.61 MB/sec warmup 2 sec latency 7.857 ms 2023-10-18T12:27:34.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 8339 20.44 MB/sec warmup 3 sec latency 19.381 ms 2023-10-18T12:27:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:35 smithi070 ceph-mon[98639]: pgmap v361: 129 pgs: 129 active+clean; 1.7 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 39 KiB/s wr, 2 op/s 2023-10-18T12:27:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:35 smithi152 ceph-mon[134767]: pgmap v361: 129 pgs: 129 active+clean; 1.7 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 39 KiB/s wr, 2 op/s 2023-10-18T12:27:35.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:35 smithi186 ceph-mon[135253]: pgmap v361: 129 pgs: 129 active+clean; 1.7 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 39 KiB/s wr, 2 op/s 2023-10-18T12:27:35.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 11370 19.64 MB/sec warmup 4 sec latency 7.745 ms 2023-10-18T12:27:36.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 14587 19.48 MB/sec warmup 5 sec latency 8.442 ms 2023-10-18T12:27:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:37 smithi070 ceph-mon[98639]: pgmap v362: 129 pgs: 129 active+clean; 2.0 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 72 KiB/s wr, 4 op/s 2023-10-18T12:27:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:37 smithi152 ceph-mon[134767]: pgmap v362: 129 pgs: 129 active+clean; 2.0 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 72 KiB/s wr, 4 op/s 2023-10-18T12:27:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:37 smithi186 ceph-mon[135253]: pgmap v362: 129 pgs: 129 active+clean; 2.0 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 72 KiB/s wr, 4 op/s 2023-10-18T12:27:37.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 17583 18.66 MB/sec warmup 6 sec latency 24.624 ms 2023-10-18T12:27:38.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 20359 17.80 MB/sec warmup 7 sec latency 10.006 ms 2023-10-18T12:27:39.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:39 smithi070 ceph-mon[98639]: pgmap v363: 129 pgs: 129 active+clean; 7.7 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 617 KiB/s wr, 79 op/s 2023-10-18T12:27:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:39 smithi152 ceph-mon[134767]: pgmap v363: 129 pgs: 129 active+clean; 7.7 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 617 KiB/s wr, 79 op/s 2023-10-18T12:27:39.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:39 smithi186 ceph-mon[135253]: pgmap v363: 129 pgs: 129 active+clean; 7.7 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 617 KiB/s wr, 79 op/s 2023-10-18T12:27:39.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 22776 17.60 MB/sec warmup 8 sec latency 23.187 ms 2023-10-18T12:27:40.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 25563 17.23 MB/sec warmup 9 sec latency 9.190 ms 2023-10-18T12:27:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:41 smithi070 ceph-mon[98639]: pgmap v364: 129 pgs: 129 active+clean; 8.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 772 KiB/s wr, 87 op/s 2023-10-18T12:27:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:41 smithi152 ceph-mon[134767]: pgmap v364: 129 pgs: 129 active+clean; 8.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 772 KiB/s wr, 87 op/s 2023-10-18T12:27:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:41 smithi186 ceph-mon[135253]: pgmap v364: 129 pgs: 129 active+clean; 8.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 772 KiB/s wr, 87 op/s 2023-10-18T12:27:41.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 28798 17.40 MB/sec warmup 10 sec latency 8.678 ms 2023-10-18T12:27:42.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 32035 17.42 MB/sec warmup 11 sec latency 23.344 ms 2023-10-18T12:27:43.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:43 smithi070 ceph-mon[98639]: pgmap v365: 129 pgs: 129 active+clean; 10 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 907 KiB/s wr, 99 op/s 2023-10-18T12:27:43.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:43 smithi152 ceph-mon[134767]: pgmap v365: 129 pgs: 129 active+clean; 10 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 907 KiB/s wr, 99 op/s 2023-10-18T12:27:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:43 smithi186 ceph-mon[135253]: pgmap v365: 129 pgs: 129 active+clean; 10 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 907 KiB/s wr, 99 op/s 2023-10-18T12:27:43.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 34858 17.08 MB/sec warmup 12 sec latency 6.784 ms 2023-10-18T12:27:44.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 37200 16.73 MB/sec warmup 13 sec latency 30.629 ms 2023-10-18T12:27:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:45 smithi070 ceph-mon[98639]: pgmap v366: 129 pgs: 129 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2023-10-18T12:27:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:45 smithi152 ceph-mon[134767]: pgmap v366: 129 pgs: 129 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2023-10-18T12:27:45.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:45 smithi186 ceph-mon[135253]: pgmap v366: 129 pgs: 129 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2023-10-18T12:27:45.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 40034 16.58 MB/sec warmup 14 sec latency 6.351 ms 2023-10-18T12:27:46.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:46 smithi070 ceph-mon[98639]: pgmap v367: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 209 op/s 2023-10-18T12:27:46.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:46 smithi152 ceph-mon[134767]: pgmap v367: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 209 op/s 2023-10-18T12:27:46.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:46 smithi186 ceph-mon[135253]: pgmap v367: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 209 op/s 2023-10-18T12:27:46.834 INFO:tasks.workunit.client.0.smithi070.stdout: 1 43197 16.70 MB/sec warmup 15 sec latency 8.563 ms 2023-10-18T12:27:47.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 46385 16.83 MB/sec warmup 16 sec latency 25.017 ms 2023-10-18T12:27:48.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 49112 16.59 MB/sec warmup 17 sec latency 7.699 ms 2023-10-18T12:27:48.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:48 smithi186 ceph-mon[135253]: pgmap v368: 129 pgs: 129 active+clean; 19 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-10-18T12:27:49.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:48 smithi070 ceph-mon[98639]: pgmap v368: 129 pgs: 129 active+clean; 19 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-10-18T12:27:49.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:48 smithi152 ceph-mon[134767]: pgmap v368: 129 pgs: 129 active+clean; 19 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-10-18T12:27:49.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 51782 16.35 MB/sec warmup 18 sec latency 8.401 ms 2023-10-18T12:27:50.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 54027 16.22 MB/sec warmup 19 sec latency 25.488 ms 2023-10-18T12:27:51.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:51 smithi186 ceph-mon[135253]: pgmap v369: 129 pgs: 129 active+clean; 22 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2023-10-18T12:27:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:51 smithi070 ceph-mon[98639]: pgmap v369: 129 pgs: 129 active+clean; 22 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2023-10-18T12:27:51.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:51 smithi152 ceph-mon[134767]: pgmap v369: 129 pgs: 129 active+clean; 22 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2023-10-18T12:27:51.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 57207 16.35 MB/sec warmup 20 sec latency 6.333 ms 2023-10-18T12:27:52.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 60306 16.40 MB/sec warmup 21 sec latency 52.193 ms 2023-10-18T12:27:53.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:53 smithi186 ceph-mon[135253]: pgmap v370: 129 pgs: 129 active+clean; 26 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-10-18T12:27:53.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:53 smithi070 ceph-mon[98639]: pgmap v370: 129 pgs: 129 active+clean; 26 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-10-18T12:27:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:53 smithi152 ceph-mon[134767]: pgmap v370: 129 pgs: 129 active+clean; 26 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-10-18T12:27:53.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 62815 16.08 MB/sec warmup 22 sec latency 20.826 ms 2023-10-18T12:27:54.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 65646 16.11 MB/sec warmup 23 sec latency 6.661 ms 2023-10-18T12:27:55.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:55 smithi186 ceph-mon[135253]: pgmap v371: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.9 MiB/s wr, 242 op/s 2023-10-18T12:27:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:55 smithi070 ceph-mon[98639]: pgmap v371: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.9 MiB/s wr, 242 op/s 2023-10-18T12:27:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:55 smithi152 ceph-mon[134767]: pgmap v371: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.9 MiB/s wr, 242 op/s 2023-10-18T12:27:55.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 68410 16.05 MB/sec warmup 24 sec latency 7.965 ms 2023-10-18T12:27:56.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 71400 16.14 MB/sec warmup 25 sec latency 34.534 ms 2023-10-18T12:27:57.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:57 smithi186 ceph-mon[135253]: pgmap v372: 129 pgs: 129 active+clean; 28 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2023-10-18T12:27:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:57 smithi070 ceph-mon[98639]: pgmap v372: 129 pgs: 129 active+clean; 28 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2023-10-18T12:27:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:57 smithi152 ceph-mon[134767]: pgmap v372: 129 pgs: 129 active+clean; 28 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2023-10-18T12:27:57.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 74614 16.20 MB/sec warmup 26 sec latency 7.946 ms 2023-10-18T12:27:58.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 77307 16.06 MB/sec warmup 27 sec latency 6.539 ms 2023-10-18T12:27:59.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:27:59 smithi186 ceph-mon[135253]: pgmap v373: 129 pgs: 129 active+clean; 33 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-10-18T12:27:59.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:27:59 smithi070 ceph-mon[98639]: pgmap v373: 129 pgs: 129 active+clean; 33 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-10-18T12:27:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:27:59 smithi152 ceph-mon[134767]: pgmap v373: 129 pgs: 129 active+clean; 33 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-10-18T12:27:59.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 79521 15.96 MB/sec warmup 28 sec latency 21.580 ms 2023-10-18T12:28:00.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 82328 15.90 MB/sec warmup 29 sec latency 6.831 ms 2023-10-18T12:28:01.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:28:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:01 smithi186 ceph-mon[135253]: pgmap v374: 129 pgs: 129 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 247 op/s 2023-10-18T12:28:01.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 -- 172.21.15.70:0/2767714870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 msgr2=0x7f828c106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 --2- 172.21.15.70:0/2767714870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 0x7f828c106b00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8280009990 tx=0x7f8280009ca0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 -- 172.21.15.70:0/2767714870 shutdown_connections 2023-10-18T12:28:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 --2- 172.21.15.70:0/2767714870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f828c1010d0 0x7f828c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 --2- 172.21.15.70:0/2767714870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 0x7f828c106b00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 --2- 172.21.15.70:0/2767714870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c100710 0x7f828c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 -- 172.21.15.70:0/2767714870 >> 172.21.15.70:0/2767714870 conn(0x7f828c0fc200 msgr2=0x7f828c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 -- 172.21.15.70:0/2767714870 shutdown_connections 2023-10-18T12:28:01.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.554+0000 7f8292fff700 1 -- 172.21.15.70:0/2767714870 wait complete. 2023-10-18T12:28:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.555+0000 7f8292fff700 1 Processor -- start 2023-10-18T12:28:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.555+0000 7f8292fff700 1 -- start start 2023-10-18T12:28:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8292fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f828c100710 0x7f828c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8292fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 0x7f828c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8292fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 0x7f828c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8292fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c197080 con 0x7f828c100710 2023-10-18T12:28:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8292fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1971f0 con 0x7f828c1010d0 2023-10-18T12:28:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8292fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1973b0 con 0x7f828c106720 2023-10-18T12:28:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f8290d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f828c100710 0x7f828c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 0x7f828c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f829159c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 0x7f828c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 0x7f828c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43988/0 (socket says 172.21.15.70:43988) 2023-10-18T12:28:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 -- 172.21.15.70:0/3915976240 learned_addr learned my addr 172.21.15.70:0/3915976240 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 -- 172.21.15.70:0/3915976240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 msgr2=0x7f828c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 0x7f828c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 -- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f828c100710 msgr2=0x7f828c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f828c100710 0x7f828c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.556+0000 7f828bfff700 1 -- 172.21.15.70:0/3915976240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8284009c40 con 0x7f828c1010d0 2023-10-18T12:28:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.557+0000 7f828bfff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 0x7f828c19ce30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f827800e9a0 tx=0x7f827800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.557+0000 7f8289ffb700 1 -- 172.21.15.70:0/3915976240 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278009790 con 0x7f828c1010d0 2023-10-18T12:28:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.557+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82800095f0 con 0x7f828c1010d0 2023-10-18T12:28:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.557+0000 7f8289ffb700 1 -- 172.21.15.70:0/3915976240 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827800ce80 con 0x7f828c1010d0 2023-10-18T12:28:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.557+0000 7f8289ffb700 1 -- 172.21.15.70:0/3915976240 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278018680 con 0x7f828c1010d0 2023-10-18T12:28:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.557+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c1a3700 con 0x7f828c1010d0 2023-10-18T12:28:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.561+0000 7f8289ffb700 1 -- 172.21.15.70:0/3915976240 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8278010430 con 0x7f828c1010d0 2023-10-18T12:28:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 Processor -- start 2023-10-18T12:28:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 -- start start 2023-10-18T12:28:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba305fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba300090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba300a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba2736e0 con 0x556bba306500 2023-10-18T12:28:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba259c60 con 0x556bba3005d0 2023-10-18T12:28:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba257580 con 0x556bba305be0 2023-10-18T12:28:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.562+0000 7f8289ffb700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f827c0a15f0 0x7f827c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f8290d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba300090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba300a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f829159c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba305fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f8290d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba300090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51892/0 (socket says 172.21.15.70:51892) 2023-10-18T12:28:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f8289ffb700 1 -- 172.21.15.70:0/3915976240 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8278014070 con 0x7f828c1010d0 2023-10-18T12:28:01.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f8290d9b700 1 -- 172.21.15.70:0/4012998281 learned_addr learned my addr 172.21.15.70:0/4012998281 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:01.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f829159c700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f827c0a15f0 0x7f827c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 -- 172.21.15.70:0/4012998281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 msgr2=0x556bba305fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba305fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 -- 172.21.15.70:0/4012998281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 msgr2=0x556bba300090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba300090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 -- 172.21.15.70:0/4012998281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bba2103f0 con 0x556bba3005d0 2023-10-18T12:28:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f8290d9b700 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba300090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.563+0000 7f828bfff700 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba300a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f82780caa30 tx=0x7f8278099f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b371b0570d3e0b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.564+0000 7f829159c700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f827c0a15f0 0x7f827c0a3ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8284009fd0 tx=0x7f828400b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.564+0000 7f8276ffd700 1 -- 172.21.15.70:0/4012998281 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827809a2c0 con 0x556bba3005d0 2023-10-18T12:28:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.564+0000 7f8276ffd700 1 -- 172.21.15.70:0/4012998281 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827809e410 con 0x556bba3005d0 2023-10-18T12:28:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.564+0000 7f8276ffd700 1 -- 172.21.15.70:0/4012998281 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780cd450 con 0x556bba3005d0 2023-10-18T12:28:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.564+0000 7f82a69fdb80 1 -- 172.21.15.70:0/4012998281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 msgr2=0x556bba300a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.564+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba300a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f82780caa30 tx=0x7f8278099f30 comp rx=0 tx=0).stop 2023-10-18T12:28:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 -- 172.21.15.70:0/4012998281 shutdown_connections 2023-10-18T12:28:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba300a50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba305fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/4012998281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba300090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 -- 172.21.15.70:0/4012998281 >> 172.21.15.70:0/4012998281 conn(0x556bba274720 msgr2=0x556bba2fab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 -- 172.21.15.70:0/4012998281 shutdown_connections 2023-10-18T12:28:01.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 -- 172.21.15.70:0/4012998281 wait complete. 2023-10-18T12:28:01.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.565+0000 7f82a69fdb80 1 -- start start 2023-10-18T12:28:01.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 Processor -- start 2023-10-18T12:28:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba30c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba309820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba309f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba252e70 con 0x556bba306500 2023-10-18T12:28:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba1ef710 con 0x556bba3005d0 2023-10-18T12:28:01.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba1ef3b0 con 0x556bba305be0 2023-10-18T12:28:01.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f82a69fdb80 10 client.0 _ll_register_callbacks cb 0x556bba139a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:01.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f828bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba309820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f828bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba309820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60350/0 (socket says 172.21.15.70:60350) 2023-10-18T12:28:01.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f828bfff700 1 -- 172.21.15.70:0/2277832051 learned_addr learned my addr 172.21.15.70:0/2277832051 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:01.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f8290d9b700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba30c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.566+0000 7f829159c700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba309f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f828bfff700 1 -- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 msgr2=0x556bba30c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f828bfff700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba30c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f828bfff700 1 -- 172.21.15.70:0/2277832051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 msgr2=0x556bba309f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f828bfff700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba309f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f828bfff700 1 -- 172.21.15.70:0/2277832051 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c10dee0 con 0x556bba305be0 2023-10-18T12:28:01.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f828bfff700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba309820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8278005830 tx=0x7f82780a2650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d198c96dc5644ed server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f826bfff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:28:01.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f826bfff700 1 -- 172.21.15.70:0/2277832051 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827809a520 con 0x556bba305be0 2023-10-18T12:28:01.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f826bfff700 1 -- 172.21.15.70:0/2277832051 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f827809a680 con 0x556bba305be0 2023-10-18T12:28:01.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bba257ef0 con 0x556bba305be0 2023-10-18T12:28:01.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f826bfff700 1 -- 172.21.15.70:0/2277832051 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82780997b0 con 0x556bba305be0 2023-10-18T12:28:01.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f82a69fdb80 10 client.25834 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:01.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.567+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f828c1a3700 con 0x556bba305be0 2023-10-18T12:28:01.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f826bfff700 1 -- 172.21.15.70:0/2277832051 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bba257ef0 con 0x556bba305be0 2023-10-18T12:28:01.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f826bfff700 1 -- 172.21.15.70:0/2277832051 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f82780a21c0 con 0x556bba305be0 2023-10-18T12:28:01.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f82a69fdb80 10 client.25834 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:01.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f82a69fdb80 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:01.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f82a69fdb80 20 client.25834 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:01.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba31df30 0x556bba320320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f82a69fdb80 4 client.25834 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:01.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bba2cf8b0 con 0x556bba31df30 2023-10-18T12:28:01.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.568+0000 7f8290d9b700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba31df30 0x556bba320320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.569+0000 7f826bfff700 10 client.25834 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:01.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.569+0000 7f8290d9b700 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba31df30 0x556bba320320 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57db750eb257e40e server_cookie=cd67c93a7bfc7d98 in_seq=0 out_seq=0 2023-10-18T12:28:01.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f826bfff700 1 -- 172.21.15.70:0/2277832051 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556bba2cf8b0 con 0x556bba31df30 2023-10-18T12:28:01.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f826bfff700 10 client.25834 handle_command_reply: tid=0 2023-10-18T12:28:01.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 client.25834 shutdown 2023-10-18T12:28:01.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 msgr2=0x556bba309820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba309820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8278005830 tx=0x7f82780a2650 comp rx=0 tx=0).stop 2023-10-18T12:28:01.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 shutdown_connections 2023-10-18T12:28:01.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba31df30 0x556bba320320 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba309f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba30c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/2277832051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba309820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f826bfff700 0 client.25834 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:01.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 >> 172.21.15.70:0/2277832051 conn(0x556bba274720 msgr2=0x556bba2ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 shutdown_connections 2023-10-18T12:28:01.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 1 -- 172.21.15.70:0/2277832051 wait complete. 2023-10-18T12:28:01.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.570+0000 7f82a69fdb80 20 client.25834 trim_cache size 0 max 16384 2023-10-18T12:28:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.589+0000 7f82a69fdb80 1 Processor -- start 2023-10-18T12:28:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.589+0000 7f82a69fdb80 1 -- start start 2023-10-18T12:28:01.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.589+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba3005d0 0x556bba315340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba2fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba306500 0x556bba30c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba0d3920 con 0x556bba3005d0 2023-10-18T12:28:01.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba2cf8e0 con 0x556bba306500 2023-10-18T12:28:01.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba30a560 con 0x556bba305be0 2023-10-18T12:28:01.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba2fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f829159c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba306500 0x556bba30c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba2fa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60358/0 (socket says 172.21.15.70:60358) 2023-10-18T12:28:01.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 -- 172.21.15.70:0/1004911815 learned_addr learned my addr 172.21.15.70:0/1004911815 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:01.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f8290d9b700 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba3005d0 0x556bba315340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 -- 172.21.15.70:0/1004911815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba306500 msgr2=0x556bba30c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba306500 0x556bba30c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 -- 172.21.15.70:0/1004911815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba3005d0 msgr2=0x556bba315340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba3005d0 0x556bba315340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 -- 172.21.15.70:0/1004911815 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bba07dde0 con 0x556bba305be0 2023-10-18T12:28:01.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.590+0000 7f828bfff700 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba2fa810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8278012020 tx=0x7f8278099f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce358ece0392158e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82777fe700 1 -- 172.21.15.70:0/1004911815 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780b85d0 con 0x556bba305be0 2023-10-18T12:28:01.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82777fe700 1 -- 172.21.15.70:0/1004911815 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82780b0070 con 0x556bba305be0 2023-10-18T12:28:01.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82777fe700 1 -- 172.21.15.70:0/1004911815 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780997b0 con 0x556bba305be0 2023-10-18T12:28:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 -- 172.21.15.70:0/1004911815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 msgr2=0x556bba2fa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba2fa810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8278012020 tx=0x7f8278099f30 comp rx=0 tx=0).stop 2023-10-18T12:28:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 -- 172.21.15.70:0/1004911815 shutdown_connections 2023-10-18T12:28:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba306500 0x556bba30c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba3005d0 0x556bba315340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/1004911815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba2fa810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.591+0000 7f82a69fdb80 1 -- 172.21.15.70:0/1004911815 >> 172.21.15.70:0/1004911815 conn(0x556bba274720 msgr2=0x556bba2ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.592+0000 7f82a69fdb80 1 -- 172.21.15.70:0/1004911815 shutdown_connections 2023-10-18T12:28:01.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.592+0000 7f82a69fdb80 1 -- 172.21.15.70:0/1004911815 wait complete. 2023-10-18T12:28:01.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.592+0000 7f82a69fdb80 1 -- start start 2023-10-18T12:28:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.592+0000 7f82a69fdb80 1 Processor -- start 2023-10-18T12:28:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba313510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba313a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba36ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba36b360 con 0x556bba306500 2023-10-18T12:28:01.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba36b4d0 con 0x556bba3005d0 2023-10-18T12:28:01.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bba36b690 con 0x556bba305be0 2023-10-18T12:28:01.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f82a69fdb80 10 client.0 _ll_register_callbacks cb 0x556bba139a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:01.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f828bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba313a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f8290d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba313510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f828bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba313a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60366/0 (socket says 172.21.15.70:60366) 2023-10-18T12:28:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f828bfff700 1 -- 172.21.15.70:0/417937690 learned_addr learned my addr 172.21.15.70:0/417937690 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.593+0000 7f829159c700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba36ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f828bfff700 1 -- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 msgr2=0x556bba313510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f828bfff700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba313510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f828bfff700 1 -- 172.21.15.70:0/417937690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 msgr2=0x556bba36ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:01.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f828bfff700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba36ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f828bfff700 1 -- 172.21.15.70:0/417937690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c1a3700 con 0x556bba305be0 2023-10-18T12:28:01.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f8290d9b700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba313510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:01.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f828bfff700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba313a50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f82780b8310 tx=0x7f82780caae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f90050c3237f1ad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f826bfff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:28:01.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f826bfff700 1 -- 172.21.15.70:0/417937690 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780b85d0 con 0x556bba305be0 2023-10-18T12:28:01.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827809e410 con 0x556bba305be0 2023-10-18T12:28:01.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f826bfff700 1 -- 172.21.15.70:0/417937690 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f82780997b0 con 0x556bba305be0 2023-10-18T12:28:01.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f826bfff700 1 -- 172.21.15.70:0/417937690 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82780b83e0 con 0x556bba305be0 2023-10-18T12:28:01.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f82a69fdb80 10 client.25840 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:01.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.594+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556bba066c60 con 0x556bba305be0 2023-10-18T12:28:01.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f826bfff700 1 -- 172.21.15.70:0/417937690 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827809e410 con 0x556bba305be0 2023-10-18T12:28:01.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f826bfff700 1 -- 172.21.15.70:0/417937690 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f82780c8e90 con 0x556bba305be0 2023-10-18T12:28:01.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f82a69fdb80 10 client.25840 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:01.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f82a69fdb80 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:01.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f82a69fdb80 20 client.25840 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:01.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba372a60 0x556bba374e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f82a69fdb80 4 client.25840 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:28:01.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556bba24be50 con 0x556bba372a60 2023-10-18T12:28:01.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f8290d9b700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba372a60 0x556bba374e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f826bfff700 10 client.25840 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.595+0000 7f8290d9b700 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba372a60 0x556bba374e50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19bf202abeb5a97e server_cookie=dfc9577eb7fcbdc0 in_seq=0 out_seq=0 2023-10-18T12:28:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f826bfff700 1 -- 172.21.15.70:0/417937690 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556bba24be50 con 0x556bba372a60 2023-10-18T12:28:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f826bfff700 10 client.25840 handle_command_reply: tid=0 2023-10-18T12:28:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 client.25840 shutdown 2023-10-18T12:28:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 msgr2=0x556bba313a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba313a50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f82780b8310 tx=0x7f82780caae0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 shutdown_connections 2023-10-18T12:28:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556bba372a60 0x556bba374e50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556bba306500 0x556bba36ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bba3005d0 0x556bba313510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f826bfff700 0 client.25840 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 --2- 172.21.15.70:0/417937690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bba305be0 0x556bba313a50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 >> 172.21.15.70:0/417937690 conn(0x556bba274720 msgr2=0x556bba2f44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 shutdown_connections 2023-10-18T12:28:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 1 -- 172.21.15.70:0/417937690 wait complete. 2023-10-18T12:28:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.596+0000 7f82a69fdb80 20 client.25840 trim_cache size 0 max 16384 2023-10-18T12:28:01.613 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:28:01.613 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:28:01.614 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:28:01.614 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:28:01.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f827c0a15f0 msgr2=0x7f827c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f827c0a15f0 0x7f827c0a3ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8284009fd0 tx=0x7f828400b040 comp rx=0 tx=0).stop 2023-10-18T12:28:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 msgr2=0x7f828c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 0x7f828c19ce30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f827800e9a0 tx=0x7f827800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 shutdown_connections 2023-10-18T12:28:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f828c106720 0x7f828c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f828c100710 0x7f828c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f827c0a15f0 0x7f827c0a3ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 --2- 172.21.15.70:0/3915976240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c1010d0 0x7f828c19ce30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 >> 172.21.15.70:0/3915976240 conn(0x7f828c0fc200 msgr2=0x7f828c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 shutdown_connections 2023-10-18T12:28:01.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.597+0000 7f8292fff700 1 -- 172.21.15.70:0/3915976240 wait complete. 2023-10-18T12:28:01.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:01 smithi070 ceph-mon[98639]: pgmap v374: 129 pgs: 129 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 247 op/s 2023-10-18T12:28:01.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:28:01.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:28:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:01 smithi152 ceph-mon[134767]: pgmap v374: 129 pgs: 129 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 247 op/s 2023-10-18T12:28:01.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3167070695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d41066b0 msgr2=0x7f10d4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3167070695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d41066b0 0x7f10d4106a90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098f0 tx=0x7f10d0009c00 comp rx=0 tx=0).stop 2023-10-18T12:28:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3167070695 shutdown_connections 2023-10-18T12:28:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3167070695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d4101060 0x7f10d410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3167070695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d41066b0 0x7f10d4106a90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3167070695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10d41006a0 0x7f10d4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.828+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3167070695 >> 172.21.15.70:0/3167070695 conn(0x7f10d40fc170 msgr2=0x7f10d40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:01.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.829+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3167070695 shutdown_connections 2023-10-18T12:28:01.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.829+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3167070695 wait complete. 2023-10-18T12:28:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 Processor -- start 2023-10-18T12:28:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 -- start start 2023-10-18T12:28:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 0x7f10d4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d4101060 0x7f10d4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10d41066b0 0x7f10d4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41a4f00 con 0x7f10d4101060 2023-10-18T12:28:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41a5070 con 0x7f10d41066b0 2023-10-18T12:28:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dc7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41a5230 con 0x7f10d41006a0 2023-10-18T12:28:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10da59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 0x7f10d4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10da59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 0x7f10d4072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60390/0 (socket says 172.21.15.70:60390) 2023-10-18T12:28:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10da59b700 1 -- 172.21.15.70:0/3480395729 learned_addr learned my addr 172.21.15.70:0/3480395729 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10dad9c700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10d41066b0 0x7f10d4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.830+0000 7f10d9d9a700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d4101060 0x7f10d4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10da59b700 1 -- 172.21.15.70:0/3480395729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10d41066b0 msgr2=0x7f10d4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10da59b700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10d41066b0 0x7f10d4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10da59b700 1 -- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d4101060 msgr2=0x7f10d4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10da59b700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d4101060 0x7f10d4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10da59b700 1 -- 172.21.15.70:0/3480395729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40095f0 con 0x7f10d41006a0 2023-10-18T12:28:01.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10d9d9a700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d4101060 0x7f10d4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:01.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10da59b700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 0x7f10d4072af0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10d0005950 tx=0x7f10d000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d001c070 con 0x7f10d41006a0 2023-10-18T12:28:01.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d0009520 con 0x7f10d41006a0 2023-10-18T12:28:01.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10d000baa0 con 0x7f10d41006a0 2023-10-18T12:28:01.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00038e0 con 0x7f10d41006a0 2023-10-18T12:28:01.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.831+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d41a5800 con 0x7f10d41006a0 2023-10-18T12:28:01.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.832+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d4108ca0 con 0x7f10d41006a0 2023-10-18T12:28:01.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 83443 15.54 MB/sec warmup 30 sec latency 56.634 ms 2023-10-18T12:28:01.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.834+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f10d0020030 con 0x7f10d41006a0 2023-10-18T12:28:01.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.836+0000 7f10cb7fe700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f10c00a15f0 0x7f10c00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:01.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.836+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f10d00ce150 con 0x7f10d41006a0 2023-10-18T12:28:01.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.836+0000 7f10d9d9a700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f10c00a15f0 0x7f10c00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:01.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.837+0000 7f10d9d9a700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f10c00a15f0 0x7f10c00a3ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10d4074220 tx=0x7f10c4009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:01.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:01.838+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f10d00971a0 con 0x7f10d41006a0 2023-10-18T12:28:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.105+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10d4073fd0 con 0x7f10d41006a0 2023-10-18T12:28:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.106+0000 7f10cb7fe700 1 -- 172.21.15.70:0/3480395729 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f10d009a7c0 con 0x7f10d41006a0 2023-10-18T12:28:02.109 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:28:02.109 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:28:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f10c00a15f0 msgr2=0x7f10c00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f10c00a15f0 0x7f10c00a3ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10d4074220 tx=0x7f10c4009310 comp rx=0 tx=0).stop 2023-10-18T12:28:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 msgr2=0x7f10d4072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 0x7f10d4072af0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10d0005950 tx=0x7f10d000fa00 comp rx=0 tx=0).stop 2023-10-18T12:28:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 shutdown_connections 2023-10-18T12:28:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f10c00a15f0 0x7f10c00a3ab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10d41066b0 0x7f10d4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10d41006a0 0x7f10d4072af0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 --2- 172.21.15.70:0/3480395729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10d4101060 0x7f10d4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 >> 172.21.15.70:0/3480395729 conn(0x7f10d40fc170 msgr2=0x7f10d40fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 shutdown_connections 2023-10-18T12:28:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.109+0000 7f10dc7ff700 1 -- 172.21.15.70:0/3480395729 wait complete. 2023-10-18T12:28:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:28:02.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:28:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/578495171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100720 msgr2=0x7f5b08100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/578495171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100720 0x7f5b08100ba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009990 tx=0x7f5af8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/578495171 shutdown_connections 2023-10-18T12:28:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/578495171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b081010e0 0x7f5b0810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/578495171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08106730 0x7f5b08106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/578495171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100720 0x7f5b08100ba0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/578495171 >> 172.21.15.70:0/578495171 conn(0x7f5b080fc1b0 msgr2=0x7f5b080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/578495171 shutdown_connections 2023-10-18T12:28:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.344+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/578495171 wait complete. 2023-10-18T12:28:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.345+0000 7f5b0f5ff700 1 Processor -- start 2023-10-18T12:28:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.345+0000 7f5b0f5ff700 1 -- start start 2023-10-18T12:28:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0f5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 0x7f5b08198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0f5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b081010e0 0x7f5b081992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0f5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b08106730 0x7f5b0819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0f5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819dcd0 con 0x7f5b08106730 2023-10-18T12:28:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0f5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819de40 con 0x7f5b081010e0 2023-10-18T12:28:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0f5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819e000 con 0x7f5b08100720 2023-10-18T12:28:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 0x7f5b08198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0db9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b08106730 0x7f5b0819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 0x7f5b08198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60404/0 (socket says 172.21.15.70:60404) 2023-10-18T12:28:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1059249173 learned_addr learned my addr 172.21.15.70:0/1059249173 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.346+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b081010e0 0x7f5b081992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1059249173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b081010e0 msgr2=0x7f5b081992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b081010e0 0x7f5b081992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b08106730 msgr2=0x7f5b0819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b08106730 0x7f5b0819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1059249173 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80095f0 con 0x7f5b08100720 2023-10-18T12:28:02.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 0x7f5b08198db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e9a0 tx=0x7f5b0400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:02.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5afe7fc700 1 -- 172.21.15.70:0/1059249173 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04009790 con 0x7f5b08100720 2023-10-18T12:28:02.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5afe7fc700 1 -- 172.21.15.70:0/1059249173 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b04010430 con 0x7f5b08100720 2023-10-18T12:28:02.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5afe7fc700 1 -- 172.21.15.70:0/1059249173 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04018680 con 0x7f5b08100720 2023-10-18T12:28:02.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0819e2e0 con 0x7f5b08100720 2023-10-18T12:28:02.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.347+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b081a5b20 con 0x7f5b08100720 2023-10-18T12:28:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.351+0000 7f5afe7fc700 1 -- 172.21.15.70:0/1059249173 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5b040105a0 con 0x7f5b08100720 2023-10-18T12:28:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.352+0000 7f5b22e86b80 1 Processor -- start 2023-10-18T12:28:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.352+0000 7f5b22e86b80 1 -- start start 2023-10-18T12:28:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.352+0000 7f5afe7fc700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5af40a15f0 0x7f5af40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.352+0000 7f5afe7fc700 1 -- 172.21.15.70:0/1059249173 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5b04014070 con 0x7f5b08100720 2023-10-18T12:28:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.352+0000 7f5b0db9c700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5af40a15f0 0x7f5af40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af2819e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 0x555af281dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281e500 0x555af281e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af27846e0 con 0x555af2819a90 2023-10-18T12:28:02.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af276ac60 con 0x555af281a3b0 2023-10-18T12:28:02.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af2768580 con 0x555af281e500 2023-10-18T12:28:02.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af2819e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0cb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 0x555af281dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af2819e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51954/0 (socket says 172.21.15.70:51954) 2023-10-18T12:28:02.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0cb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 0x555af281dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44084/0 (socket says 172.21.15.70:44084) 2023-10-18T12:28:02.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 -- 172.21.15.70:0/727523415 learned_addr learned my addr 172.21.15.70:0/727523415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:02.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0db9c700 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281e500 0x555af281e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 -- 172.21.15.70:0/727523415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281e500 msgr2=0x555af281e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281e500 0x555af281e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 -- 172.21.15.70:0/727523415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 msgr2=0x555af281dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 0x555af281dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 -- 172.21.15.70:0/727523415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555af27213f0 con 0x555af2819a90 2023-10-18T12:28:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 0x555af281dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0db9c700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5af40a15f0 0x7f5af40a3ab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ec0 tx=0x7f5b0000fc40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.353+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af2819e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5b04096cb0 tx=0x7f5b04099c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b35243d4ef9e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5af37fe700 1 -- 172.21.15.70:0/727523415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0409e390 con 0x555af2819a90 2023-10-18T12:28:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5af37fe700 1 -- 172.21.15.70:0/727523415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b040a25d0 con 0x555af2819a90 2023-10-18T12:28:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5af37fe700 1 -- 172.21.15.70:0/727523415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0409a6a0 con 0x555af2819a90 2023-10-18T12:28:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5b22e86b80 1 -- 172.21.15.70:0/727523415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 msgr2=0x555af2819e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af2819e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5b04096cb0 tx=0x7f5b04099c90 comp rx=0 tx=0).stop 2023-10-18T12:28:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5b22e86b80 1 -- 172.21.15.70:0/727523415 shutdown_connections 2023-10-18T12:28:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281e500 0x555af281e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af2819e70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.354+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/727523415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281a3b0 0x555af281dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.355+0000 7f5b22e86b80 1 -- 172.21.15.70:0/727523415 >> 172.21.15.70:0/727523415 conn(0x555af2785720 msgr2=0x555af280bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.355+0000 7f5b22e86b80 1 -- 172.21.15.70:0/727523415 shutdown_connections 2023-10-18T12:28:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.355+0000 7f5b22e86b80 1 -- 172.21.15.70:0/727523415 wait complete. 2023-10-18T12:28:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.355+0000 7f5b22e86b80 1 -- start start 2023-10-18T12:28:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.355+0000 7f5b22e86b80 1 Processor -- start 2023-10-18T12:28:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af2819a90 0x555af2823020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af2823560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281e500 0x555af2823aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af2763e70 con 0x555af281e500 2023-10-18T12:28:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af2700710 con 0x555af2819a90 2023-10-18T12:28:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af27003b0 con 0x555af281a3b0 2023-10-18T12:28:02.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b22e86b80 10 client.0 _ll_register_callbacks cb 0x555af264aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:02.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af2823560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af2819a90 0x555af2823020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af2823560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60424/0 (socket says 172.21.15.70:60424) 2023-10-18T12:28:02.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 -- 172.21.15.70:0/180060767 learned_addr learned my addr 172.21.15.70:0/180060767 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:02.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0db9c700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281e500 0x555af2823aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 -- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af2819a90 msgr2=0x555af2823020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af2819a90 0x555af2823020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 -- 172.21.15.70:0/180060767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281e500 msgr2=0x555af2823aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281e500 0x555af2823aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0cb9a700 1 -- 172.21.15.70:0/180060767 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0810def0 con 0x555af281a3b0 2023-10-18T12:28:02.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af2819a90 0x555af2823020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:02.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.356+0000 7f5b0db9c700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281e500 0x555af2823aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:02.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af2823560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5af8015000 tx=0x7f5af8005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ce27728dcab961 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:02.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5af0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:28:02.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5af0ff9700 1 -- 172.21.15.70:0/180060767 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801d070 con 0x555af281a3b0 2023-10-18T12:28:02.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555af2768ef0 con 0x555af281a3b0 2023-10-18T12:28:02.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5af0ff9700 1 -- 172.21.15.70:0/180060767 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5af8004500 con 0x555af281a3b0 2023-10-18T12:28:02.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5af0ff9700 1 -- 172.21.15.70:0/180060767 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af80046c0 con 0x555af281a3b0 2023-10-18T12:28:02.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5b22e86b80 10 client.25858 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:02.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f5b081a5b20 con 0x555af281a3b0 2023-10-18T12:28:02.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.357+0000 7f5af0ff9700 1 -- 172.21.15.70:0/180060767 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555af2768ef0 con 0x555af281a3b0 2023-10-18T12:28:02.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5af0ff9700 1 -- 172.21.15.70:0/180060767 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5af800f6c0 con 0x555af281a3b0 2023-10-18T12:28:02.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b22e86b80 10 client.25858 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:02.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b22e86b80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:02.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b22e86b80 20 client.25858 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:02.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af28337f0 0x555af2835c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b22e86b80 4 client.25858 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555af27e08b0 con 0x555af28337f0 2023-10-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af28337f0 0x555af2835c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af28337f0 0x555af2835c10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9b4f62f2886281 server_cookie=70ec0091e8849c47 in_seq=0 out_seq=0 2023-10-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.358+0000 7f5af0ff9700 10 client.25858 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:02.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5af0ff9700 1 -- 172.21.15.70:0/180060767 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555af27e08b0 con 0x555af28337f0 2023-10-18T12:28:02.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5af0ff9700 10 client.25858 handle_command_reply: tid=0 2023-10-18T12:28:02.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 client.25858 shutdown 2023-10-18T12:28:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 msgr2=0x555af2823560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af2823560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5af8015000 tx=0x7f5af8005800 comp rx=0 tx=0).stop 2023-10-18T12:28:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 shutdown_connections 2023-10-18T12:28:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af28337f0 0x555af2835c10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281e500 0x555af2823aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af2819a90 0x555af2823020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5af0ff9700 0 client.25858 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/180060767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af2823560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 >> 172.21.15.70:0/180060767 conn(0x555af2785720 msgr2=0x555af2810080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 shutdown_connections 2023-10-18T12:28:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 1 -- 172.21.15.70:0/180060767 wait complete. 2023-10-18T12:28:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.360+0000 7f5b22e86b80 20 client.25858 trim_cache size 0 max 16384 2023-10-18T12:28:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.373+0000 7f5b22e86b80 1 Processor -- start 2023-10-18T12:28:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 -- start start 2023-10-18T12:28:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 0x555af281bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281a3b0 0x555af28065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af28225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af25e4920 con 0x555af281a3b0 2023-10-18T12:28:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af280d600 con 0x555af281e500 2023-10-18T12:28:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af2826060 con 0x555af2819a90 2023-10-18T12:28:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0cb9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281a3b0 0x555af28065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 0x555af281bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0db9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af28225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 0x555af281bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60428/0 (socket says 172.21.15.70:60428) 2023-10-18T12:28:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0db9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af28225a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44098/0 (socket says 172.21.15.70:44098) 2023-10-18T12:28:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1233691890 learned_addr learned my addr 172.21.15.70:0/1233691890 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1233691890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 msgr2=0x555af28225a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af28225a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1233691890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281a3b0 msgr2=0x555af28065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281a3b0 0x555af28065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.374+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1233691890 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555af258ede0 con 0x555af2819a90 2023-10-18T12:28:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.375+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 0x555af281bec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b04099f40 tx=0x7f5b04010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a6c71ae21b1a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.375+0000 7f5af3fff700 1 -- 172.21.15.70:0/1233691890 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04010880 con 0x555af2819a90 2023-10-18T12:28:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.375+0000 7f5af3fff700 1 -- 172.21.15.70:0/1233691890 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b040985e0 con 0x555af2819a90 2023-10-18T12:28:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.375+0000 7f5af3fff700 1 -- 172.21.15.70:0/1233691890 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0409a700 con 0x555af2819a90 2023-10-18T12:28:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1233691890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 msgr2=0x555af281bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 0x555af281bec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b04099f40 tx=0x7f5b04010b80 comp rx=0 tx=0).stop 2023-10-18T12:28:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1233691890 shutdown_connections 2023-10-18T12:28:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af28225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af2819a90 0x555af281bec0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1233691890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af281a3b0 0x555af28065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1233691890 >> 172.21.15.70:0/1233691890 conn(0x555af2785720 msgr2=0x555af283a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1233691890 shutdown_connections 2023-10-18T12:28:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1233691890 wait complete. 2023-10-18T12:28:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 -- start start 2023-10-18T12:28:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.376+0000 7f5b22e86b80 1 Processor -- start 2023-10-18T12:28:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af282a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af281b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af281be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af2822770 con 0x555af2819a90 2023-10-18T12:28:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af28228e0 con 0x555af281e500 2023-10-18T12:28:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af2822aa0 con 0x555af281a3b0 2023-10-18T12:28:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b22e86b80 10 client.0 _ll_register_callbacks cb 0x555af264aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af282a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0db9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af281be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af282a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51984/0 (socket says 172.21.15.70:51984) 2023-10-18T12:28:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1072061554 learned_addr learned my addr 172.21.15.70:0/1072061554 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af281b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1072061554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 msgr2=0x555af281b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af281b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 msgr2=0x555af281be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af281be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.377+0000 7f5b0d39b700 1 -- 172.21.15.70:0/1072061554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555af27213f0 con 0x555af2819a90 2023-10-18T12:28:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b0d39b700 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af282a8b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5b04010c10 tx=0x7f5b04099cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c604729183f3dab server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5af0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:28:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5af0ff9700 1 -- 172.21.15.70:0/1072061554 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0409a6a0 con 0x555af2819a90 2023-10-18T12:28:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5af0ff9700 1 -- 172.21.15.70:0/1072061554 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5b04099730 con 0x555af2819a90 2023-10-18T12:28:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b040a25d0 con 0x555af2819a90 2023-10-18T12:28:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5af0ff9700 1 -- 172.21.15.70:0/1072061554 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b040998f0 con 0x555af2819a90 2023-10-18T12:28:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b22e86b80 10 client.15849 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555af2577c60 con 0x555af2819a90 2023-10-18T12:28:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5af0ff9700 1 -- 172.21.15.70:0/1072061554 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b040a25d0 con 0x555af2819a90 2023-10-18T12:28:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5af0ff9700 1 -- 172.21.15.70:0/1072061554 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5b040991a0 con 0x555af2819a90 2023-10-18T12:28:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b22e86b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b22e86b80 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.378+0000 7f5b22e86b80 20 client.15849 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.379+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af2887d60 0x555af288a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.379+0000 7f5b22e86b80 4 client.15849 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:28:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.379+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555af275ce50 con 0x555af2887d60 2023-10-18T12:28:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.379+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af2887d60 0x555af288a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.379+0000 7f5af0ff9700 10 client.15849 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.379+0000 7f5b0cb9a700 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af2887d60 0x555af288a150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162cc60a5248f6ef server_cookie=77eace3251653559 in_seq=0 out_seq=0 2023-10-18T12:28:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.380+0000 7f5af0ff9700 1 -- 172.21.15.70:0/1072061554 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555af275ce50 con 0x555af2887d60 2023-10-18T12:28:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.380+0000 7f5af0ff9700 10 client.15849 handle_command_reply: tid=0 2023-10-18T12:28:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.380+0000 7f5b22e86b80 1 client.15849 shutdown 2023-10-18T12:28:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 msgr2=0x555af282a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af282a8b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5b04010c10 tx=0x7f5b04099cb0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 shutdown_connections 2023-10-18T12:28:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af2887d60 0x555af288a150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af281e500 0x555af281be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af2819a90 0x555af282a8b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 --2- 172.21.15.70:0/1072061554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af281a3b0 0x555af281b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5af0ff9700 0 client.15849 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 >> 172.21.15.70:0/1072061554 conn(0x555af2785720 msgr2=0x555af280b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 shutdown_connections 2023-10-18T12:28:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 1 -- 172.21.15.70:0/1072061554 wait complete. 2023-10-18T12:28:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.381+0000 7f5b22e86b80 20 client.15849 trim_cache size 0 max 16384 2023-10-18T12:28:02.406 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:28:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5af40a15f0 msgr2=0x7f5af40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5af40a15f0 0x7f5af40a3ab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ec0 tx=0x7f5b0000fc40 comp rx=0 tx=0).stop 2023-10-18T12:28:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 msgr2=0x7f5b08198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 0x7f5b08198db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e9a0 tx=0x7f5b0400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 shutdown_connections 2023-10-18T12:28:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b08106730 0x7f5b0819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b08100720 0x7f5b08198db0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5af40a15f0 0x7f5af40a3ab0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 --2- 172.21.15.70:0/1059249173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b081010e0 0x7f5b081992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 >> 172.21.15.70:0/1059249173 conn(0x7f5b080fc1b0 msgr2=0x7f5b08072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:02.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 shutdown_connections 2023-10-18T12:28:02.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:02.382+0000 7f5b0f5ff700 1 -- 172.21.15.70:0/1059249173 wait complete. 2023-10-18T12:28:02.409 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:02 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3480395729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:28:02.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:02 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3480395729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:28:02.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:02 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3480395729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:28:02.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 85862 15.49 MB/sec warmup 31 sec latency 52.326 ms 2023-10-18T12:28:03.411 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:28:03.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:28:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:03 smithi186 ceph-mon[135253]: pgmap v375: 129 pgs: 129 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2023-10-18T12:28:03.472 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:03 smithi070 ceph-mon[98639]: pgmap v375: 129 pgs: 129 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2023-10-18T12:28:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 -- 172.21.15.70:0/3299464104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 msgr2=0x7f162410f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 --2- 172.21.15.70:0/3299464104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 0x7f162410f3e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f161c009f90 tx=0x7f161c00b330 comp rx=0 tx=0).stop 2023-10-18T12:28:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 -- 172.21.15.70:0/3299464104 shutdown_connections 2023-10-18T12:28:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 --2- 172.21.15.70:0/3299464104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 0x7f162410f3e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 --2- 172.21.15.70:0/3299464104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1624072bc0 0x7f1624072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 --2- 172.21.15.70:0/3299464104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16240734e0 0x7f162410ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 -- 172.21.15.70:0/3299464104 >> 172.21.15.70:0/3299464104 conn(0x7f16240fc200 msgr2=0x7f16240fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 -- 172.21.15.70:0/3299464104 shutdown_connections 2023-10-18T12:28:03.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.622+0000 7f162b7ff700 1 -- 172.21.15.70:0/3299464104 wait complete. 2023-10-18T12:28:03.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.623+0000 7f162b7ff700 1 Processor -- start 2023-10-18T12:28:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 -- start start 2023-10-18T12:28:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 0x7f1624198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16240734e0 0x7f1624199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 0x7f162419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419de20 con 0x7f162410cfb0 2023-10-18T12:28:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419df90 con 0x7f1624072bc0 2023-10-18T12:28:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.624+0000 7f162b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419e150 con 0x7f16240734e0 2023-10-18T12:28:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 0x7f1624198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 0x7f1624198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44108/0 (socket says 172.21.15.70:44108) 2023-10-18T12:28:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 -- 172.21.15.70:0/2232925125 learned_addr learned my addr 172.21.15.70:0/2232925125 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f1629d9c700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 0x7f162419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f1628d9a700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16240734e0 0x7f1624199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 -- 172.21.15.70:0/2232925125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16240734e0 msgr2=0x7f1624199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16240734e0 0x7f1624199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 -- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 msgr2=0x7f162419d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 0x7f162419d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 -- 172.21.15.70:0/2232925125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c009c40 con 0x7f1624072bc0 2023-10-18T12:28:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f162959b700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 0x7f1624198f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f162000e9a0 tx=0x7f162000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.625+0000 7f161a7fc700 1 -- 172.21.15.70:0/2232925125 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620009790 con 0x7f1624072bc0 2023-10-18T12:28:03.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.626+0000 7f161a7fc700 1 -- 172.21.15.70:0/2232925125 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f162000ce80 con 0x7f1624072bc0 2023-10-18T12:28:03.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.626+0000 7f161a7fc700 1 -- 172.21.15.70:0/2232925125 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620018680 con 0x7f1624072bc0 2023-10-18T12:28:03.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.626+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162419e430 con 0x7f1624072bc0 2023-10-18T12:28:03.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.626+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16241a5c70 con 0x7f1624072bc0 2023-10-18T12:28:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.629+0000 7f161a7fc700 1 -- 172.21.15.70:0/2232925125 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1620010430 con 0x7f1624072bc0 2023-10-18T12:28:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 Processor -- start 2023-10-18T12:28:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 -- start start 2023-10-18T12:28:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630bf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f161a7fc700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16100a15f0 0x7f16100a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863032760 con 0x5648630c4c60 2023-10-18T12:28:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863018ce0 con 0x5648630bf650 2023-10-18T12:28:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.631+0000 7f163f125b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863016600 con 0x5648630c5580 2023-10-18T12:28:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f161a7fc700 1 -- 172.21.15.70:0/2232925125 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1620014070 con 0x7f1624072bc0 2023-10-18T12:28:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f1629d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f1629d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52002/0 (socket says 172.21.15.70:52002) 2023-10-18T12:28:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f1629d9c700 1 -- 172.21.15.70:0/1945549279 learned_addr learned my addr 172.21.15.70:0/1945549279 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f162959b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630bf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f1628d9a700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16100a15f0 0x7f16100a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f1629d9c700 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f162959b700 1 -- 172.21.15.70:0/1945549279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 msgr2=0x5648630bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f162959b700 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f162959b700 1 -- 172.21.15.70:0/1945549279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 msgr2=0x5648630c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f162959b700 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f162959b700 1 -- 172.21.15.70:0/1945549279 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564862fcf470 con 0x5648630c5580 2023-10-18T12:28:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.632+0000 7f1629d9c700 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f1628d9a700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16100a15f0 0x7f16100a3ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f162419a450 tx=0x7f1614009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f162959b700 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630bf110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16200caa30 tx=0x7f1620099ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e486ea911b521e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f160f7fe700 1 -- 172.21.15.70:0/1945549279 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162009e410 con 0x5648630c5580 2023-10-18T12:28:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f160f7fe700 1 -- 172.21.15.70:0/1945549279 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16200a2650 con 0x5648630c5580 2023-10-18T12:28:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f160f7fe700 1 -- 172.21.15.70:0/1945549279 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162009a520 con 0x5648630c5580 2023-10-18T12:28:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f163f125b80 1 -- 172.21.15.70:0/1945549279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 msgr2=0x5648630bf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.633+0000 7f163f125b80 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630bf110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16200caa30 tx=0x7f1620099ca0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 -- 172.21.15.70:0/1945549279 shutdown_connections 2023-10-18T12:28:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 --2- 172.21.15.70:0/1945549279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630bf110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 -- 172.21.15.70:0/1945549279 >> 172.21.15.70:0/1945549279 conn(0x5648630337a0 msgr2=0x5648630b9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 -- 172.21.15.70:0/1945549279 shutdown_connections 2023-10-18T12:28:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 -- 172.21.15.70:0/1945549279 wait complete. 2023-10-18T12:28:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 -- start start 2023-10-18T12:28:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.634+0000 7f163f125b80 1 Processor -- start 2023-10-18T12:28:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 0x5648630cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630c4c60 0x5648630c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c5580 0x5648630c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863011ef0 con 0x5648630c5580 2023-10-18T12:28:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564862fae790 con 0x5648630c4c60 2023-10-18T12:28:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564862fae430 con 0x5648630bf650 2023-10-18T12:28:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f163f125b80 10 client.0 _ll_register_callbacks cb 0x564862ef8b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f162959b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 0x5648630cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f162959b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 0x5648630cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60464/0 (socket says 172.21.15.70:60464) 2023-10-18T12:28:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f162959b700 1 -- 172.21.15.70:0/3303460413 learned_addr learned my addr 172.21.15.70:0/3303460413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f1628d9a700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630c4c60 0x5648630c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.635+0000 7f1629d9c700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c5580 0x5648630c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f162959b700 1 -- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630c4c60 msgr2=0x5648630c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f162959b700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630c4c60 0x5648630c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f162959b700 1 -- 172.21.15.70:0/3303460413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c5580 msgr2=0x5648630c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f162959b700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c5580 0x5648630c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f162959b700 1 -- 172.21.15.70:0/3303460413 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16241a5c70 con 0x5648630bf650 2023-10-18T12:28:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f1628d9a700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630c4c60 0x5648630c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f162959b700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 0x5648630cb4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f16200caa90 tx=0x7f162009e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e5239241740c29 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f160cff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:28:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f160cff9700 1 -- 172.21.15.70:0/3303460413 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16200999b0 con 0x5648630bf650 2023-10-18T12:28:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f160cff9700 1 -- 172.21.15.70:0/3303460413 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f162006a2a0 con 0x5648630bf650 2023-10-18T12:28:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564863016f70 con 0x5648630bf650 2023-10-18T12:28:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f160cff9700 1 -- 172.21.15.70:0/3303460413 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f162009a520 con 0x5648630bf650 2023-10-18T12:28:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f163f125b80 10 client.25876 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564862e3ce60 con 0x5648630bf650 2023-10-18T12:28:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.636+0000 7f160cff9700 1 -- 172.21.15.70:0/3303460413 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564863016f70 con 0x5648630bf650 2023-10-18T12:28:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f160cff9700 1 -- 172.21.15.70:0/3303460413 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f16200960b0 con 0x5648630bf650 2023-10-18T12:28:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f163f125b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f163f125b80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f163f125b80 20 client.25876 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f163f125b80 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5648630d8b60 0x5648630daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f163f125b80 4 client.25876 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56486308e930 con 0x5648630d8b60 2023-10-18T12:28:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.637+0000 7f1628d9a700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5648630d8b60 0x5648630daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f1628d9a700 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5648630d8b60 0x5648630daf80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8194a62a863fa3e server_cookie=4b139534b9045248 in_seq=0 out_seq=0 2023-10-18T12:28:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f160cff9700 10 client.25876 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f160cff9700 1 -- 172.21.15.70:0/3303460413 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56486308e930 con 0x5648630d8b60 2023-10-18T12:28:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f160cff9700 10 client.25876 handle_command_reply: tid=0 2023-10-18T12:28:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 client.25876 shutdown 2023-10-18T12:28:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 msgr2=0x5648630cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 0x5648630cb4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f16200caa90 tx=0x7f162009e490 comp rx=0 tx=0).stop 2023-10-18T12:28:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 shutdown_connections 2023-10-18T12:28:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5648630d8b60 0x5648630daf80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c5580 0x5648630c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630bf650 0x5648630cb4d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 --2- 172.21.15.70:0/3303460413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630c4c60 0x5648630c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f160cff9700 0 client.25876 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.638+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 >> 172.21.15.70:0/3303460413 conn(0x5648630337a0 msgr2=0x5648630be100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.639+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 shutdown_connections 2023-10-18T12:28:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.639+0000 7f163f125b80 1 -- 172.21.15.70:0/3303460413 wait complete. 2023-10-18T12:28:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.639+0000 7f163f125b80 20 client.25876 trim_cache size 0 max 16384 2023-10-18T12:28:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.656+0000 7f163f125b80 1 Processor -- start 2023-10-18T12:28:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 -- start start 2023-10-18T12:28:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564862e929a0 con 0x5648630c4c60 2023-10-18T12:28:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648630c9dc0 con 0x5648630bf650 2023-10-18T12:28:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f163f125b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648630c93d0 con 0x5648630c5580 2023-10-18T12:28:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f162959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f1629d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f1628d9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f162959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630d4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44126/0 (socket says 172.21.15.70:44126) 2023-10-18T12:28:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f162959b700 1 -- 172.21.15.70:0/3079846484 learned_addr learned my addr 172.21.15.70:0/3079846484 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.657+0000 7f1629d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60474/0 (socket says 172.21.15.70:60474) 2023-10-18T12:28:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1629d9c700 1 -- 172.21.15.70:0/3079846484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 msgr2=0x5648630d4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1629d9c700 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630d4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1629d9c700 1 -- 172.21.15.70:0/3079846484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 msgr2=0x5648630cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1629d9c700 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1629d9c700 1 -- 172.21.15.70:0/3079846484 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648630b9df0 con 0x5648630c5580 2023-10-18T12:28:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1628d9a700 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f1629d9c700 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cb000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b6c0 tx=0x7f161c009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ecda1a221623ce server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f160ffff700 1 -- 172.21.15.70:0/3079846484 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00e040 con 0x5648630c5580 2023-10-18T12:28:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f160ffff700 1 -- 172.21.15.70:0/3079846484 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f161c003c00 con 0x5648630c5580 2023-10-18T12:28:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.658+0000 7f160ffff700 1 -- 172.21.15.70:0/3079846484 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c005340 con 0x5648630c5580 2023-10-18T12:28:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 -- 172.21.15.70:0/3079846484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 msgr2=0x5648630cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cb000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b6c0 tx=0x7f161c009700 comp rx=0 tx=0).stop 2023-10-18T12:28:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 -- 172.21.15.70:0/3079846484 shutdown_connections 2023-10-18T12:28:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cb000 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648630d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 --2- 172.21.15.70:0/3079846484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x5648630cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 -- 172.21.15.70:0/3079846484 >> 172.21.15.70:0/3079846484 conn(0x5648630337a0 msgr2=0x5648630c2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 -- 172.21.15.70:0/3079846484 shutdown_connections 2023-10-18T12:28:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 -- 172.21.15.70:0/3079846484 wait complete. 2023-10-18T12:28:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.659+0000 7f163f125b80 1 -- start start 2023-10-18T12:28:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 Processor -- start 2023-10-18T12:28:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648631271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x564863127730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648630cf8a0 con 0x5648630c4c60 2023-10-18T12:28:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648630cfa10 con 0x5648630bf650 2023-10-18T12:28:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648630cfbd0 con 0x5648630c5580 2023-10-18T12:28:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f163f125b80 10 client.0 _ll_register_callbacks cb 0x564862ef8b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f1628d9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x564863127730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f162959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648631271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f162959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648631271f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44128/0 (socket says 172.21.15.70:44128) 2023-10-18T12:28:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f162959b700 1 -- 172.21.15.70:0/1212948153 learned_addr learned my addr 172.21.15.70:0/1212948153 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.660+0000 7f1629d9c700 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f162959b700 1 -- 172.21.15.70:0/1212948153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 msgr2=0x5648630cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f162959b700 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f162959b700 1 -- 172.21.15.70:0/1212948153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 msgr2=0x564863127730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f162959b700 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x564863127730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f162959b700 1 -- 172.21.15.70:0/1212948153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1614017070 con 0x5648630bf650 2023-10-18T12:28:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f162959b700 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648631271f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16200a25c0 tx=0x7f162009aa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65768bbf7dc9142d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f160cff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f160cff9700 1 -- 172.21.15.70:0/1212948153 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16200a23c0 con 0x5648630bf650 2023-10-18T12:28:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16200a2650 con 0x5648630bf650 2023-10-18T12:28:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f160cff9700 1 -- 172.21.15.70:0/1212948153 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1620099580 con 0x5648630bf650 2023-10-18T12:28:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f160cff9700 1 -- 172.21.15.70:0/1212948153 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f162009a520 con 0x5648630bf650 2023-10-18T12:28:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f163f125b80 10 client.25880 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.661+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f16241100f0 con 0x5648630bf650 2023-10-18T12:28:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f160cff9700 1 -- 172.21.15.70:0/1212948153 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16200a2650 con 0x5648630bf650 2023-10-18T12:28:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f160cff9700 1 -- 172.21.15.70:0/1212948153 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f16200b82b0 con 0x5648630bf650 2023-10-18T12:28:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f163f125b80 10 client.25880 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f163f125b80 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f163f125b80 20 client.25880 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f163f125b80 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56486312ecf0 0x5648631310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f163f125b80 4 client.25880 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:28:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56486300aed0 con 0x56486312ecf0 2023-10-18T12:28:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.662+0000 7f1628d9a700 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56486312ecf0 0x5648631310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.663+0000 7f160cff9700 10 client.25880 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.663+0000 7f1628d9a700 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56486312ecf0 0x5648631310e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8018271f632d4e3 server_cookie=1e8a047bf43120d2 in_seq=0 out_seq=0 2023-10-18T12:28:03.682 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:03 smithi152 ceph-mon[134767]: pgmap v375: 129 pgs: 129 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2023-10-18T12:28:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f160cff9700 1 -- 172.21.15.70:0/1212948153 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56486300aed0 con 0x56486312ecf0 2023-10-18T12:28:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f160cff9700 10 client.25880 handle_command_reply: tid=0 2023-10-18T12:28:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 client.25880 shutdown 2023-10-18T12:28:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 msgr2=0x5648631271f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648631271f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16200a25c0 tx=0x7f162009aa40 comp rx=0 tx=0).stop 2023-10-18T12:28:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 shutdown_connections 2023-10-18T12:28:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56486312ecf0 0x5648631310e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648630c5580 0x5648630cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648630bf650 0x5648631271f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 --2- 172.21.15.70:0/1212948153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648630c4c60 0x564863127730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f160cff9700 0 client.25880 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 >> 172.21.15.70:0/1212948153 conn(0x5648630337a0 msgr2=0x5648630b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.664+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 shutdown_connections 2023-10-18T12:28:03.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.665+0000 7f163f125b80 1 -- 172.21.15.70:0/1212948153 wait complete. 2023-10-18T12:28:03.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.665+0000 7f163f125b80 20 client.25880 trim_cache size 0 max 16384 2023-10-18T12:28:03.686 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:28:03.687 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:28:03.687 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "97d96ed6-914b-4500-8373-b856c742121f", 2023-10-18T12:28:03.687 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:28:03.687 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:28:03.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.665+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16100a15f0 msgr2=0x7f16100a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.665+0000 7f162b7ff700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16100a15f0 0x7f16100a3ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f162419a450 tx=0x7f1614009310 comp rx=0 tx=0).stop 2023-10-18T12:28:03.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.665+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 msgr2=0x7f1624198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.665+0000 7f162b7ff700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 0x7f1624198f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f162000e9a0 tx=0x7f162000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 shutdown_connections 2023-10-18T12:28:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f162410cfb0 0x7f162419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f16100a15f0 0x7f16100a3ab0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1624072bc0 0x7f1624198f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 --2- 172.21.15.70:0/2232925125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16240734e0 0x7f1624199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 >> 172.21.15.70:0/2232925125 conn(0x7f16240fc200 msgr2=0x7f16240fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 shutdown_connections 2023-10-18T12:28:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.666+0000 7f162b7ff700 1 -- 172.21.15.70:0/2232925125 wait complete. 2023-10-18T12:28:03.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:28:03.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 88998 15.59 MB/sec warmup 32 sec latency 19.844 ms 2023-10-18T12:28:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 -- 172.21.15.70:0/28351659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 msgr2=0x7f4230106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 --2- 172.21.15.70:0/28351659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f4230106b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4220009990 tx=0x7f4220009ca0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 -- 172.21.15.70:0/28351659 shutdown_connections 2023-10-18T12:28:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 --2- 172.21.15.70:0/28351659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f423010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 --2- 172.21.15.70:0/28351659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f4230106b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 --2- 172.21.15.70:0/28351659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 0x7f4230100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 -- 172.21.15.70:0/28351659 >> 172.21.15.70:0/28351659 conn(0x7f42300fc200 msgr2=0x7f42300fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 -- 172.21.15.70:0/28351659 shutdown_connections 2023-10-18T12:28:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.900+0000 7f42361ff700 1 -- 172.21.15.70:0/28351659 wait complete. 2023-10-18T12:28:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.901+0000 7f42361ff700 1 Processor -- start 2023-10-18T12:28:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.901+0000 7f42361ff700 1 -- start start 2023-10-18T12:28:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f42361ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 0x7f4230198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f42361ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f42301992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f42361ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f423019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f42361ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423019dca0 con 0x7f4230100710 2023-10-18T12:28:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f42361ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423019de10 con 0x7f42301010d0 2023-10-18T12:28:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f42361ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423019dfd0 con 0x7f4230106720 2023-10-18T12:28:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f422effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f42301992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f422effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f42301992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44148/0 (socket says 172.21.15.70:44148) 2023-10-18T12:28:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f422effd700 1 -- 172.21.15.70:0/1455679088 learned_addr learned my addr 172.21.15.70:0/1455679088 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 0x7f4230198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.902+0000 7f422ffff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f423019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422effd700 1 -- 172.21.15.70:0/1455679088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 msgr2=0x7f423019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422effd700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f423019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422effd700 1 -- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 msgr2=0x7f4230198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422effd700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 0x7f4230198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422effd700 1 -- 172.21.15.70:0/1455679088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224009520 con 0x7f42301010d0 2023-10-18T12:28:03.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422ffff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f423019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:03.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 0x7f4230198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:03.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422effd700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f42301992c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f422400eb30 tx=0x7f422400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422cff9700 1 -- 172.21.15.70:0/1455679088 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224009880 con 0x7f42301010d0 2023-10-18T12:28:03.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42200095f0 con 0x7f42301010d0 2023-10-18T12:28:03.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422cff9700 1 -- 172.21.15.70:0/1455679088 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42240044e0 con 0x7f42301010d0 2023-10-18T12:28:03.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f422cff9700 1 -- 172.21.15.70:0/1455679088 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224010430 con 0x7f42301010d0 2023-10-18T12:28:03.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.903+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423019e4a0 con 0x7f42301010d0 2023-10-18T12:28:03.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.906+0000 7f422cff9700 1 -- 172.21.15.70:0/1455679088 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4224003ee0 con 0x7f42301010d0 2023-10-18T12:28:03.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.907+0000 7f4249a8ab80 1 Processor -- start 2023-10-18T12:28:03.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 -- start start 2023-10-18T12:28:03.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422cff9700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f42180a15f0 0x7f42180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422cff9700 1 -- 172.21.15.70:0/1455679088 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4224014070 con 0x7f42301010d0 2023-10-18T12:28:03.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e462fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e45d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e45da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e3d06e0 con 0x55ae2e462be0 2023-10-18T12:28:03.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e3b6c60 con 0x55ae2e463500 2023-10-18T12:28:03.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e3b4580 con 0x55ae2e45d5d0 2023-10-18T12:28:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422ffff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f42180a15f0 0x7f42180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e45d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e462fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e45d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44164/0 (socket says 172.21.15.70:44164) 2023-10-18T12:28:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422effd700 1 -- 172.21.15.70:0/600256182 learned_addr learned my addr 172.21.15.70:0/600256182 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.908+0000 7f422ffff700 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e45da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422f7fe700 1 -- 172.21.15.70:0/600256182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 msgr2=0x55ae2e45da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422f7fe700 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e45da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422f7fe700 1 -- 172.21.15.70:0/600256182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 msgr2=0x55ae2e45d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422f7fe700 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e45d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422f7fe700 1 -- 172.21.15.70:0/600256182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae2e36d3f0 con 0x55ae2e462be0 2023-10-18T12:28:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422effd700 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e45d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422ffff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f42180a15f0 0x7f42180a3ab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42280094f0 tx=0x7f4228009240 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f422f7fe700 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e462fc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4220006010 tx=0x7f4220005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee79c7014013bc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f4215ffb700 1 -- 172.21.15.70:0/600256182 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200049e0 con 0x55ae2e462be0 2023-10-18T12:28:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f4215ffb700 1 -- 172.21.15.70:0/600256182 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422001e040 con 0x55ae2e462be0 2023-10-18T12:28:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.909+0000 7f4215ffb700 1 -- 172.21.15.70:0/600256182 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000b9b0 con 0x55ae2e462be0 2023-10-18T12:28:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 -- 172.21.15.70:0/600256182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 msgr2=0x55ae2e462fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e462fc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4220006010 tx=0x7f4220005c80 comp rx=0 tx=0).stop 2023-10-18T12:28:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 -- 172.21.15.70:0/600256182 shutdown_connections 2023-10-18T12:28:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e45da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e462fc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/600256182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e45d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 -- 172.21.15.70:0/600256182 >> 172.21.15.70:0/600256182 conn(0x55ae2e3d1720 msgr2=0x55ae2e457b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 -- 172.21.15.70:0/600256182 shutdown_connections 2023-10-18T12:28:03.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.910+0000 7f4249a8ab80 1 -- 172.21.15.70:0/600256182 wait complete. 2023-10-18T12:28:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.911+0000 7f4249a8ab80 1 -- start start 2023-10-18T12:28:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.911+0000 7f4249a8ab80 1 Processor -- start 2023-10-18T12:28:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e45d5d0 0x55ae2e469530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e462be0 0x55ae2e466820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e463500 0x55ae2e466f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e3afe70 con 0x55ae2e463500 2023-10-18T12:28:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e34c710 con 0x55ae2e45d5d0 2023-10-18T12:28:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e34c3b0 con 0x55ae2e462be0 2023-10-18T12:28:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f4249a8ab80 10 client.0 _ll_register_callbacks cb 0x55ae2e296a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e45d5d0 0x55ae2e469530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e45d5d0 0x55ae2e469530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44174/0 (socket says 172.21.15.70:44174) 2023-10-18T12:28:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e462be0 0x55ae2e466820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e463500 0x55ae2e466f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422f7fe700 1 -- 172.21.15.70:0/3195481359 learned_addr learned my addr 172.21.15.70:0/3195481359 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 -- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e45d5d0 msgr2=0x55ae2e469530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e45d5d0 0x55ae2e469530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 -- 172.21.15.70:0/3195481359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e463500 msgr2=0x55ae2e466f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e463500 0x55ae2e466f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 -- 172.21.15.70:0/3195481359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423019e4a0 con 0x55ae2e462be0 2023-10-18T12:28:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f422effd700 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e462be0 0x55ae2e466820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4224097060 tx=0x7f4224099f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5682042b89f6a3b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.912+0000 7f420effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:28:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f420effd700 1 -- 172.21.15.70:0/3195481359 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422409aa50 con 0x55ae2e462be0 2023-10-18T12:28:03.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f420effd700 1 -- 172.21.15.70:0/3195481359 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f422409acf0 con 0x55ae2e462be0 2023-10-18T12:28:03.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae2e3b4ef0 con 0x55ae2e462be0 2023-10-18T12:28:03.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f420effd700 1 -- 172.21.15.70:0/3195481359 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f422400cd40 con 0x55ae2e462be0 2023-10-18T12:28:03.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 10 client.25894 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:03.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f423010dee0 con 0x55ae2e462be0 2023-10-18T12:28:03.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f420effd700 1 -- 172.21.15.70:0/3195481359 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae2e3b4ef0 con 0x55ae2e462be0 2023-10-18T12:28:03.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f420effd700 1 -- 172.21.15.70:0/3195481359 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4224099ae0 con 0x55ae2e462be0 2023-10-18T12:28:03.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 10 client.25894 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:03.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:03.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 20 client.25894 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:03.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e476f60 0x55ae2e479380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 4 client.25894 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:03.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.913+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae2e42c8b0 con 0x55ae2e476f60 2023-10-18T12:28:03.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.914+0000 7f422f7fe700 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e476f60 0x55ae2e479380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.914+0000 7f420effd700 10 client.25894 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.914+0000 7f422f7fe700 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e476f60 0x55ae2e479380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3539bc3fee05e465 server_cookie=5a144972eb74773d in_seq=0 out_seq=0 2023-10-18T12:28:03.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f420effd700 1 -- 172.21.15.70:0/3195481359 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae2e42c8b0 con 0x55ae2e476f60 2023-10-18T12:28:03.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f420effd700 10 client.25894 handle_command_reply: tid=0 2023-10-18T12:28:03.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 client.25894 shutdown 2023-10-18T12:28:03.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e462be0 msgr2=0x55ae2e466820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e462be0 0x55ae2e466820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4224097060 tx=0x7f4224099f50 comp rx=0 tx=0).stop 2023-10-18T12:28:03.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 shutdown_connections 2023-10-18T12:28:03.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e476f60 0x55ae2e479380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e463500 0x55ae2e466f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e45d5d0 0x55ae2e469530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/3195481359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e462be0 0x55ae2e466820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f420effd700 0 client.25894 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 >> 172.21.15.70:0/3195481359 conn(0x55ae2e3d1720 msgr2=0x55ae2e45c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 shutdown_connections 2023-10-18T12:28:03.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 1 -- 172.21.15.70:0/3195481359 wait complete. 2023-10-18T12:28:03.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.915+0000 7f4249a8ab80 20 client.25894 trim_cache size 0 max 16384 2023-10-18T12:28:03.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.932+0000 7f4249a8ab80 1 Processor -- start 2023-10-18T12:28:03.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.932+0000 7f4249a8ab80 1 -- start start 2023-10-18T12:28:03.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e460ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e46deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e469060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e452af0 con 0x55ae2e462be0 2023-10-18T12:28:03.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e230920 con 0x55ae2e463500 2023-10-18T12:28:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e467e20 con 0x55ae2e45d5d0 2023-10-18T12:28:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e460ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e46deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e469060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e460ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60544/0 (socket says 172.21.15.70:60544) 2023-10-18T12:28:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e46deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52070/0 (socket says 172.21.15.70:52070) 2023-10-18T12:28:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422f7fe700 1 -- 172.21.15.70:0/1697747020 learned_addr learned my addr 172.21.15.70:0/1697747020 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e469060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44180/0 (socket says 172.21.15.70:44180) 2023-10-18T12:28:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422f7fe700 1 -- 172.21.15.70:0/1697747020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 msgr2=0x55ae2e469060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.933+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e469060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f422f7fe700 1 -- 172.21.15.70:0/1697747020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 msgr2=0x55ae2e46deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e46deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f422f7fe700 1 -- 172.21.15.70:0/1697747020 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae2e1dade0 con 0x55ae2e45d5d0 2023-10-18T12:28:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f422effd700 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e46deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f422ffff700 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e469060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e460ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f422000b490 tx=0x7f4220005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72e19670a55d5dc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f42167fc700 1 -- 172.21.15.70:0/1697747020 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422001b030 con 0x55ae2e45d5d0 2023-10-18T12:28:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f42167fc700 1 -- 172.21.15.70:0/1697747020 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422000b890 con 0x55ae2e45d5d0 2023-10-18T12:28:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f42167fc700 1 -- 172.21.15.70:0/1697747020 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220022430 con 0x55ae2e45d5d0 2023-10-18T12:28:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1697747020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 msgr2=0x55ae2e460ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e460ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f422000b490 tx=0x7f4220005680 comp rx=0 tx=0).stop 2023-10-18T12:28:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1697747020 shutdown_connections 2023-10-18T12:28:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e463500 0x55ae2e469060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e45d5d0 0x55ae2e460ed0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1697747020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e462be0 0x55ae2e46deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.934+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1697747020 >> 172.21.15.70:0/1697747020 conn(0x55ae2e3d1720 msgr2=0x55ae2e45d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.935+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1697747020 shutdown_connections 2023-10-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.935+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1697747020 wait complete. 2023-10-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.935+0000 7f4249a8ab80 1 -- start start 2023-10-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.935+0000 7f4249a8ab80 1 Processor -- start 2023-10-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e45d5d0 0x55ae2e46c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 0x55ae2e46c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e463500 0x55ae2e4c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e46cca0 con 0x55ae2e45d5d0 2023-10-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e4ca2d0 con 0x55ae2e462be0 2023-10-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e4ca410 con 0x55ae2e463500 2023-10-18T12:28:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f4249a8ab80 10 client.0 _ll_register_callbacks cb 0x55ae2e296a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 0x55ae2e46c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 0x55ae2e46c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:44182/0 (socket says 172.21.15.70:44182) 2023-10-18T12:28:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 -- 172.21.15.70:0/1162963070 learned_addr learned my addr 172.21.15.70:0/1162963070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e45d5d0 0x55ae2e46c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422ffff700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e463500 0x55ae2e4c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 -- 172.21.15.70:0/1162963070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e463500 msgr2=0x55ae2e4c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e463500 0x55ae2e4c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 -- 172.21.15.70:0/1162963070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e45d5d0 msgr2=0x55ae2e46c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e45d5d0 0x55ae2e46c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 -- 172.21.15.70:0/1162963070 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423010dee0 con 0x55ae2e462be0 2023-10-18T12:28:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f422effd700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 0x55ae2e46c5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4224098940 tx=0x7f4224099d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb37673f57e5d25 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.936+0000 7f420effd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f420effd700 1 -- 172.21.15.70:0/1162963070 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224096930 con 0x55ae2e462be0 2023-10-18T12:28:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f420effd700 1 -- 172.21.15.70:0/1162963070 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f42240b6070 con 0x55ae2e462be0 2023-10-18T12:28:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f422001e040 con 0x55ae2e462be0 2023-10-18T12:28:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f420effd700 1 -- 172.21.15.70:0/1162963070 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f422409acf0 con 0x55ae2e462be0 2023-10-18T12:28:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 10 client.25898 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f4228007580 con 0x55ae2e462be0 2023-10-18T12:28:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f420effd700 1 -- 172.21.15.70:0/1162963070 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422001e040 con 0x55ae2e462be0 2023-10-18T12:28:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f420effd700 1 -- 172.21.15.70:0/1162963070 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f42240963e0 con 0x55ae2e462be0 2023-10-18T12:28:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 10 client.25898 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 20 client.25898 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e4cb920 0x55ae2e4cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:03.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 4 client.25898 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:28:03.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae2e3a8e50 con 0x55ae2e4cb920 2023-10-18T12:28:03.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.937+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e4cb920 0x55ae2e4cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:03.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.938+0000 7f422f7fe700 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e4cb920 0x55ae2e4cdd40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9246de650c4c59c8 server_cookie=e34059e31e9cdfdb in_seq=0 out_seq=0 2023-10-18T12:28:03.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.938+0000 7f420effd700 10 client.25898 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f420effd700 1 -- 172.21.15.70:0/1162963070 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ae2e3a8e50 con 0x55ae2e4cb920 2023-10-18T12:28:03.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f420effd700 10 client.25898 handle_command_reply: tid=0 2023-10-18T12:28:03.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 client.25898 shutdown 2023-10-18T12:28:03.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 msgr2=0x55ae2e46c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 0x55ae2e46c5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4224098940 tx=0x7f4224099d10 comp rx=0 tx=0).stop 2023-10-18T12:28:03.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 shutdown_connections 2023-10-18T12:28:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ae2e4cb920 0x55ae2e4cdd40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae2e463500 0x55ae2e4c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae2e45d5d0 0x55ae2e46c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f420effd700 0 client.25898 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 --2- 172.21.15.70:0/1162963070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ae2e462be0 0x55ae2e46c5c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 >> 172.21.15.70:0/1162963070 conn(0x55ae2e3d1720 msgr2=0x55ae2e4514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 shutdown_connections 2023-10-18T12:28:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 1 -- 172.21.15.70:0/1162963070 wait complete. 2023-10-18T12:28:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.940+0000 7f4249a8ab80 20 client.25898 trim_cache size 0 max 16384 2023-10-18T12:28:03.962 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:28:03.962 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:28:03.962 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:28:03.962 INFO:teuthology.orchestra.run.smithi070.stdout: "97d96ed6-914b-4500-8373-b856c742121f": { 2023-10-18T12:28:03.962 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:28:03.963 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "97d96ed6-914b-4500-8373-b856c742121f", 2023-10-18T12:28:03.963 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:28:03.963 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:28:03.963 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:28:03.963 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:28:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f42180a15f0 msgr2=0x7f42180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f42180a15f0 0x7f42180a3ab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42280094f0 tx=0x7f4228009240 comp rx=0 tx=0).stop 2023-10-18T12:28:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 msgr2=0x7f42301992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f42301992c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f422400eb30 tx=0x7f422400eef0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 shutdown_connections 2023-10-18T12:28:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4230106720 0x7f423019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230100710 0x7f4230198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f42180a15f0 0x7f42180a3ab0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 --2- 172.21.15.70:0/1455679088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42301010d0 0x7f42301992c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 >> 172.21.15.70:0/1455679088 conn(0x7f42300fc200 msgr2=0x7f42300fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 shutdown_connections 2023-10-18T12:28:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:03.941+0000 7f42361ff700 1 -- 172.21.15.70:0/1455679088 wait complete. 2023-10-18T12:28:03.967 INFO:tasks.cephfs.filesystem:scrub status for tag:97d96ed6-914b-4500-8373-b856c742121f - {'path': '/', 'tag': '97d96ed6-914b-4500-8373-b856c742121f', 'options': 'recursive,force'} 2023-10-18T12:28:04.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:04 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:28:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:04 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:04 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:28:04.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:04 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:04.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:04 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:28:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:04 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:04 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:28:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:04 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:04.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:04 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:28:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:04 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:04 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:28:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:04 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:04.835 INFO:tasks.workunit.client.0.smithi070.stdout: 1 91640 15.50 MB/sec warmup 33 sec latency 7.367 ms 2023-10-18T12:28:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:05 smithi186 ceph-mon[135253]: pgmap v376: 129 pgs: 129 active+clean; 45 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2023-10-18T12:28:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:05 smithi070 ceph-mon[98639]: pgmap v376: 129 pgs: 129 active+clean; 45 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2023-10-18T12:28:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:05 smithi152 ceph-mon[134767]: pgmap v376: 129 pgs: 129 active+clean; 45 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2023-10-18T12:28:05.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 93788 15.39 MB/sec warmup 34 sec latency 17.092 ms 2023-10-18T12:28:06.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 96554 15.36 MB/sec warmup 35 sec latency 18.239 ms 2023-10-18T12:28:07.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:07 smithi186 ceph-mon[135253]: pgmap v377: 129 pgs: 129 active+clean; 58 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 2.9 MiB/s wr, 227 op/s 2023-10-18T12:28:07.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:07 smithi070 ceph-mon[98639]: pgmap v377: 129 pgs: 129 active+clean; 58 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 2.9 MiB/s wr, 227 op/s 2023-10-18T12:28:07.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:07 smithi152 ceph-mon[134767]: pgmap v377: 129 pgs: 129 active+clean; 58 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 2.9 MiB/s wr, 227 op/s 2023-10-18T12:28:07.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 99477 15.42 MB/sec warmup 36 sec latency 74.069 ms 2023-10-18T12:28:08.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:08 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:28:08.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:08 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:28:08.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:08 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:28:08.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 101657 15.22 MB/sec warmup 37 sec latency 30.199 ms 2023-10-18T12:28:09.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:09 smithi070 ceph-mon[98639]: pgmap v378: 129 pgs: 129 active+clean; 64 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-10-18T12:28:09.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:09 smithi152 ceph-mon[134767]: pgmap v378: 129 pgs: 129 active+clean; 64 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-10-18T12:28:09.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:09 smithi186 ceph-mon[135253]: pgmap v378: 129 pgs: 129 active+clean; 64 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-10-18T12:28:09.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 103899 15.17 MB/sec warmup 38 sec latency 10.533 ms 2023-10-18T12:28:10.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 106901 15.25 MB/sec warmup 39 sec latency 24.667 ms 2023-10-18T12:28:11.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:11 smithi070 ceph-mon[98639]: pgmap v379: 129 pgs: 129 active+clean; 73 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 254 op/s 2023-10-18T12:28:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:11 smithi152 ceph-mon[134767]: pgmap v379: 129 pgs: 129 active+clean; 73 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 254 op/s 2023-10-18T12:28:11.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:11 smithi186 ceph-mon[135253]: pgmap v379: 129 pgs: 129 active+clean; 73 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 254 op/s 2023-10-18T12:28:11.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 110117 15.31 MB/sec warmup 40 sec latency 6.129 ms 2023-10-18T12:28:12.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 112204 15.11 MB/sec warmup 41 sec latency 38.899 ms 2023-10-18T12:28:13.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:13 smithi070 ceph-mon[98639]: pgmap v380: 129 pgs: 129 active+clean; 72 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 220 op/s 2023-10-18T12:28:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:13 smithi152 ceph-mon[134767]: pgmap v380: 129 pgs: 129 active+clean; 72 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 220 op/s 2023-10-18T12:28:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:13 smithi186 ceph-mon[135253]: pgmap v380: 129 pgs: 129 active+clean; 72 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 220 op/s 2023-10-18T12:28:13.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 114242 15.08 MB/sec warmup 42 sec latency 52.086 ms 2023-10-18T12:28:14.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 117383 15.16 MB/sec warmup 43 sec latency 7.163 ms 2023-10-18T12:28:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:15 smithi070 ceph-mon[98639]: pgmap v381: 129 pgs: 129 active+clean; 78 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 295 op/s 2023-10-18T12:28:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:15 smithi152 ceph-mon[134767]: pgmap v381: 129 pgs: 129 active+clean; 78 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 295 op/s 2023-10-18T12:28:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:15 smithi186 ceph-mon[135253]: pgmap v381: 129 pgs: 129 active+clean; 78 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 295 op/s 2023-10-18T12:28:15.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 120264 15.12 MB/sec warmup 44 sec latency 7.943 ms 2023-10-18T12:28:16.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 122139 15.03 MB/sec warmup 45 sec latency 27.917 ms 2023-10-18T12:28:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:17 smithi070 ceph-mon[98639]: pgmap v382: 129 pgs: 129 active+clean; 78 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-10-18T12:28:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:17 smithi152 ceph-mon[134767]: pgmap v382: 129 pgs: 129 active+clean; 78 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-10-18T12:28:17.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:17 smithi186 ceph-mon[135253]: pgmap v382: 129 pgs: 129 active+clean; 78 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-10-18T12:28:17.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 124875 15.01 MB/sec warmup 46 sec latency 15.396 ms 2023-10-18T12:28:18.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 127372 14.97 MB/sec warmup 47 sec latency 94.085 ms 2023-10-18T12:28:19.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:19 smithi070 ceph-mon[98639]: pgmap v383: 129 pgs: 129 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2023-10-18T12:28:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:19 smithi152 ceph-mon[134767]: pgmap v383: 129 pgs: 129 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2023-10-18T12:28:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:19 smithi186 ceph-mon[135253]: pgmap v383: 129 pgs: 129 active+clean; 77 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2023-10-18T12:28:19.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 129987 14.90 MB/sec warmup 48 sec latency 6.905 ms 2023-10-18T12:28:20.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 132014 14.87 MB/sec warmup 49 sec latency 83.314 ms 2023-10-18T12:28:21.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:21 smithi070 ceph-mon[98639]: pgmap v384: 129 pgs: 129 active+clean; 81 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 275 op/s 2023-10-18T12:28:21.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:21 smithi152 ceph-mon[134767]: pgmap v384: 129 pgs: 129 active+clean; 81 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 275 op/s 2023-10-18T12:28:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:21 smithi186 ceph-mon[135253]: pgmap v384: 129 pgs: 129 active+clean; 81 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 275 op/s 2023-10-18T12:28:21.836 INFO:tasks.workunit.client.0.smithi070.stdout: 1 135130 14.95 MB/sec warmup 50 sec latency 6.560 ms 2023-10-18T12:28:22.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 137810 14.91 MB/sec warmup 51 sec latency 17.359 ms 2023-10-18T12:28:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:23 smithi070 ceph-mon[98639]: pgmap v385: 129 pgs: 129 active+clean; 85 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 264 op/s 2023-10-18T12:28:23.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:23 smithi152 ceph-mon[134767]: pgmap v385: 129 pgs: 129 active+clean; 85 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 264 op/s 2023-10-18T12:28:23.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:23 smithi186 ceph-mon[135253]: pgmap v385: 129 pgs: 129 active+clean; 85 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 264 op/s 2023-10-18T12:28:23.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 139304 14.76 MB/sec warmup 52 sec latency 60.440 ms 2023-10-18T12:28:24.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 142382 14.83 MB/sec warmup 53 sec latency 5.879 ms 2023-10-18T12:28:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:25 smithi186 ceph-mon[135253]: pgmap v386: 129 pgs: 129 active+clean; 87 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 318 op/s 2023-10-18T12:28:25.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:25 smithi070 ceph-mon[98639]: pgmap v386: 129 pgs: 129 active+clean; 87 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 318 op/s 2023-10-18T12:28:25.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:25 smithi152 ceph-mon[134767]: pgmap v386: 129 pgs: 129 active+clean; 87 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 318 op/s 2023-10-18T12:28:25.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 145285 14.82 MB/sec warmup 54 sec latency 32.470 ms 2023-10-18T12:28:26.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 147807 14.76 MB/sec warmup 55 sec latency 6.693 ms 2023-10-18T12:28:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:27 smithi186 ceph-mon[135253]: pgmap v387: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2023-10-18T12:28:27.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:27 smithi070 ceph-mon[98639]: pgmap v387: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2023-10-18T12:28:27.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:27 smithi152 ceph-mon[134767]: pgmap v387: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2023-10-18T12:28:27.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 150003 14.73 MB/sec warmup 56 sec latency 14.168 ms 2023-10-18T12:28:28.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 152772 14.79 MB/sec warmup 57 sec latency 55.737 ms 2023-10-18T12:28:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:29 smithi186 ceph-mon[135253]: pgmap v388: 129 pgs: 129 active+clean; 87 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2023-10-18T12:28:29.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:29 smithi070 ceph-mon[98639]: pgmap v388: 129 pgs: 129 active+clean; 87 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2023-10-18T12:28:29.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:29 smithi152 ceph-mon[134767]: pgmap v388: 129 pgs: 129 active+clean; 87 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2023-10-18T12:28:29.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 155468 14.75 MB/sec warmup 58 sec latency 7.391 ms 2023-10-18T12:28:30.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 157904 14.71 MB/sec warmup 59 sec latency 31.509 ms 2023-10-18T12:28:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:31 smithi186 ceph-mon[135253]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 302 op/s 2023-10-18T12:28:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:31 smithi070 ceph-mon[98639]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 302 op/s 2023-10-18T12:28:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:31 smithi152 ceph-mon[134767]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 302 op/s 2023-10-18T12:28:31.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 160449 14.70 MB/sec warmup 60 sec latency 20.218 ms 2023-10-18T12:28:32.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:32 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:28:32.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:32 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:28:32.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:32 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:28:32.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 163463 14.76 MB/sec warmup 61 sec latency 15.627 ms 2023-10-18T12:28:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:33 smithi186 ceph-mon[135253]: pgmap v390: 129 pgs: 129 active+clean; 92 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2023-10-18T12:28:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:28:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:28:33.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:33 smithi070 ceph-mon[98639]: pgmap v390: 129 pgs: 129 active+clean; 92 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2023-10-18T12:28:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:28:33.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:28:33.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:33 smithi152 ceph-mon[134767]: pgmap v390: 129 pgs: 129 active+clean; 92 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2023-10-18T12:28:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:28:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:28:33.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:28:33.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 165009 14.61 MB/sec warmup 62 sec latency 68.837 ms 2023-10-18T12:28:33.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:28:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.193+0000 7f6b025ff700 1 -- 172.21.15.70:0/2013871810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ffaa0 msgr2=0x7f6afc10bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.193+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2013871810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ffaa0 0x7f6afc10bc20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b0a0 tx=0x7f6af400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 -- 172.21.15.70:0/2013871810 shutdown_connections 2023-10-18T12:28:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2013871810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ffaa0 0x7f6afc10bc20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2013871810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6afc104e80 0x7f6afc105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2013871810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 -- 172.21.15.70:0/2013871810 >> 172.21.15.70:0/2013871810 conn(0x7f6afc0757e0 msgr2=0x7f6afc0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 -- 172.21.15.70:0/2013871810 shutdown_connections 2023-10-18T12:28:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.194+0000 7f6b025ff700 1 -- 172.21.15.70:0/2013871810 wait complete. 2023-10-18T12:28:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.195+0000 7f6b025ff700 1 Processor -- start 2023-10-18T12:28:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 -- start start 2023-10-18T12:28:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6afc0ffaa0 0x7f6afc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6afc104e80 0x7f6afc196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1970c0 con 0x7f6afc0ffaa0 2023-10-18T12:28:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc197230 con 0x7f6afc104e80 2023-10-18T12:28:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b025ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1973f0 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52730/0 (socket says 172.21.15.70:52730) 2023-10-18T12:28:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6afbfff700 1 -- 172.21.15.70:0/2884788006 learned_addr learned my addr 172.21.15.70:0/2884788006 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6afc0ffaa0 0x7f6afc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.196+0000 7f6b00b9c700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6afc104e80 0x7f6afc196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6afbfff700 1 -- 172.21.15.70:0/2884788006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6afc104e80 msgr2=0x7f6afc196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6afbfff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6afc104e80 0x7f6afc196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6afbfff700 1 -- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6afc0ffaa0 msgr2=0x7f6afc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6afbfff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6afc0ffaa0 0x7f6afc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6afbfff700 1 -- 172.21.15.70:0/2884788006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4009d00 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6afbfff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc19c930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00eab0 tx=0x7f6aec00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6af97fa700 1 -- 172.21.15.70:0/2884788006 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00ccd0 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6af97fa700 1 -- 172.21.15.70:0/2884788006 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aec00ce30 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc1976d0 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6af97fa700 1 -- 172.21.15.70:0/2884788006 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0157c0 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.197+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc1a3920 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.202+0000 7f6af97fa700 1 -- 172.21.15.70:0/2884788006 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6aec0163e0 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 Processor -- start 2023-10-18T12:28:34.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6af97fa700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6ae40a1670 0x7f6ae40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 -- start start 2023-10-18T12:28:34.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6af97fa700 1 -- 172.21.15.70:0/2884788006 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6aec0cd5b0 con 0x7f6afc0ff0e0 2023-10-18T12:28:34.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 0x559b308a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a15d0 0x559b308a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b308146e0 con 0x559b308a7500 2023-10-18T12:28:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b307fac60 con 0x559b308a6be0 2023-10-18T12:28:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.203+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b307f8580 con 0x559b308a15d0 2023-10-18T12:28:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6b00b9c700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6ae40a1670 0x7f6ae40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 0x559b308a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 0x559b308a6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43298/0 (socket says 172.21.15.70:43298) 2023-10-18T12:28:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 -- 172.21.15.70:0/1063155766 learned_addr learned my addr 172.21.15.70:0/1063155766 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6b00b9c700 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a15d0 0x559b308a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 -- 172.21.15.70:0/1063155766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a15d0 msgr2=0x559b308a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a15d0 0x559b308a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 -- 172.21.15.70:0/1063155766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 msgr2=0x559b308a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6afbfff700 1 -- 172.21.15.70:0/1063155766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b307b13f0 con 0x559b308a6be0 2023-10-18T12:28:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.204+0000 7f6b00b9c700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6ae40a1670 0x7f6ae40a3b30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b780 tx=0x7f6af4006230 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.205+0000 7f6afbfff700 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 0x559b308a6fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6aec09a220 tx=0x7f6aec09aa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c247a394d192cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.205+0000 7f6ae27fc700 1 -- 172.21.15.70:0/1063155766 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec094ea0 con 0x559b308a6be0 2023-10-18T12:28:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.205+0000 7f6ae27fc700 1 -- 172.21.15.70:0/1063155766 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aec0a2620 con 0x559b308a6be0 2023-10-18T12:28:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.205+0000 7f6ae27fc700 1 -- 172.21.15.70:0/1063155766 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec096b10 con 0x559b308a6be0 2023-10-18T12:28:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.205+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1063155766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 msgr2=0x559b308a6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.205+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 0x559b308a6fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6aec09a220 tx=0x7f6aec09aa80 comp rx=0 tx=0).stop 2023-10-18T12:28:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1063155766 shutdown_connections 2023-10-18T12:28:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a15d0 0x559b308a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a6be0 0x559b308a6fc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1063155766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1063155766 >> 172.21.15.70:0/1063155766 conn(0x559b30815720 msgr2=0x559b3089bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1063155766 shutdown_connections 2023-10-18T12:28:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1063155766 wait complete. 2023-10-18T12:28:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.206+0000 7f6b15ea8b80 1 -- start start 2023-10-18T12:28:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 Processor -- start 2023-10-18T12:28:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a6be0 0x559b308a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a7500 0x559b308a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b307f3e70 con 0x559b308a6be0 2023-10-18T12:28:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b30790710 con 0x559b308a15d0 2023-10-18T12:28:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b307903b0 con 0x559b308a7500 2023-10-18T12:28:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b15ea8b80 10 client.0 _ll_register_callbacks cb 0x559b306daa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6afb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a6be0 0x559b308a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43306/0 (socket says 172.21.15.70:43306) 2023-10-18T12:28:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6afbfff700 1 -- 172.21.15.70:0/3914218659 learned_addr learned my addr 172.21.15.70:0/3914218659 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.207+0000 7f6b00b9c700 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a7500 0x559b308a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6afbfff700 1 -- 172.21.15.70:0/3914218659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a7500 msgr2=0x559b308a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6afbfff700 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a7500 0x559b308a8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6afbfff700 1 -- 172.21.15.70:0/3914218659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a6be0 msgr2=0x559b308a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6afbfff700 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a6be0 0x559b308a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6afbfff700 1 -- 172.21.15.70:0/3914218659 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc1a3920 con 0x559b308a15d0 2023-10-18T12:28:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6afbfff700 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308ab3e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0a20f0 tx=0x7f6aec09a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3835c8fbc565c9cd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6adb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6adb7fe700 1 -- 172.21.15.70:0/3914218659 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0986f0 con 0x559b308a15d0 2023-10-18T12:28:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b307f8ef0 con 0x559b308a15d0 2023-10-18T12:28:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.208+0000 7f6adb7fe700 1 -- 172.21.15.70:0/3914218659 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6aec016690 con 0x559b308a15d0 2023-10-18T12:28:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6adb7fe700 1 -- 172.21.15.70:0/3914218659 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aec016850 con 0x559b308a15d0 2023-10-18T12:28:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 10 client.25910 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559b3061ede0 con 0x559b308a15d0 2023-10-18T12:28:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6adb7fe700 1 -- 172.21.15.70:0/3914218659 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b307f8ef0 con 0x559b308a15d0 2023-10-18T12:28:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6adb7fe700 1 -- 172.21.15.70:0/3914218659 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6aec099810 con 0x559b308a15d0 2023-10-18T12:28:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 10 client.25910 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 20 client.25910 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b308bed60 0x559b308c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 4 client.25910 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b308708b0 con 0x559b308bed60 2023-10-18T12:28:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.209+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b308bed60 0x559b308c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.210+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b308bed60 0x559b308c1150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e110554a6a7d33 server_cookie=7ede4860eb4c596b in_seq=0 out_seq=0 2023-10-18T12:28:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.210+0000 7f6adb7fe700 10 client.25910 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.211+0000 7f6adb7fe700 1 -- 172.21.15.70:0/3914218659 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559b308708b0 con 0x559b308bed60 2023-10-18T12:28:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.211+0000 7f6adb7fe700 10 client.25910 handle_command_reply: tid=0 2023-10-18T12:28:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.211+0000 7f6b15ea8b80 1 client.25910 shutdown 2023-10-18T12:28:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 msgr2=0x559b308ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308ab3e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0a20f0 tx=0x7f6aec09a410 comp rx=0 tx=0).stop 2023-10-18T12:28:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 shutdown_connections 2023-10-18T12:28:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b308bed60 0x559b308c1150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a7500 0x559b308a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308ab3e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/3914218659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a6be0 0x559b308a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6adb7fe700 0 client.25910 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 >> 172.21.15.70:0/3914218659 conn(0x559b30815720 msgr2=0x559b3089e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 shutdown_connections 2023-10-18T12:28:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/3914218659 wait complete. 2023-10-18T12:28:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.212+0000 7f6b15ea8b80 20 client.25910 trim_cache size 0 max 16384 2023-10-18T12:28:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 Processor -- start 2023-10-18T12:28:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 -- start start 2023-10-18T12:28:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a15d0 0x559b308a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a7500 0x559b308ba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b30674920 con 0x559b308a15d0 2023-10-18T12:28:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b308bbd40 con 0x559b308a7500 2023-10-18T12:28:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b308a2000 con 0x559b308a6be0 2023-10-18T12:28:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a15d0 0x559b308a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6afb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6b00b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a7500 0x559b308ba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.232+0000 7f6afb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52770/0 (socket says 172.21.15.70:52770) 2023-10-18T12:28:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 -- 172.21.15.70:0/1998584266 learned_addr learned my addr 172.21.15.70:0/1998584266 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 -- 172.21.15.70:0/1998584266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a7500 msgr2=0x559b308ba9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a7500 0x559b308ba9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 -- 172.21.15.70:0/1998584266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a15d0 msgr2=0x559b308a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a15d0 0x559b308a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 -- 172.21.15.70:0/1998584266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b30607c60 con 0x559b308a6be0 2023-10-18T12:28:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afbfff700 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a15d0 0x559b308a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:34.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005d90 tx=0x7f6af0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafede3c3968a41d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6ae2ffd700 1 -- 172.21.15.70:0/1998584266 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0017040 con 0x559b308a6be0 2023-10-18T12:28:34.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6ae2ffd700 1 -- 172.21.15.70:0/1998584266 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af0018040 con 0x559b308a6be0 2023-10-18T12:28:34.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.233+0000 7f6ae2ffd700 1 -- 172.21.15.70:0/1998584266 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0011a00 con 0x559b308a6be0 2023-10-18T12:28:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1998584266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 msgr2=0x559b308b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005d90 tx=0x7f6af0009260 comp rx=0 tx=0).stop 2023-10-18T12:28:34.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1998584266 shutdown_connections 2023-10-18T12:28:34.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a7500 0x559b308ba9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a15d0 0x559b308a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/1998584266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b6260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1998584266 >> 172.21.15.70:0/1998584266 conn(0x559b30815720 msgr2=0x559b3089b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1998584266 shutdown_connections 2023-10-18T12:28:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.234+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/1998584266 wait complete. 2023-10-18T12:28:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 -- start start 2023-10-18T12:28:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 Processor -- start 2023-10-18T12:28:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b308b5330 con 0x559b308a7500 2023-10-18T12:28:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b308b5470 con 0x559b308a15d0 2023-10-18T12:28:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b30912b00 con 0x559b308a6be0 2023-10-18T12:28:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.235+0000 7f6b15ea8b80 10 client.0 _ll_register_callbacks cb 0x559b306daa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43336/0 (socket says 172.21.15.70:43336) 2023-10-18T12:28:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 -- 172.21.15.70:0/4151505612 learned_addr learned my addr 172.21.15.70:0/4151505612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6b00b9c700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 -- 172.21.15.70:0/4151505612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 msgr2=0x559b308b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 -- 172.21.15.70:0/4151505612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 msgr2=0x559b308b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 -- 172.21.15.70:0/4151505612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b3061ede0 con 0x559b308a15d0 2023-10-18T12:28:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6afbfff700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308b6220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6aec016e40 tx=0x7f6aec099a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=579211678f225ed7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.236+0000 7f6adb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6adb7fe700 1 -- 172.21.15.70:0/4151505612 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec094ea0 con 0x559b308a15d0 2023-10-18T12:28:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6adb7fe700 1 -- 172.21.15.70:0/4151505612 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6aec016890 con 0x559b308a15d0 2023-10-18T12:28:34.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aec0a2620 con 0x559b308a15d0 2023-10-18T12:28:34.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6adb7fe700 1 -- 172.21.15.70:0/4151505612 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aec09aa00 con 0x559b308a15d0 2023-10-18T12:28:34.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 10 client.25916 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:34.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f6af001f9f0 con 0x559b308a15d0 2023-10-18T12:28:34.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6adb7fe700 1 -- 172.21.15.70:0/4151505612 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aec0a2620 con 0x559b308a15d0 2023-10-18T12:28:34.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6adb7fe700 1 -- 172.21.15.70:0/4151505612 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6aec09a4f0 con 0x559b308a15d0 2023-10-18T12:28:34.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 10 client.25916 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:34.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:34.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 20 client.25916 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:34.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b30913dc0 0x559b309161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 4 client.25916 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:28:34.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.237+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b307ece50 con 0x559b30913dc0 2023-10-18T12:28:34.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.238+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b30913dc0 0x559b309161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.238+0000 7f6adb7fe700 10 client.25916 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:34.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.238+0000 7f6afb7fe700 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b30913dc0 0x559b309161e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fdde50be25a0dde server_cookie=520b79bd969e8408 in_seq=0 out_seq=0 2023-10-18T12:28:34.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6adb7fe700 1 -- 172.21.15.70:0/4151505612 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559b307ece50 con 0x559b30913dc0 2023-10-18T12:28:34.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6adb7fe700 10 client.25916 handle_command_reply: tid=0 2023-10-18T12:28:34.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 client.25916 shutdown 2023-10-18T12:28:34.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 msgr2=0x559b308b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308b6220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6aec016e40 tx=0x7f6aec099a10 comp rx=0 tx=0).stop 2023-10-18T12:28:34.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 shutdown_connections 2023-10-18T12:28:34.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559b30913dc0 0x559b309161e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b308a7500 0x559b308b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b308a15d0 0x559b308b6220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 --2- 172.21.15.70:0/4151505612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b308a6be0 0x559b308b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6adb7fe700 0 client.25916 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:34.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 >> 172.21.15.70:0/4151505612 conn(0x559b30815720 msgr2=0x559b308954b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 shutdown_connections 2023-10-18T12:28:34.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.239+0000 7f6b15ea8b80 1 -- 172.21.15.70:0/4151505612 wait complete. 2023-10-18T12:28:34.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b15ea8b80 20 client.25916 trim_cache size 0 max 16384 2023-10-18T12:28:34.257 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:28:34.257 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:28:34.257 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:28:34.257 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:28:34.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6ae40a1670 msgr2=0x7f6ae40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6ae40a1670 0x7f6ae40a3b30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b780 tx=0x7f6af4006230 comp rx=0 tx=0).stop 2023-10-18T12:28:34.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 msgr2=0x7f6afc19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc19c930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00eab0 tx=0x7f6aec00edc0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 shutdown_connections 2023-10-18T12:28:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6ae40a1670 0x7f6ae40a3b30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6afc104e80 0x7f6afc196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6afc0ff0e0 0x7f6afc19c930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 --2- 172.21.15.70:0/2884788006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6afc0ffaa0 0x7f6afc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 >> 172.21.15.70:0/2884788006 conn(0x7f6afc0757e0 msgr2=0x7f6afc0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.240+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 shutdown_connections 2023-10-18T12:28:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.241+0000 7f6b025ff700 1 -- 172.21.15.70:0/2884788006 wait complete. 2023-10-18T12:28:34.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:28:34.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:28:34.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/2364401406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c34101060 msgr2=0x7f6c3410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/2364401406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c34101060 0x7f6c3410d570 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00b0a0 tx=0x7f6c2c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/2364401406 shutdown_connections 2023-10-18T12:28:34.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/2364401406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c34101060 0x7f6c3410d570 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/2364401406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c341066b0 0x7f6c34106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/2364401406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.492+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/2364401406 >> 172.21.15.70:0/2364401406 conn(0x7f6c340fc1d0 msgr2=0x7f6c340fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.493+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/2364401406 shutdown_connections 2023-10-18T12:28:34.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.493+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/2364401406 wait complete. 2023-10-18T12:28:34.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 Processor -- start 2023-10-18T12:28:34.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 -- start start 2023-10-18T12:28:34.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101060 0x7f6c34073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c341066b0 0x7f6c34073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a4ef0 con 0x7f6c34101060 2023-10-18T12:28:34.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a5060 con 0x7f6c341006a0 2023-10-18T12:28:34.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3c7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a5220 con 0x7f6c341066b0 2023-10-18T12:28:34.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3a59b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3a59b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43352/0 (socket says 172.21.15.70:43352) 2023-10-18T12:28:34.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.494+0000 7f6c3a59b700 1 -- 172.21.15.70:0/1434108751 learned_addr learned my addr 172.21.15.70:0/1434108751 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:34.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c39d9a700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101060 0x7f6c34073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3ad9c700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c341066b0 0x7f6c34073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3a59b700 1 -- 172.21.15.70:0/1434108751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c341066b0 msgr2=0x7f6c34073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3a59b700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c341066b0 0x7f6c34073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3a59b700 1 -- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101060 msgr2=0x7f6c34073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3a59b700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101060 0x7f6c34073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3a59b700 1 -- 172.21.15.70:0/1434108751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c009d00 con 0x7f6c341006a0 2023-10-18T12:28:34.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3ad9c700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c341066b0 0x7f6c34073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3a59b700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34072af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e9a0 tx=0x7f6c3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30009790 con 0x7f6c341006a0 2023-10-18T12:28:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c341a5500 con 0x7f6c341006a0 2023-10-18T12:28:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c3000cbd0 con 0x7f6c341006a0 2023-10-18T12:28:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.495+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30018680 con 0x7f6c341006a0 2023-10-18T12:28:34.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.496+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c341a5a10 con 0x7f6c341006a0 2023-10-18T12:28:34.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.497+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c34108ca0 con 0x7f6c341006a0 2023-10-18T12:28:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.499+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6c3000cd40 con 0x7f6c341006a0 2023-10-18T12:28:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.500+0000 7f6c2b7fe700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c200a15f0 0x7f6c200a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.501+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6c30014070 con 0x7f6c341006a0 2023-10-18T12:28:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.501+0000 7f6c39d9a700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c200a15f0 0x7f6c200a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.502+0000 7f6c39d9a700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c200a15f0 0x7f6c200a3ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6c34074220 tx=0x7f6c24009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.504+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6c30096910 con 0x7f6c341006a0 2023-10-18T12:28:34.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.785+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c34073fd0 con 0x7f6c341006a0 2023-10-18T12:28:34.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.786+0000 7f6c2b7fe700 1 -- 172.21.15.70:0/1434108751 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f6c30099f30 con 0x7f6c341006a0 2023-10-18T12:28:34.789 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:28:34.790 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:28:34.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c200a15f0 msgr2=0x7f6c200a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c200a15f0 0x7f6c200a3ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6c34074220 tx=0x7f6c24009260 comp rx=0 tx=0).stop 2023-10-18T12:28:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 msgr2=0x7f6c34072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34072af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e9a0 tx=0x7f6c3000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 shutdown_connections 2023-10-18T12:28:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c341066b0 0x7f6c34073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c200a15f0 0x7f6c200a3ab0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c341006a0 0x7f6c34072af0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 --2- 172.21.15.70:0/1434108751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101060 0x7f6c34073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:34.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 >> 172.21.15.70:0/1434108751 conn(0x7f6c340fc1d0 msgr2=0x7f6c340fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:34.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 shutdown_connections 2023-10-18T12:28:34.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:34.789+0000 7f6c3c7ff700 1 -- 172.21.15.70:0/1434108751 wait complete. 2023-10-18T12:28:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:28:34.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:28:34.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 167470 14.60 MB/sec warmup 63 sec latency 10.904 ms 2023-10-18T12:28:35.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 -- 172.21.15.70:0/878100304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 msgr2=0x7f1a6c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 --2- 172.21.15.70:0/878100304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 0x7f1a6c106b10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009990 tx=0x7f1a5c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 -- 172.21.15.70:0/878100304 shutdown_connections 2023-10-18T12:28:35.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 --2- 172.21.15.70:0/878100304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c1010e0 0x7f1a6c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 --2- 172.21.15.70:0/878100304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 0x7f1a6c106b10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 --2- 172.21.15.70:0/878100304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c100720 0x7f1a6c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 -- 172.21.15.70:0/878100304 >> 172.21.15.70:0/878100304 conn(0x7f1a6c0fc1b0 msgr2=0x7f1a6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:35.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.032+0000 7f1a70dff700 1 -- 172.21.15.70:0/878100304 shutdown_connections 2023-10-18T12:28:35.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.033+0000 7f1a70dff700 1 -- 172.21.15.70:0/878100304 wait complete. 2023-10-18T12:28:35.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.034+0000 7f1a70dff700 1 Processor -- start 2023-10-18T12:28:35.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.034+0000 7f1a70dff700 1 -- start start 2023-10-18T12:28:35.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a70dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 0x7f1a6c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a70dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c1010e0 0x7f1a6c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a70dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 0x7f1a6c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a70dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c199200 con 0x7f1a6c100720 2023-10-18T12:28:35.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a70dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c199370 con 0x7f1a6c1010e0 2023-10-18T12:28:35.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a70dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c199530 con 0x7f1a6c106730 2023-10-18T12:28:35.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 0x7f1a6c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 0x7f1a6c19ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39180/0 (socket says 172.21.15.70:39180) 2023-10-18T12:28:35.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 -- 172.21.15.70:0/1791515405 learned_addr learned my addr 172.21.15.70:0/1791515405 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:35.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 0x7f1a6c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c1010e0 0x7f1a6c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 -- 172.21.15.70:0/1791515405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 msgr2=0x7f1a6c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 0x7f1a6c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 -- 172.21.15.70:0/1791515405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c1010e0 msgr2=0x7f1a6c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c1010e0 0x7f1a6c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a6a59c700 1 -- 172.21.15.70:0/1791515405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0095f0 con 0x7f1a6c100720 2023-10-18T12:28:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.035+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c1010e0 0x7f1a6c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.036+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 0x7f1a6c19ea70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000c00 tx=0x7f1a5c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.036+0000 7f1a5b7fe700 1 -- 172.21.15.70:0/1791515405 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c01d070 con 0x7f1a6c100720 2023-10-18T12:28:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.036+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c1997b0 con 0x7f1a6c100720 2023-10-18T12:28:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.036+0000 7f1a5b7fe700 1 -- 172.21.15.70:0/1791515405 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a5c022470 con 0x7f1a6c100720 2023-10-18T12:28:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.036+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c1a5a60 con 0x7f1a6c100720 2023-10-18T12:28:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.036+0000 7f1a5b7fe700 1 -- 172.21.15.70:0/1791515405 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0175f0 con 0x7f1a6c100720 2023-10-18T12:28:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.041+0000 7f1a5b7fe700 1 -- 172.21.15.70:0/1791515405 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1a5c0225e0 con 0x7f1a6c100720 2023-10-18T12:28:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.042+0000 7f1a846f8b80 1 Processor -- start 2023-10-18T12:28:35.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 -- start start 2023-10-18T12:28:35.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a5b7fe700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1a540a15f0 0x7f1a540a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919761be0 0x55c919761fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c91975c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c91975ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9196cf6e0 con 0x55c919762500 2023-10-18T12:28:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9196b5c60 con 0x55c91975c5d0 2023-10-18T12:28:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9196b3580 con 0x55c919761be0 2023-10-18T12:28:35.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a5b7fe700 1 -- 172.21.15.70:0/1791515405 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1a5c0ce720 con 0x7f1a6c100720 2023-10-18T12:28:35.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.043+0000 7f1a69d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c91975c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1a540a15f0 0x7f1a540a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c91975c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39194/0 (socket says 172.21.15.70:39194) 2023-10-18T12:28:35.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 -- 172.21.15.70:0/1048126720 learned_addr learned my addr 172.21.15.70:0/1048126720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:35.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c91975ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919761be0 0x55c919761fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 -- 172.21.15.70:0/1048126720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919761be0 msgr2=0x55c919761fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919761be0 0x55c919761fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 -- 172.21.15.70:0/1048126720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 msgr2=0x55c91975ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c91975ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.044+0000 7f1a69d9b700 1 -- 172.21.15.70:0/1048126720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c91966c3f0 con 0x55c919762500 2023-10-18T12:28:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.045+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c91975c090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009d80 tx=0x7f1a600116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f663607413391a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.045+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1a540a15f0 0x7f1a540a3ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009ec0 tx=0x7f1a64009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.045+0000 7f1a58ff9700 1 -- 172.21.15.70:0/1048126720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60011e20 con 0x55c919762500 2023-10-18T12:28:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.045+0000 7f1a58ff9700 1 -- 172.21.15.70:0/1048126720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a60012910 con 0x55c919762500 2023-10-18T12:28:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.045+0000 7f1a58ff9700 1 -- 172.21.15.70:0/1048126720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000f7e0 con 0x55c919762500 2023-10-18T12:28:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 -- 172.21.15.70:0/1048126720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 msgr2=0x55c91975c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c91975c090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009d80 tx=0x7f1a600116c0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 -- 172.21.15.70:0/1048126720 shutdown_connections 2023-10-18T12:28:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c91975ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919761be0 0x55c919761fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/1048126720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c91975c090 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 -- 172.21.15.70:0/1048126720 >> 172.21.15.70:0/1048126720 conn(0x55c9196d0720 msgr2=0x55c919756b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 -- 172.21.15.70:0/1048126720 shutdown_connections 2023-10-18T12:28:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 -- 172.21.15.70:0/1048126720 wait complete. 2023-10-18T12:28:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.046+0000 7f1a846f8b80 1 -- start start 2023-10-18T12:28:35.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 Processor -- start 2023-10-18T12:28:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c919768410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919761be0 0x55c9197657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c919765e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9196aee70 con 0x55c919761be0 2023-10-18T12:28:35.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91964b710 con 0x55c91975c5d0 2023-10-18T12:28:35.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91964b3b0 con 0x55c919762500 2023-10-18T12:28:35.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.047+0000 7f1a846f8b80 10 client.0 _ll_register_callbacks cb 0x55c919595a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:35.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c919768410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c919768410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43372/0 (socket says 172.21.15.70:43372) 2023-10-18T12:28:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 -- 172.21.15.70:0/3554439426 learned_addr learned my addr 172.21.15.70:0/3554439426 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919761be0 0x55c9197657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 -- 172.21.15.70:0/3554439426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 msgr2=0x55c919765e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c919765e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 -- 172.21.15.70:0/3554439426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919761be0 msgr2=0x55c9197657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919761be0 0x55c9197657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 -- 172.21.15.70:0/3554439426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a600099b0 con 0x55c91975c5d0 2023-10-18T12:28:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919761be0 0x55c9197657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c919768410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c097f00 tx=0x7f1a5c09b080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd26fb875ffe7f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a51ffb700 1 -- 172.21.15.70:0/3554439426 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0ce5c0 con 0x55c91975c5d0 2023-10-18T12:28:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a51ffb700 1 -- 172.21.15.70:0/3554439426 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1a5c09b490 con 0x55c91975c5d0 2023-10-18T12:28:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.048+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9196b3ef0 con 0x55c91975c5d0 2023-10-18T12:28:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.049+0000 7f1a51ffb700 1 -- 172.21.15.70:0/3554439426 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a5c09f5e0 con 0x55c91975c5d0 2023-10-18T12:28:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.049+0000 7f1a846f8b80 10 client.25934 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.049+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c9194d9de0 con 0x55c91975c5d0 2023-10-18T12:28:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.049+0000 7f1a51ffb700 1 -- 172.21.15.70:0/3554439426 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9196b3ef0 con 0x55c91975c5d0 2023-10-18T12:28:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a51ffb700 1 -- 172.21.15.70:0/3554439426 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1a5c0a3300 con 0x55c91975c5d0 2023-10-18T12:28:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a846f8b80 10 client.25934 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a846f8b80 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a846f8b80 20 client.25934 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c91977e390 0x55c9197807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a846f8b80 4 client.25934 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c91972b8b0 con 0x55c91977e390 2023-10-18T12:28:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.050+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c91977e390 0x55c9197807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.051+0000 7f1a51ffb700 10 client.25934 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.051+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c91977e390 0x55c9197807b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cecb3173c2d3f2 server_cookie=9769b3d15b9495ea in_seq=0 out_seq=0 2023-10-18T12:28:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.052+0000 7f1a51ffb700 1 -- 172.21.15.70:0/3554439426 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c91972b8b0 con 0x55c91977e390 2023-10-18T12:28:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.052+0000 7f1a51ffb700 10 client.25934 handle_command_reply: tid=0 2023-10-18T12:28:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 client.25934 shutdown 2023-10-18T12:28:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 msgr2=0x55c919768410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c919768410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c097f00 tx=0x7f1a5c09b080 comp rx=0 tx=0).stop 2023-10-18T12:28:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 shutdown_connections 2023-10-18T12:28:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c91977e390 0x55c9197807b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c919765e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c91975c5d0 0x55c919768410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/3554439426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919761be0 0x55c9197657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a51ffb700 0 client.25934 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 >> 172.21.15.70:0/3554439426 conn(0x55c9196d0720 msgr2=0x55c919756ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 shutdown_connections 2023-10-18T12:28:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 1 -- 172.21.15.70:0/3554439426 wait complete. 2023-10-18T12:28:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.053+0000 7f1a846f8b80 20 client.25934 trim_cache size 0 max 16384 2023-10-18T12:28:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 Processor -- start 2023-10-18T12:28:35.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 -- start start 2023-10-18T12:28:35.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c91975c5d0 0x55c91975c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c9197515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c919764090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91952f920 con 0x55c919762500 2023-10-18T12:28:35.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c919768a20 con 0x55c919761be0 2023-10-18T12:28:35.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c919758430 con 0x55c91975c5d0 2023-10-18T12:28:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c91975c5d0 0x55c91975c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a69d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c9197515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a69d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c9197515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43388/0 (socket says 172.21.15.70:43388) 2023-10-18T12:28:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a69d9b700 1 -- 172.21.15.70:0/797530625 learned_addr learned my addr 172.21.15.70:0/797530625 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:35.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.069+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c91975c5d0 0x55c91975c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52832/0 (socket says 172.21.15.70:52832) 2023-10-18T12:28:35.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c919764090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a69d9b700 1 -- 172.21.15.70:0/797530625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c91975c5d0 msgr2=0x55c91975c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c91975c5d0 0x55c91975c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a69d9b700 1 -- 172.21.15.70:0/797530625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 msgr2=0x55c919764090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c919764090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a69d9b700 1 -- 172.21.15.70:0/797530625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9194c2c60 con 0x55c919761be0 2023-10-18T12:28:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c9197515d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1a60011ff0 tx=0x7f1a60009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800caf88017fa434 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a597fa700 1 -- 172.21.15.70:0/797530625 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60009ed0 con 0x55c919761be0 2023-10-18T12:28:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a597fa700 1 -- 172.21.15.70:0/797530625 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a600097d0 con 0x55c919761be0 2023-10-18T12:28:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.070+0000 7f1a597fa700 1 -- 172.21.15.70:0/797530625 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a600280e0 con 0x55c919761be0 2023-10-18T12:28:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 -- 172.21.15.70:0/797530625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 msgr2=0x55c9197515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c9197515d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1a60011ff0 tx=0x7f1a60009260 comp rx=0 tx=0).stop 2023-10-18T12:28:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 -- 172.21.15.70:0/797530625 shutdown_connections 2023-10-18T12:28:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c919762500 0x55c919764090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c91975c5d0 0x55c91975c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/797530625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c9197515d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 -- 172.21.15.70:0/797530625 >> 172.21.15.70:0/797530625 conn(0x55c9196d0720 msgr2=0x55c91975f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.071+0000 7f1a846f8b80 1 -- 172.21.15.70:0/797530625 shutdown_connections 2023-10-18T12:28:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.072+0000 7f1a846f8b80 1 -- 172.21.15.70:0/797530625 wait complete. 2023-10-18T12:28:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.072+0000 7f1a846f8b80 1 -- start start 2023-10-18T12:28:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.072+0000 7f1a846f8b80 1 Processor -- start 2023-10-18T12:28:35.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c91975c5d0 0x55c91975ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c91975ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c91975d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91975dbe0 con 0x55c91975c5d0 2023-10-18T12:28:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91975dd50 con 0x55c919761be0 2023-10-18T12:28:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91975dec0 con 0x55c919762500 2023-10-18T12:28:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a846f8b80 10 client.0 _ll_register_callbacks cb 0x55c919595a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c91975c5d0 0x55c91975ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c91975ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c91975ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43404/0 (socket says 172.21.15.70:43404) 2023-10-18T12:28:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 -- 172.21.15.70:0/2759638278 learned_addr learned my addr 172.21.15.70:0/2759638278 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c91975d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 -- 172.21.15.70:0/2759638278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 msgr2=0x55c91975d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c91975d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 -- 172.21.15.70:0/2759638278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c91975c5d0 msgr2=0x55c91975ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c91975c5d0 0x55c91975ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a69d9b700 1 -- 172.21.15.70:0/2759638278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c91966c3f0 con 0x55c919761be0 2023-10-18T12:28:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.073+0000 7f1a6ad9d700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c91975d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a69d9b700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c91975ef90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1a60012ac0 tx=0x7f1a60005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b491d7ced40547 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a51ffb700 1 -- 172.21.15.70:0/2759638278 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60012650 con 0x55c919761be0 2023-10-18T12:28:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9196a7e50 con 0x55c919761be0 2023-10-18T12:28:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a51ffb700 1 -- 172.21.15.70:0/2759638278 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1a60012b20 con 0x55c919761be0 2023-10-18T12:28:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a51ffb700 1 -- 172.21.15.70:0/2759638278 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a600127b0 con 0x55c919761be0 2023-10-18T12:28:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a846f8b80 10 client.25940 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.074+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c919500250 con 0x55c919761be0 2023-10-18T12:28:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a51ffb700 1 -- 172.21.15.70:0/2759638278 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9196a7e50 con 0x55c919761be0 2023-10-18T12:28:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a51ffb700 1 -- 172.21.15.70:0/2759638278 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1a60028430 con 0x55c919761be0 2023-10-18T12:28:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a846f8b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a846f8b80 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a846f8b80 20 client.25940 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9197d2ea0 0x55c9197d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a846f8b80 4 client.25940 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:28:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9196a7b70 con 0x55c9197d2ea0 2023-10-18T12:28:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.075+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9197d2ea0 0x55c9197d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a51ffb700 10 client.25940 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a6a59c700 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9197d2ea0 0x55c9197d52c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d1088285c8fe51 server_cookie=eb3869385c2a173f in_seq=0 out_seq=0 2023-10-18T12:28:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a51ffb700 1 -- 172.21.15.70:0/2759638278 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9196a7b70 con 0x55c9197d2ea0 2023-10-18T12:28:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a51ffb700 10 client.25940 handle_command_reply: tid=0 2023-10-18T12:28:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 client.25940 shutdown 2023-10-18T12:28:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 msgr2=0x55c91975ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c91975ef90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1a60012ac0 tx=0x7f1a60005d50 comp rx=0 tx=0).stop 2023-10-18T12:28:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 shutdown_connections 2023-10-18T12:28:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9197d2ea0 0x55c9197d52c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c919762500 0x55c91975d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c91975c5d0 0x55c91975ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 --2- 172.21.15.70:0/2759638278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c919761be0 0x55c91975ef90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 >> 172.21.15.70:0/2759638278 conn(0x55c9196d0720 msgr2=0x55c919763fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a51ffb700 0 client.25940 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 shutdown_connections 2023-10-18T12:28:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 1 -- 172.21.15.70:0/2759638278 wait complete. 2023-10-18T12:28:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.076+0000 7f1a846f8b80 20 client.25940 trim_cache size 0 max 16384 2023-10-18T12:28:35.096 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:28:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1a540a15f0 msgr2=0x7f1a540a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1a540a15f0 0x7f1a540a3ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009ec0 tx=0x7f1a64009920 comp rx=0 tx=0).stop 2023-10-18T12:28:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 msgr2=0x7f1a6c19ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 0x7f1a6c19ea70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000c00 tx=0x7f1a5c005740 comp rx=0 tx=0).stop 2023-10-18T12:28:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 shutdown_connections 2023-10-18T12:28:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c106730 0x7f1a6c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c100720 0x7f1a6c19ea70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1a540a15f0 0x7f1a540a3ab0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 --2- 172.21.15.70:0/1791515405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a6c1010e0 0x7f1a6c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 >> 172.21.15.70:0/1791515405 conn(0x7f1a6c0fc1b0 msgr2=0x7f1a6c10be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 shutdown_connections 2023-10-18T12:28:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:35.077+0000 7f1a70dff700 1 -- 172.21.15.70:0/1791515405 wait complete. 2023-10-18T12:28:35.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:35 smithi070 ceph-mon[98639]: pgmap v391: 129 pgs: 129 active+clean; 96 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2023-10-18T12:28:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1434108751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:28:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:35 smithi186 ceph-mon[135253]: pgmap v391: 129 pgs: 129 active+clean; 96 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2023-10-18T12:28:35.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1434108751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:28:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:35 smithi152 ceph-mon[134767]: pgmap v391: 129 pgs: 129 active+clean; 96 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2023-10-18T12:28:35.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1434108751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:28:35.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 170595 14.66 MB/sec warmup 64 sec latency 13.824 ms 2023-10-18T12:28:36.101 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:28:36.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:28:36.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 -- 172.21.15.70:0/1964232647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4106720 msgr2=0x7f52e4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 --2- 172.21.15.70:0/1964232647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4106720 0x7f52e4106b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009990 tx=0x7f52d4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 -- 172.21.15.70:0/1964232647 shutdown_connections 2023-10-18T12:28:36.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 --2- 172.21.15.70:0/1964232647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e41010d0 0x7f52e410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 --2- 172.21.15.70:0/1964232647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4106720 0x7f52e4106b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 --2- 172.21.15.70:0/1964232647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e4100710 0x7f52e4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 -- 172.21.15.70:0/1964232647 >> 172.21.15.70:0/1964232647 conn(0x7f52e40fc200 msgr2=0x7f52e40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 -- 172.21.15.70:0/1964232647 shutdown_connections 2023-10-18T12:28:36.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.329+0000 7f52e99ff700 1 -- 172.21.15.70:0/1964232647 wait complete. 2023-10-18T12:28:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.330+0000 7f52e99ff700 1 Processor -- start 2023-10-18T12:28:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 -- start start 2023-10-18T12:28:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 0x7f52e4198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e41010d0 0x7f52e4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e4106720 0x7f52e419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e419cb30 con 0x7f52e4106720 2023-10-18T12:28:36.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e419cca0 con 0x7f52e4100710 2023-10-18T12:28:36.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e419ce60 con 0x7f52e41010d0 2023-10-18T12:28:36.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 0x7f52e4198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 0x7f52e4198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43418/0 (socket says 172.21.15.70:43418) 2023-10-18T12:28:36.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e2ffd700 1 -- 172.21.15.70:0/4263772991 learned_addr learned my addr 172.21.15.70:0/4263772991 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e27fc700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e41010d0 0x7f52e4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.331+0000 7f52e37fe700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e4106720 0x7f52e419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e2ffd700 1 -- 172.21.15.70:0/4263772991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e41010d0 msgr2=0x7f52e4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e41010d0 0x7f52e4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e2ffd700 1 -- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e4106720 msgr2=0x7f52e419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e4106720 0x7f52e419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e2ffd700 1 -- 172.21.15.70:0/4263772991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40095f0 con 0x7f52e4100710 2023-10-18T12:28:36.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e37fe700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e4106720 0x7f52e419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:36.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 0x7f52e4198d00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e89fd700 1 -- 172.21.15.70:0/4263772991 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d401d070 con 0x7f52e4100710 2023-10-18T12:28:36.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e89fd700 1 -- 172.21.15.70:0/4263772991 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d4022470 con 0x7f52e4100710 2023-10-18T12:28:36.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e419d0e0 con 0x7f52e4100710 2023-10-18T12:28:36.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e89fd700 1 -- 172.21.15.70:0/4263772991 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4017650 con 0x7f52e4100710 2023-10-18T12:28:36.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.332+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e41a5aa0 con 0x7f52e4100710 2023-10-18T12:28:36.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.335+0000 7f52e89fd700 1 -- 172.21.15.70:0/4263772991 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f52d40225e0 con 0x7f52e4100710 2023-10-18T12:28:36.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 Processor -- start 2023-10-18T12:28:36.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 -- start start 2023-10-18T12:28:36.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52e89fd700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52cc0a15f0 0x7f52cc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 0x5591ea86d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea867110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52e89fd700 1 -- 172.21.15.70:0/4263772991 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52d40ce780 con 0x7f52e4100710 2023-10-18T12:28:36.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea867ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea7da760 con 0x5591ea86d580 2023-10-18T12:28:36.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea7c0ce0 con 0x5591ea86cc60 2023-10-18T12:28:36.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.337+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea7be600 con 0x5591ea867650 2023-10-18T12:28:36.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e37fe700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52cc0a15f0 0x7f52cc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 0x5591ea86d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 0x5591ea86d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43428/0 (socket says 172.21.15.70:43428) 2023-10-18T12:28:36.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 -- 172.21.15.70:0/2104848579 learned_addr learned my addr 172.21.15.70:0/2104848579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e37fe700 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea867ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e27fc700 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea867110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 -- 172.21.15.70:0/2104848579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 msgr2=0x5591ea867ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea867ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 -- 172.21.15.70:0/2104848579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 msgr2=0x5591ea867110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea867110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 -- 172.21.15.70:0/2104848579 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ea777470 con 0x5591ea86cc60 2023-10-18T12:28:36.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e27fc700 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea867110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:28:36.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 0x5591ea86d040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52d4097f00 tx=0x7f52d4097900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1f92961f23d0e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52e37fe700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52cc0a15f0 0x7f52cc0a3ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0094f0 tx=0x7f52dc009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.338+0000 7f52c9ffb700 1 -- 172.21.15.70:0/2104848579 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d409b490 con 0x5591ea86cc60 2023-10-18T12:28:36.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52c9ffb700 1 -- 172.21.15.70:0/2104848579 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d409f5e0 con 0x5591ea86cc60 2023-10-18T12:28:36.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52c9ffb700 1 -- 172.21.15.70:0/2104848579 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400fce0 con 0x5591ea86cc60 2023-10-18T12:28:36.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 -- 172.21.15.70:0/2104848579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 msgr2=0x5591ea86d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 0x5591ea86d040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52d4097f00 tx=0x7f52d4097900 comp rx=0 tx=0).stop 2023-10-18T12:28:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 -- 172.21.15.70:0/2104848579 shutdown_connections 2023-10-18T12:28:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea867ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86cc60 0x5591ea86d040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/2104848579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea867110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 -- 172.21.15.70:0/2104848579 >> 172.21.15.70:0/2104848579 conn(0x5591ea7db7a0 msgr2=0x5591ea861be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.339+0000 7f52fd38cb80 1 -- 172.21.15.70:0/2104848579 shutdown_connections 2023-10-18T12:28:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.340+0000 7f52fd38cb80 1 -- 172.21.15.70:0/2104848579 wait complete. 2023-10-18T12:28:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.340+0000 7f52fd38cb80 1 -- start start 2023-10-18T12:28:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.340+0000 7f52fd38cb80 1 Processor -- start 2023-10-18T12:28:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea873530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea870820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea870e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea7b9ef0 con 0x5591ea86d580 2023-10-18T12:28:36.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea756790 con 0x5591ea867650 2023-10-18T12:28:36.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea756430 con 0x5591ea86cc60 2023-10-18T12:28:36.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52fd38cb80 10 client.0 _ll_register_callbacks cb 0x5591ea6a0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea873530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea870e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea873530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43442/0 (socket says 172.21.15.70:43442) 2023-10-18T12:28:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 -- 172.21.15.70:0/849102818 learned_addr learned my addr 172.21.15.70:0/849102818 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e27fc700 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea870820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 -- 172.21.15.70:0/849102818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 msgr2=0x5591ea870820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea870820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 -- 172.21.15.70:0/849102818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 msgr2=0x5591ea870e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea870e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.341+0000 7f52e2ffd700 1 -- 172.21.15.70:0/849102818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e41a5aa0 con 0x5591ea867650 2023-10-18T12:28:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea873530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52d40a3260 tx=0x7f52d4022b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d40f96f1488ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52c2ffd700 1 -- 172.21.15.70:0/849102818 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40a3790 con 0x5591ea867650 2023-10-18T12:28:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52c2ffd700 1 -- 172.21.15.70:0/849102818 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52d400fa80 con 0x5591ea867650 2023-10-18T12:28:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591ea7bef70 con 0x5591ea867650 2023-10-18T12:28:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52c2ffd700 1 -- 172.21.15.70:0/849102818 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d400fc40 con 0x5591ea867650 2023-10-18T12:28:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52fd38cb80 10 client.25952 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5591ea5e4e60 con 0x5591ea867650 2023-10-18T12:28:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52c2ffd700 1 -- 172.21.15.70:0/849102818 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591ea7bef70 con 0x5591ea867650 2023-10-18T12:28:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.342+0000 7f52c2ffd700 1 -- 172.21.15.70:0/849102818 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52d409a980 con 0x5591ea867650 2023-10-18T12:28:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52fd38cb80 10 client.25952 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52fd38cb80 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52fd38cb80 20 client.25952 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea880640 0x5591ea882a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52fd38cb80 4 client.25952 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591ea836930 con 0x5591ea880640 2023-10-18T12:28:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52e27fc700 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea880640 0x5591ea882a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52e27fc700 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea880640 0x5591ea882a60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1f8facfda67313 server_cookie=f694b4bf5f7e0cec in_seq=0 out_seq=0 2023-10-18T12:28:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.343+0000 7f52c2ffd700 10 client.25952 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52c2ffd700 1 -- 172.21.15.70:0/849102818 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5591ea836930 con 0x5591ea880640 2023-10-18T12:28:36.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52c2ffd700 10 client.25952 handle_command_reply: tid=0 2023-10-18T12:28:36.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 client.25952 shutdown 2023-10-18T12:28:36.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 msgr2=0x5591ea873530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea873530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52d40a3260 tx=0x7f52d4022b90 comp rx=0 tx=0).stop 2023-10-18T12:28:36.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 shutdown_connections 2023-10-18T12:28:36.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea880640 0x5591ea882a60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea870e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea873530 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/849102818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea870820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52c2ffd700 0 client.25952 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.345+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 >> 172.21.15.70:0/849102818 conn(0x5591ea7db7a0 msgr2=0x5591ea866100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.346+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 shutdown_connections 2023-10-18T12:28:36.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.346+0000 7f52fd38cb80 1 -- 172.21.15.70:0/849102818 wait complete. 2023-10-18T12:28:36.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.346+0000 7f52fd38cb80 20 client.25952 trim_cache size 0 max 16384 2023-10-18T12:28:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 Processor -- start 2023-10-18T12:28:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 -- start start 2023-10-18T12:28:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea8707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86cc60 0x5591ea877cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86d580 0x5591ea86a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea8680f0 con 0x5591ea86cc60 2023-10-18T12:28:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea63a9a0 con 0x5591ea86d580 2023-10-18T12:28:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.366+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea863430 con 0x5591ea867650 2023-10-18T12:28:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea8707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86d580 0x5591ea86a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea8707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52890/0 (socket says 172.21.15.70:52890) 2023-10-18T12:28:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 -- 172.21.15.70:0/1703225444 learned_addr learned my addr 172.21.15.70:0/1703225444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e27fc700 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86cc60 0x5591ea877cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 -- 172.21.15.70:0/1703225444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86d580 msgr2=0x5591ea86a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86d580 0x5591ea86a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 -- 172.21.15.70:0/1703225444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86cc60 msgr2=0x5591ea877cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86cc60 0x5591ea877cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 -- 172.21.15.70:0/1703225444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ea5cdce0 con 0x5591ea867650 2023-10-18T12:28:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.367+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea8707c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52d4097f00 tx=0x7f52d4097900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4a0f19c7235e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52ca7fc700 1 -- 172.21.15.70:0/1703225444 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d409afd0 con 0x5591ea867650 2023-10-18T12:28:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52ca7fc700 1 -- 172.21.15.70:0/1703225444 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d40a3790 con 0x5591ea867650 2023-10-18T12:28:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52ca7fc700 1 -- 172.21.15.70:0/1703225444 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400fa80 con 0x5591ea867650 2023-10-18T12:28:36.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 -- 172.21.15.70:0/1703225444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 msgr2=0x5591ea8707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea8707c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52d4097f00 tx=0x7f52d4097900 comp rx=0 tx=0).stop 2023-10-18T12:28:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 -- 172.21.15.70:0/1703225444 shutdown_connections 2023-10-18T12:28:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea86d580 0x5591ea86a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea867650 0x5591ea8707c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/1703225444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86cc60 0x5591ea877cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 -- 172.21.15.70:0/1703225444 >> 172.21.15.70:0/1703225444 conn(0x5591ea7db7a0 msgr2=0x5591ea8700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 -- 172.21.15.70:0/1703225444 shutdown_connections 2023-10-18T12:28:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.368+0000 7f52fd38cb80 1 -- 172.21.15.70:0/1703225444 wait complete. 2023-10-18T12:28:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.369+0000 7f52fd38cb80 1 -- start start 2023-10-18T12:28:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.369+0000 7f52fd38cb80 1 Processor -- start 2023-10-18T12:28:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea8ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea8cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea876d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea877350 con 0x5591ea86d580 2023-10-18T12:28:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea8774c0 con 0x5591ea867650 2023-10-18T12:28:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ea877680 con 0x5591ea86cc60 2023-10-18T12:28:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52fd38cb80 10 client.0 _ll_register_callbacks cb 0x5591ea6a0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea8ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e27fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea8cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea8ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43454/0 (socket says 172.21.15.70:43454) 2023-10-18T12:28:36.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 -- 172.21.15.70:0/156207929 learned_addr learned my addr 172.21.15.70:0/156207929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e37fe700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea876d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 -- 172.21.15.70:0/156207929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 msgr2=0x5591ea8cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea8cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 -- 172.21.15.70:0/156207929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 msgr2=0x5591ea876d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.370+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea876d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52e2ffd700 1 -- 172.21.15.70:0/156207929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591ea5e4e60 con 0x5591ea867650 2023-10-18T12:28:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52e27fc700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea8cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52e2ffd700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea8ce8b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098e0 tx=0x7f52d40cbc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937d78f8572e842a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52c2ffd700 1 -- 172.21.15.70:0/156207929 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d409f360 con 0x5591ea867650 2023-10-18T12:28:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52c2ffd700 1 -- 172.21.15.70:0/156207929 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52d409b890 con 0x5591ea867650 2023-10-18T12:28:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52d409f5e0 con 0x5591ea867650 2023-10-18T12:28:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52c2ffd700 1 -- 172.21.15.70:0/156207929 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d409b6f0 con 0x5591ea867650 2023-10-18T12:28:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52fd38cb80 10 client.25958 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5591ea60b2d0 con 0x5591ea867650 2023-10-18T12:28:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.371+0000 7f52c2ffd700 1 -- 172.21.15.70:0/156207929 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d409f5e0 con 0x5591ea867650 2023-10-18T12:28:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52c2ffd700 1 -- 172.21.15.70:0/156207929 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52d40ad070 con 0x5591ea867650 2023-10-18T12:28:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52fd38cb80 10 client.25958 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52fd38cb80 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52fd38cb80 20 client.25958 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea8d62b0 0x5591ea8d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52fd38cb80 4 client.25958 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:28:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5591ea7b2ed0 con 0x5591ea8d62b0 2023-10-18T12:28:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52e27fc700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea8d62b0 0x5591ea8d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52e27fc700 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea8d62b0 0x5591ea8d86a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27110f6005cd3d1f server_cookie=80a5f4221be84755 in_seq=0 out_seq=0 2023-10-18T12:28:36.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.372+0000 7f52c2ffd700 10 client.25958 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.374+0000 7f52c2ffd700 1 -- 172.21.15.70:0/156207929 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5591ea7b2ed0 con 0x5591ea8d62b0 2023-10-18T12:28:36.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.374+0000 7f52c2ffd700 10 client.25958 handle_command_reply: tid=0 2023-10-18T12:28:36.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.374+0000 7f52fd38cb80 1 client.25958 shutdown 2023-10-18T12:28:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 msgr2=0x5591ea8ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea8ce8b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098e0 tx=0x7f52d40cbc50 comp rx=0 tx=0).stop 2023-10-18T12:28:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 shutdown_connections 2023-10-18T12:28:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5591ea8d62b0 0x5591ea8d86a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591ea86d580 0x5591ea876d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591ea867650 0x5591ea8ce8b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52c2ffd700 0 client.25958 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 --2- 172.21.15.70:0/156207929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591ea86cc60 0x5591ea8cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 >> 172.21.15.70:0/156207929 conn(0x5591ea7db7a0 msgr2=0x5591ea86a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 shutdown_connections 2023-10-18T12:28:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 1 -- 172.21.15.70:0/156207929 wait complete. 2023-10-18T12:28:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.375+0000 7f52fd38cb80 20 client.25958 trim_cache size 0 max 16384 2023-10-18T12:28:36.391 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:28:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:28:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "f48df54e-b2cb-484e-99a0-74ef55462955", 2023-10-18T12:28:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:28:36.392 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:28:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52cc0a15f0 msgr2=0x7f52cc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52cc0a15f0 0x7f52cc0a3ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0094f0 tx=0x7f52dc009920 comp rx=0 tx=0).stop 2023-10-18T12:28:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 msgr2=0x7f52e4198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 0x7f52e4198d00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4004730 comp rx=0 tx=0).stop 2023-10-18T12:28:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 shutdown_connections 2023-10-18T12:28:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e4106720 0x7f52e419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52cc0a15f0 0x7f52cc0a3ab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e4100710 0x7f52e4198d00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 --2- 172.21.15.70:0/4263772991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e41010d0 0x7f52e4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 >> 172.21.15.70:0/4263772991 conn(0x7f52e40fc200 msgr2=0x7f52e40fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 shutdown_connections 2023-10-18T12:28:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.376+0000 7f52e99ff700 1 -- 172.21.15.70:0/4263772991 wait complete. 2023-10-18T12:28:36.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:28:36.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 -- 172.21.15.70:0/1989830710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105170 msgr2=0x7f48901055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 --2- 172.21.15.70:0/1989830710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105170 0x7f48901055f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f48840099c0 tx=0x7f4884009cd0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 -- 172.21.15.70:0/1989830710 shutdown_connections 2023-10-18T12:28:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 --2- 172.21.15.70:0/1989830710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105b30 0x7f489010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 --2- 172.21.15.70:0/1989830710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 0x7f4890068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 --2- 172.21.15.70:0/1989830710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105170 0x7f48901055f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 -- 172.21.15.70:0/1989830710 >> 172.21.15.70:0/1989830710 conn(0x7f48900767f0 msgr2=0x7f4890076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 -- 172.21.15.70:0/1989830710 shutdown_connections 2023-10-18T12:28:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.613+0000 7f4895bff700 1 -- 172.21.15.70:0/1989830710 wait complete. 2023-10-18T12:28:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.614+0000 7f4895bff700 1 Processor -- start 2023-10-18T12:28:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.614+0000 7f4895bff700 1 -- start start 2023-10-18T12:28:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f4895bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 0x7f489019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f4895bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 0x7f489019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f4895bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105b30 0x7f4890196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f4895bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890197090 con 0x7f4890105b30 2023-10-18T12:28:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f4895bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890197200 con 0x7f4890067d40 2023-10-18T12:28:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f4895bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901973c0 con 0x7f4890105170 2023-10-18T12:28:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f488effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 0x7f489019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f488effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 0x7f489019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52918/0 (socket says 172.21.15.70:52918) 2023-10-18T12:28:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f488effd700 1 -- 172.21.15.70:0/2501060861 learned_addr learned my addr 172.21.15.70:0/2501060861 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f488ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105b30 0x7f4890196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.615+0000 7f488f7fe700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 0x7f489019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488effd700 1 -- 172.21.15.70:0/2501060861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 msgr2=0x7f489019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488effd700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 0x7f489019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488effd700 1 -- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105b30 msgr2=0x7f4890196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488effd700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105b30 0x7f4890196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488effd700 1 -- 172.21.15.70:0/2501060861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840095f0 con 0x7f4890105170 2023-10-18T12:28:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488f7fe700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 0x7f489019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:28:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488effd700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 0x7f489019ce40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f488400b580 tx=0x7f488400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488cff9700 1 -- 172.21.15.70:0/2501060861 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488401c070 con 0x7f4890105170 2023-10-18T12:28:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4890197640 con 0x7f4890105170 2023-10-18T12:28:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488cff9700 1 -- 172.21.15.70:0/2501060861 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f488400fdf0 con 0x7f4890105170 2023-10-18T12:28:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f488cff9700 1 -- 172.21.15.70:0/2501060861 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884022c40 con 0x7f4890105170 2023-10-18T12:28:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.616+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48901a38f0 con 0x7f4890105170 2023-10-18T12:28:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.621+0000 7f488cff9700 1 -- 172.21.15.70:0/2501060861 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4884020070 con 0x7f4890105170 2023-10-18T12:28:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 Processor -- start 2023-10-18T12:28:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 -- start start 2023-10-18T12:28:36.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f488cff9700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48780a15f0 0x7f48780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2934266e0 con 0x55b2934b79e0 2023-10-18T12:28:36.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29340cc60 con 0x55b2934bbee0 2023-10-18T12:28:36.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f48a9591b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29340a580 con 0x55b2934bc800 2023-10-18T12:28:36.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f488cff9700 1 -- 172.21.15.70:0/2501060861 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f48840ce1e0 con 0x7f4890105170 2023-10-18T12:28:36.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f488effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f488effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52932/0 (socket says 172.21.15.70:52932) 2023-10-18T12:28:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.622+0000 7f488effd700 1 -- 172.21.15.70:0/3782747752 learned_addr learned my addr 172.21.15.70:0/3782747752 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488f7fe700 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488ffff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48780a15f0 0x7f48780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488effd700 1 -- 172.21.15.70:0/3782747752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 msgr2=0x55b2934bc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488effd700 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934bc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488ffff700 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488effd700 1 -- 172.21.15.70:0/3782747752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 msgr2=0x55b2934b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488effd700 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488effd700 1 -- 172.21.15.70:0/3782747752 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2933c33f0 con 0x55b2934bc800 2023-10-18T12:28:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488ffff700 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:28:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f488effd700 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b74a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f48840973f0 tx=0x7f488409aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4d5ec9c3e28bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f4875ffb700 1 -- 172.21.15.70:0/3782747752 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48840a1110 con 0x55b2934bc800 2023-10-18T12:28:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f4875ffb700 1 -- 172.21.15.70:0/3782747752 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48840669f0 con 0x55b2934bc800 2023-10-18T12:28:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.623+0000 7f4875ffb700 1 -- 172.21.15.70:0/3782747752 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48840a3280 con 0x55b2934bc800 2023-10-18T12:28:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 -- 172.21.15.70:0/3782747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 msgr2=0x55b2934b74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f488ffff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48780a15f0 0x7f48780a3ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55b2934aea30 tx=0x7f4888009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b74a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f48840973f0 tx=0x7f488409aa30 comp rx=0 tx=0).stop 2023-10-18T12:28:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 -- 172.21.15.70:0/3782747752 shutdown_connections 2023-10-18T12:28:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934bc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3782747752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b74a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 -- 172.21.15.70:0/3782747752 >> 172.21.15.70:0/3782747752 conn(0x55b293427720 msgr2=0x55b2934adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 -- 172.21.15.70:0/3782747752 shutdown_connections 2023-10-18T12:28:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.624+0000 7f48a9591b80 1 -- 172.21.15.70:0/3782747752 wait complete. 2023-10-18T12:28:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.625+0000 7f48a9591b80 1 -- start start 2023-10-18T12:28:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.625+0000 7f48a9591b80 1 Processor -- start 2023-10-18T12:28:36.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934b79e0 0x55b2934c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934bc800 0x55b2934c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293405e70 con 0x55b2934bc800 2023-10-18T12:28:36.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2933a2710 con 0x55b2934bbee0 2023-10-18T12:28:36.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2933a23b0 con 0x55b2934b79e0 2023-10-18T12:28:36.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f48a9591b80 10 client.0 _ll_register_callbacks cb 0x55b2932eca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:36.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f488effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f488effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43482/0 (socket says 172.21.15.70:43482) 2023-10-18T12:28:36.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f488effd700 1 -- 172.21.15.70:0/1906224753 learned_addr learned my addr 172.21.15.70:0/1906224753 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.626+0000 7f488f7fe700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934b79e0 0x55b2934c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488ffff700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934bc800 0x55b2934c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488effd700 1 -- 172.21.15.70:0/1906224753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934b79e0 msgr2=0x55b2934c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488effd700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934b79e0 0x55b2934c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488effd700 1 -- 172.21.15.70:0/1906224753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934bc800 msgr2=0x55b2934c1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488effd700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934bc800 0x55b2934c1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488effd700 1 -- 172.21.15.70:0/1906224753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48901a38f0 con 0x55b2934bbee0 2023-10-18T12:28:36.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488f7fe700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934b79e0 0x55b2934c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:36.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f488effd700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934c0af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f488409ae80 tx=0x7f4884005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7671827c52972f3a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f486effd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:28:36.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f486effd700 1 -- 172.21.15.70:0/1906224753 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488409aef0 con 0x55b2934bbee0 2023-10-18T12:28:36.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f486effd700 1 -- 172.21.15.70:0/1906224753 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f488409f040 con 0x55b2934bbee0 2023-10-18T12:28:36.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b29340aef0 con 0x55b2934bbee0 2023-10-18T12:28:36.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f486effd700 1 -- 172.21.15.70:0/1906224753 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48840a31f0 con 0x55b2934bbee0 2023-10-18T12:28:36.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f48a9591b80 10 client.25970 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:36.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.627+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b293230de0 con 0x55b2934bbee0 2023-10-18T12:28:36.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f486effd700 1 -- 172.21.15.70:0/1906224753 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b29340aef0 con 0x55b2934bbee0 2023-10-18T12:28:36.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f486effd700 1 -- 172.21.15.70:0/1906224753 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48840cbd20 con 0x55b2934bbee0 2023-10-18T12:28:36.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f48a9591b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:36.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f48a9591b80 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:36.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f48a9591b80 20 client.25970 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:36.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f48a9591b80 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b2934b6ae0 0x55b2934b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f48a9591b80 4 client.25970 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:28:36.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2934828b0 con 0x55b2934b6ae0 2023-10-18T12:28:36.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.628+0000 7f488f7fe700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b2934b6ae0 0x55b2934b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.629+0000 7f486effd700 10 client.25970 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.629+0000 7f488f7fe700 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b2934b6ae0 0x55b2934b6ec0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4630076fddb18352 server_cookie=53baa220b4edad9 in_seq=0 out_seq=0 2023-10-18T12:28:36.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.629+0000 7f486effd700 1 -- 172.21.15.70:0/1906224753 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b2934828b0 con 0x55b2934b6ae0 2023-10-18T12:28:36.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.629+0000 7f486effd700 10 client.25970 handle_command_reply: tid=0 2023-10-18T12:28:36.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.629+0000 7f48a9591b80 1 client.25970 shutdown 2023-10-18T12:28:36.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 msgr2=0x55b2934c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934c0af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f488409ae80 tx=0x7f4884005c90 comp rx=0 tx=0).stop 2023-10-18T12:28:36.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 shutdown_connections 2023-10-18T12:28:36.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b2934b6ae0 0x55b2934b6ec0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934bc800 0x55b2934c1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934b79e0 0x55b2934c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 --2- 172.21.15.70:0/1906224753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934c0af0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f486effd700 0 client.25970 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 >> 172.21.15.70:0/1906224753 conn(0x55b293427720 msgr2=0x55b2934adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 shutdown_connections 2023-10-18T12:28:36.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 1 -- 172.21.15.70:0/1906224753 wait complete. 2023-10-18T12:28:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.630+0000 7f48a9591b80 20 client.25970 trim_cache size 0 max 16384 2023-10-18T12:28:36.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.647+0000 7f48a9591b80 1 Processor -- start 2023-10-18T12:28:36.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.647+0000 7f48a9591b80 1 -- start start 2023-10-18T12:28:36.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bbee0 0x55b2934bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bc800 0x55b2934bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f48a9591b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293286920 con 0x55b2934b79e0 2023-10-18T12:28:36.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f48a9591b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2934828b0 con 0x55b2934bc800 2023-10-18T12:28:36.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f48a9591b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29340aef0 con 0x55b2934bbee0 2023-10-18T12:28:36.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39334/0 (socket says 172.21.15.70:39334) 2023-10-18T12:28:36.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 -- 172.21.15.70:0/2385122929 learned_addr learned my addr 172.21.15.70:0/2385122929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488effd700 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bbee0 0x55b2934bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488ffff700 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bc800 0x55b2934bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 -- 172.21.15.70:0/2385122929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bbee0 msgr2=0x55b2934bf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bbee0 0x55b2934bf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 -- 172.21.15.70:0/2385122929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bc800 msgr2=0x55b2934bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bc800 0x55b2934bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.648+0000 7f488f7fe700 1 -- 172.21.15.70:0/2385122929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b293219c60 con 0x55b2934b79e0 2023-10-18T12:28:36.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.649+0000 7f488f7fe700 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934c3430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4880000c00 tx=0x7f48800043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e3bb7dd4c9249e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.649+0000 7f48767fc700 1 -- 172.21.15.70:0/2385122929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880017040 con 0x55b2934b79e0 2023-10-18T12:28:36.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.649+0000 7f48767fc700 1 -- 172.21.15.70:0/2385122929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4880019040 con 0x55b2934b79e0 2023-10-18T12:28:36.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.649+0000 7f48767fc700 1 -- 172.21.15.70:0/2385122929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48800038e0 con 0x55b2934b79e0 2023-10-18T12:28:36.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 -- 172.21.15.70:0/2385122929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 msgr2=0x55b2934c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934c3430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4880000c00 tx=0x7f48800043d0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 -- 172.21.15.70:0/2385122929 shutdown_connections 2023-10-18T12:28:36.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bc800 0x55b2934bf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934c3430 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 --2- 172.21.15.70:0/2385122929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bbee0 0x55b2934bf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 -- 172.21.15.70:0/2385122929 >> 172.21.15.70:0/2385122929 conn(0x55b293427720 msgr2=0x55b2934b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 -- 172.21.15.70:0/2385122929 shutdown_connections 2023-10-18T12:28:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 -- 172.21.15.70:0/2385122929 wait complete. 2023-10-18T12:28:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 -- start start 2023-10-18T12:28:36.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.650+0000 7f48a9591b80 1 Processor -- start 2023-10-18T12:28:36.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293528ca0 con 0x55b2934b79e0 2023-10-18T12:28:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293528e10 con 0x55b2934bbee0 2023-10-18T12:28:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293528fd0 con 0x55b2934bc800 2023-10-18T12:28:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f48a9591b80 10 client.0 _ll_register_callbacks cb 0x55b2932eca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:28:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f488ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f488effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f488ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:52968/0 (socket says 172.21.15.70:52968) 2023-10-18T12:28:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f488ffff700 1 -- 172.21.15.70:0/3620127305 learned_addr learned my addr 172.21.15.70:0/3620127305 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:28:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.651+0000 7f488effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934b9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43486/0 (socket says 172.21.15.70:43486) 2023-10-18T12:28:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488ffff700 1 -- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 msgr2=0x55b2934b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488ffff700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934b9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488ffff700 1 -- 172.21.15.70:0/3620127305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 msgr2=0x55b2934b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:28:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488ffff700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488ffff700 1 -- 172.21.15.70:0/3620127305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4880009e20 con 0x55b2934bc800 2023-10-18T12:28:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488f7fe700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488effd700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:28:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f488ffff700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b9d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4888017e50 tx=0x7f4888004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964865847ce28b7f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:28:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f486effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:28:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f486effd700 1 -- 172.21.15.70:0/3620127305 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800e040 con 0x55b2934bc800 2023-10-18T12:28:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48840669f0 con 0x55b2934bc800 2023-10-18T12:28:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f486effd700 1 -- 172.21.15.70:0/3620127305 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4888008dd0 con 0x55b2934bc800 2023-10-18T12:28:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f486effd700 1 -- 172.21.15.70:0/3620127305 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4888008f90 con 0x55b2934bc800 2023-10-18T12:28:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f48a9591b80 10 client.25972 fetch_fsmap learned FSMap version 20 2023-10-18T12:28:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.652+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b293257250 con 0x55b2934bc800 2023-10-18T12:28:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f486effd700 1 -- 172.21.15.70:0/3620127305 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48840669f0 con 0x55b2934bc800 2023-10-18T12:28:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f486effd700 1 -- 172.21.15.70:0/3620127305 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4888024d20 con 0x55b2934bc800 2023-10-18T12:28:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f48a9591b80 10 client.25972 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:28:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f48a9591b80 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:28:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f48a9591b80 20 client.25972 populate_metadata read hostname 'smithi070' 2023-10-18T12:28:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b29352a8f0 0x55b29352cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:28:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f48a9591b80 4 client.25972 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:28:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2933fee50 con 0x55b29352a8f0 2023-10-18T12:28:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.653+0000 7f488f7fe700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b29352a8f0 0x55b29352cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:28:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f488f7fe700 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b29352a8f0 0x55b29352cd10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21ff61c1830e01e server_cookie=58802733a91c5269 in_seq=0 out_seq=0 2023-10-18T12:28:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f486effd700 10 client.25972 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f486effd700 1 -- 172.21.15.70:0/3620127305 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b2933fee50 con 0x55b29352a8f0 2023-10-18T12:28:36.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f486effd700 10 client.25972 handle_command_reply: tid=0 2023-10-18T12:28:36.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 client.25972 shutdown 2023-10-18T12:28:36.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 msgr2=0x55b2934b9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b9d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4888017e50 tx=0x7f4888004ad0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 shutdown_connections 2023-10-18T12:28:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b29352a8f0 0x55b29352cd10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2934bc800 0x55b2934b9d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b2934b79e0 0x55b2934b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f486effd700 0 client.25972 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:28:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 --2- 172.21.15.70:0/3620127305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2934bbee0 0x55b2934b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.654+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 >> 172.21.15.70:0/3620127305 conn(0x55b293427720 msgr2=0x55b2934bf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 shutdown_connections 2023-10-18T12:28:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f48a9591b80 1 -- 172.21.15.70:0/3620127305 wait complete. 2023-10-18T12:28:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f48a9591b80 20 client.25972 trim_cache size 0 max 16384 2023-10-18T12:28:36.673 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:28:36.673 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:28:36.673 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:28:36.674 INFO:teuthology.orchestra.run.smithi070.stdout: "f48df54e-b2cb-484e-99a0-74ef55462955": { 2023-10-18T12:28:36.674 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:28:36.674 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "f48df54e-b2cb-484e-99a0-74ef55462955", 2023-10-18T12:28:36.674 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:28:36.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:28:36.675 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:28:36.675 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:28:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48780a15f0 msgr2=0x7f48780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48780a15f0 0x7f48780a3ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55b2934aea30 tx=0x7f4888009920 comp rx=0 tx=0).stop 2023-10-18T12:28:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 msgr2=0x7f489019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:28:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 0x7f489019ce40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f488400b580 tx=0x7f488400bfd0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 shutdown_connections 2023-10-18T12:28:36.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4890105b30 0x7f4890196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48780a15f0 0x7f48780a3ab0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4890067d40 0x7f489019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 --2- 172.21.15.70:0/2501060861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4890105170 0x7f489019ce40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:28:36.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.655+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 >> 172.21.15.70:0/2501060861 conn(0x7f48900767f0 msgr2=0x7f48900fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:28:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.656+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 shutdown_connections 2023-10-18T12:28:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:28:36.656+0000 7f4895bff700 1 -- 172.21.15.70:0/2501060861 wait complete. 2023-10-18T12:28:36.679 INFO:tasks.cephfs.filesystem:scrub status for tag:f48df54e-b2cb-484e-99a0-74ef55462955 - {'path': '/', 'tag': 'f48df54e-b2cb-484e-99a0-74ef55462955', 'options': 'recursive,force'} 2023-10-18T12:28:36.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 173448 14.65 MB/sec warmup 65 sec latency 7.944 ms 2023-10-18T12:28:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:37 smithi186 ceph-mon[135253]: pgmap v392: 129 pgs: 129 active+clean; 96 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2023-10-18T12:28:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:37 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:28:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:37 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:37.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:37 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:28:37.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:37 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:37 smithi070 ceph-mon[98639]: pgmap v392: 129 pgs: 129 active+clean; 96 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2023-10-18T12:28:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:37 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:28:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:37 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:37 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:28:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:37 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:37 smithi152 ceph-mon[134767]: pgmap v392: 129 pgs: 129 active+clean; 96 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2023-10-18T12:28:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:37 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:28:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:37 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:37 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:28:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:37 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:28:37.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 175800 14.60 MB/sec warmup 66 sec latency 14.280 ms 2023-10-18T12:28:38.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:38 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:28:38.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:38 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:28:38.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:38 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:28:38.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 177868 14.58 MB/sec warmup 67 sec latency 97.077 ms 2023-10-18T12:28:39.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:39 smithi070 ceph-mon[98639]: pgmap v393: 129 pgs: 129 active+clean; 99 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 283 op/s 2023-10-18T12:28:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:39 smithi152 ceph-mon[134767]: pgmap v393: 129 pgs: 129 active+clean; 99 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 283 op/s 2023-10-18T12:28:39.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:39 smithi186 ceph-mon[135253]: pgmap v393: 129 pgs: 129 active+clean; 99 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 283 op/s 2023-10-18T12:28:39.837 INFO:tasks.workunit.client.0.smithi070.stdout: 1 180900 14.63 MB/sec warmup 68 sec latency 21.951 ms 2023-10-18T12:28:40.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 183355 14.54 MB/sec warmup 69 sec latency 16.266 ms 2023-10-18T12:28:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:41 smithi070 ceph-mon[98639]: pgmap v394: 129 pgs: 129 active+clean; 102 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2023-10-18T12:28:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:41 smithi152 ceph-mon[134767]: pgmap v394: 129 pgs: 129 active+clean; 102 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2023-10-18T12:28:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:41 smithi186 ceph-mon[135253]: pgmap v394: 129 pgs: 129 active+clean; 102 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2023-10-18T12:28:41.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 185592 14.53 MB/sec warmup 70 sec latency 11.862 ms 2023-10-18T12:28:42.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 188599 14.57 MB/sec warmup 71 sec latency 6.558 ms 2023-10-18T12:28:43.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:43 smithi070 ceph-mon[98639]: pgmap v395: 129 pgs: 129 active+clean; 104 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2023-10-18T12:28:43.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:43 smithi152 ceph-mon[134767]: pgmap v395: 129 pgs: 129 active+clean; 104 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2023-10-18T12:28:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:43 smithi186 ceph-mon[135253]: pgmap v395: 129 pgs: 129 active+clean; 104 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2023-10-18T12:28:43.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 191215 14.55 MB/sec warmup 72 sec latency 68.619 ms 2023-10-18T12:28:44.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 193389 14.51 MB/sec warmup 73 sec latency 21.885 ms 2023-10-18T12:28:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:45 smithi070 ceph-mon[98639]: pgmap v396: 129 pgs: 129 active+clean; 105 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2023-10-18T12:28:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:45 smithi152 ceph-mon[134767]: pgmap v396: 129 pgs: 129 active+clean; 105 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2023-10-18T12:28:45.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:45 smithi186 ceph-mon[135253]: pgmap v396: 129 pgs: 129 active+clean; 105 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2023-10-18T12:28:45.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 195836 14.50 MB/sec warmup 74 sec latency 22.693 ms 2023-10-18T12:28:46.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 198966 14.55 MB/sec warmup 75 sec latency 9.231 ms 2023-10-18T12:28:47.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:47 smithi070 ceph-mon[98639]: pgmap v397: 129 pgs: 129 active+clean; 106 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2023-10-18T12:28:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:47 smithi152 ceph-mon[134767]: pgmap v397: 129 pgs: 129 active+clean; 106 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2023-10-18T12:28:47.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:47 smithi186 ceph-mon[135253]: pgmap v397: 129 pgs: 129 active+clean; 106 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2023-10-18T12:28:47.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 201651 14.53 MB/sec warmup 76 sec latency 18.753 ms 2023-10-18T12:28:48.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 203278 14.45 MB/sec warmup 77 sec latency 60.897 ms 2023-10-18T12:28:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:49 smithi070 ceph-mon[98639]: pgmap v398: 129 pgs: 129 active+clean; 110 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2023-10-18T12:28:49.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:49 smithi152 ceph-mon[134767]: pgmap v398: 129 pgs: 129 active+clean; 110 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2023-10-18T12:28:49.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:49 smithi186 ceph-mon[135253]: pgmap v398: 129 pgs: 129 active+clean; 110 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2023-10-18T12:28:49.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 206168 14.49 MB/sec warmup 78 sec latency 19.164 ms 2023-10-18T12:28:50.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 209256 14.52 MB/sec warmup 79 sec latency 19.262 ms 2023-10-18T12:28:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:51 smithi070 ceph-mon[98639]: pgmap v399: 129 pgs: 129 active+clean; 113 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 276 op/s 2023-10-18T12:28:51.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:51 smithi152 ceph-mon[134767]: pgmap v399: 129 pgs: 129 active+clean; 113 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 276 op/s 2023-10-18T12:28:51.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:51 smithi186 ceph-mon[135253]: pgmap v399: 129 pgs: 129 active+clean; 113 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 276 op/s 2023-10-18T12:28:51.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 211528 14.44 MB/sec warmup 80 sec latency 78.627 ms 2023-10-18T12:28:52.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 213799 14.43 MB/sec warmup 81 sec latency 18.547 ms 2023-10-18T12:28:53.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:53 smithi070 ceph-mon[98639]: pgmap v400: 129 pgs: 129 active+clean; 113 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2023-10-18T12:28:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:53 smithi152 ceph-mon[134767]: pgmap v400: 129 pgs: 129 active+clean; 113 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2023-10-18T12:28:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:53 smithi186 ceph-mon[135253]: pgmap v400: 129 pgs: 129 active+clean; 113 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2023-10-18T12:28:53.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 216741 14.48 MB/sec warmup 82 sec latency 51.380 ms 2023-10-18T12:28:54.838 INFO:tasks.workunit.client.0.smithi070.stdout: 1 219061 14.41 MB/sec warmup 83 sec latency 40.787 ms 2023-10-18T12:28:55.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:55 smithi186 ceph-mon[135253]: pgmap v401: 129 pgs: 129 active+clean; 114 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-10-18T12:28:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:55 smithi070 ceph-mon[98639]: pgmap v401: 129 pgs: 129 active+clean; 114 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-10-18T12:28:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:55 smithi152 ceph-mon[134767]: pgmap v401: 129 pgs: 129 active+clean; 114 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-10-18T12:28:55.839 INFO:tasks.workunit.client.0.smithi070.stdout: 1 221458 14.42 MB/sec warmup 84 sec latency 10.819 ms 2023-10-18T12:28:56.839 INFO:tasks.workunit.client.0.smithi070.stdout: 1 224039 14.42 MB/sec warmup 85 sec latency 53.155 ms 2023-10-18T12:28:57.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:57 smithi186 ceph-mon[135253]: pgmap v402: 129 pgs: 129 active+clean; 122 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 284 op/s 2023-10-18T12:28:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:57 smithi070 ceph-mon[98639]: pgmap v402: 129 pgs: 129 active+clean; 122 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 284 op/s 2023-10-18T12:28:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:57 smithi152 ceph-mon[134767]: pgmap v402: 129 pgs: 129 active+clean; 122 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 284 op/s 2023-10-18T12:28:57.839 INFO:tasks.workunit.client.0.smithi070.stdout: 1 227267 14.47 MB/sec warmup 86 sec latency 6.988 ms 2023-10-18T12:28:58.840 INFO:tasks.workunit.client.0.smithi070.stdout: 1 229617 14.40 MB/sec warmup 87 sec latency 26.649 ms 2023-10-18T12:28:59.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:28:59 smithi186 ceph-mon[135253]: pgmap v403: 129 pgs: 129 active+clean; 122 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 267 op/s 2023-10-18T12:28:59.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:28:59 smithi070 ceph-mon[98639]: pgmap v403: 129 pgs: 129 active+clean; 122 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 267 op/s 2023-10-18T12:28:59.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:28:59 smithi152 ceph-mon[134767]: pgmap v403: 129 pgs: 129 active+clean; 122 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 267 op/s 2023-10-18T12:28:59.840 INFO:tasks.workunit.client.0.smithi070.stdout: 1 231427 14.37 MB/sec warmup 88 sec latency 57.314 ms 2023-10-18T12:29:00.840 INFO:tasks.workunit.client.0.smithi070.stdout: 1 234522 14.42 MB/sec warmup 89 sec latency 13.310 ms 2023-10-18T12:29:01.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:01 smithi186 ceph-mon[135253]: pgmap v404: 129 pgs: 129 active+clean; 124 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 279 op/s 2023-10-18T12:29:01.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:01 smithi070 ceph-mon[98639]: pgmap v404: 129 pgs: 129 active+clean; 124 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 279 op/s 2023-10-18T12:29:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:01 smithi152 ceph-mon[134767]: pgmap v404: 129 pgs: 129 active+clean; 124 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 279 op/s 2023-10-18T12:29:01.840 INFO:tasks.workunit.client.0.smithi070.stdout: 1 235178 14.27 MB/sec warmup 90 sec latency 71.957 ms 2023-10-18T12:29:02.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 237905 14.31 MB/sec warmup 91 sec latency 60.304 ms 2023-10-18T12:29:03.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:03 smithi186 ceph-mon[135253]: pgmap v405: 129 pgs: 129 active+clean; 124 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2023-10-18T12:29:03.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:03 smithi070 ceph-mon[98639]: pgmap v405: 129 pgs: 129 active+clean; 124 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2023-10-18T12:29:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:03 smithi152 ceph-mon[134767]: pgmap v405: 129 pgs: 129 active+clean; 124 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2023-10-18T12:29:03.840 INFO:tasks.workunit.client.0.smithi070.stdout: 1 240303 14.24 MB/sec warmup 92 sec latency 17.446 ms 2023-10-18T12:29:04.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 241955 14.21 MB/sec warmup 93 sec latency 128.680 ms 2023-10-18T12:29:05.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:05 smithi186 ceph-mon[135253]: pgmap v406: 129 pgs: 129 active+clean; 129 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 308 op/s 2023-10-18T12:29:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:05 smithi070 ceph-mon[98639]: pgmap v406: 129 pgs: 129 active+clean; 129 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 308 op/s 2023-10-18T12:29:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:05 smithi152 ceph-mon[134767]: pgmap v406: 129 pgs: 129 active+clean; 129 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 308 op/s 2023-10-18T12:29:05.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 245095 14.26 MB/sec warmup 94 sec latency 7.181 ms 2023-10-18T12:29:06.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:29:06.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 247778 14.24 MB/sec warmup 95 sec latency 17.661 ms 2023-10-18T12:29:06.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/168880196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 msgr2=0x7f6c98102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/168880196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98102dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009990 tx=0x7f6c8c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/168880196 shutdown_connections 2023-10-18T12:29:06.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/168880196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c98103310 0x7f6c9810f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/168880196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c98108960 0x7f6c98108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/168880196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98102dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/168880196 >> 172.21.15.70:0/168880196 conn(0x7f6c980fe400 msgr2=0x7f6c98100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:06.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/168880196 shutdown_connections 2023-10-18T12:29:06.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.896+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/168880196 wait complete. 2023-10-18T12:29:06.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 Processor -- start 2023-10-18T12:29:06.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 -- start start 2023-10-18T12:29:06.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c98103310 0x7f6c98199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c98108960 0x7f6c9819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9819dcb0 con 0x7f6c98108960 2023-10-18T12:29:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9819de20 con 0x7f6c98102950 2023-10-18T12:29:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.897+0000 7f6c9f1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9819dfe0 con 0x7f6c98103310 2023-10-18T12:29:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58010/0 (socket says 172.21.15.70:58010) 2023-10-18T12:29:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/3611295559 learned_addr learned my addr 172.21.15.70:0/3611295559 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9d79c700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c98108960 0x7f6c9819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c97fff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c98103310 0x7f6c98199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/3611295559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c98103310 msgr2=0x7f6c98199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c98103310 0x7f6c98199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c98108960 msgr2=0x7f6c9819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c98108960 0x7f6c9819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/3611295559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0095f0 con 0x7f6c98102950 2023-10-18T12:29:06.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98198de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800e9a0 tx=0x7f6c8800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:06.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.898+0000 7f6c95ffb700 1 -- 172.21.15.70:0/3611295559 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88009790 con 0x7f6c98102950 2023-10-18T12:29:06.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.899+0000 7f6c95ffb700 1 -- 172.21.15.70:0/3611295559 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c88010430 con 0x7f6c98102950 2023-10-18T12:29:06.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.899+0000 7f6c95ffb700 1 -- 172.21.15.70:0/3611295559 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88018680 con 0x7f6c98102950 2023-10-18T12:29:06.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.899+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9819e2c0 con 0x7f6c98102950 2023-10-18T12:29:06.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.899+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c981a5b00 con 0x7f6c98102950 2023-10-18T12:29:06.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.902+0000 7f6c95ffb700 1 -- 172.21.15.70:0/3611295559 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6c880105a0 con 0x7f6c98102950 2023-10-18T12:29:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.903+0000 7f6c95ffb700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c800a1670 0x7f6c800a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.903+0000 7f6c95ffb700 1 -- 172.21.15.70:0/3611295559 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6c88014070 con 0x7f6c98102950 2023-10-18T12:29:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.903+0000 7f6cb2b1cb80 1 Processor -- start 2023-10-18T12:29:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.903+0000 7f6c97fff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c800a1670 0x7f6c800a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.903+0000 7f6cb2b1cb80 1 -- start start 2023-10-18T12:29:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a564ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a564a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a564aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a55bd6e0 con 0x5629a564a5d0 2023-10-18T12:29:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a55a3c60 con 0x5629a564fbe0 2023-10-18T12:29:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a55a1580 con 0x5629a5650500 2023-10-18T12:29:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a564ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a564ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58014/0 (socket says 172.21.15.70:58014) 2023-10-18T12:29:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/1872343876 learned_addr learned my addr 172.21.15.70:0/1872343876 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c9d79c700 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a564aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c97fff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c800a1670 0x7f6c800a3b30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c9819a2e0 tx=0x7f6c8c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c97fff700 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a564a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/1872343876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 msgr2=0x5629a564a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.904+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a564a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/1872343876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 msgr2=0x5629a564aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a564aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/1872343876 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629a555a3f0 con 0x5629a564fbe0 2023-10-18T12:29:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c97fff700 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a564a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:29:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a564ffc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c88096c20 tx=0x7f6c88099c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd54c4f3bb892f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c7effd700 1 -- 172.21.15.70:0/1872343876 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8809e120 con 0x5629a564fbe0 2023-10-18T12:29:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c7effd700 1 -- 172.21.15.70:0/1872343876 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c8809e300 con 0x5629a564fbe0 2023-10-18T12:29:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.905+0000 7f6c7effd700 1 -- 172.21.15.70:0/1872343876 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8809a670 con 0x5629a564fbe0 2023-10-18T12:29:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1872343876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 msgr2=0x5629a564ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a564ffc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c88096c20 tx=0x7f6c88099c00 comp rx=0 tx=0).stop 2023-10-18T12:29:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1872343876 shutdown_connections 2023-10-18T12:29:06.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a564aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a564ffc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1872343876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a564a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1872343876 >> 172.21.15.70:0/1872343876 conn(0x5629a55be720 msgr2=0x5629a5644b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:06.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1872343876 shutdown_connections 2023-10-18T12:29:06.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1872343876 wait complete. 2023-10-18T12:29:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.906+0000 7f6cb2b1cb80 1 -- start start 2023-10-18T12:29:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 Processor -- start 2023-10-18T12:29:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a5656590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a564fbe0 0x5629a5653810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a5650500 0x5629a5653f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a559ce70 con 0x5629a564a5d0 2023-10-18T12:29:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5539710 con 0x5629a5650500 2023-10-18T12:29:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a55393b0 con 0x5629a564fbe0 2023-10-18T12:29:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6cb2b1cb80 10 client.0 _ll_register_callbacks cb 0x5629a5483a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.907+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a5656590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9d79c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a5650500 0x5629a5653f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a5656590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39090/0 (socket says 172.21.15.70:39090) 2023-10-18T12:29:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/4094930770 learned_addr learned my addr 172.21.15.70:0/4094930770 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c97fff700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a564fbe0 0x5629a5653810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/4094930770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a564fbe0 msgr2=0x5629a5653810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a564fbe0 0x5629a5653810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a5650500 msgr2=0x5629a5653f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a5650500 0x5629a5653f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/4094930770 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c981a5b00 con 0x5629a564a5d0 2023-10-18T12:29:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.908+0000 7f6c9d79c700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a5650500 0x5629a5653f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:29:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a5656590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c880caea0 tx=0x7f6c8809e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fe373d7bc23294 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6c77fff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:29:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6c77fff700 1 -- 172.21.15.70:0/4094930770 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880998b0 con 0x5629a564a5d0 2023-10-18T12:29:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6c77fff700 1 -- 172.21.15.70:0/4094930770 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6c880b82c0 con 0x5629a564a5d0 2023-10-18T12:29:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629a55a1ef0 con 0x5629a564a5d0 2023-10-18T12:29:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6c77fff700 1 -- 172.21.15.70:0/4094930770 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c880b8480 con 0x5629a564a5d0 2023-10-18T12:29:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6cb2b1cb80 10 client.15966 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5629a53c7de0 con 0x5629a564a5d0 2023-10-18T12:29:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.909+0000 7f6c77fff700 1 -- 172.21.15.70:0/4094930770 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629a55a1ef0 con 0x5629a564a5d0 2023-10-18T12:29:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6c77fff700 1 -- 172.21.15.70:0/4094930770 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c88096030 con 0x5629a564a5d0 2023-10-18T12:29:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6cb2b1cb80 10 client.15966 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6cb2b1cb80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6cb2b1cb80 20 client.15966 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a5663950 0x5629a5665d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6cb2b1cb80 4 client.15966 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:06.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629a56198b0 con 0x5629a5663950 2023-10-18T12:29:06.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.910+0000 7f6c97fff700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a5663950 0x5629a5665d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.911+0000 7f6c97fff700 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a5663950 0x5629a5665d40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73312553f25084af server_cookie=2972a0e82379e8e0 in_seq=0 out_seq=0 2023-10-18T12:29:06.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.911+0000 7f6c77fff700 10 client.15966 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:06.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.911+0000 7f6c77fff700 1 -- 172.21.15.70:0/4094930770 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5629a56198b0 con 0x5629a5663950 2023-10-18T12:29:06.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.911+0000 7f6c77fff700 10 client.15966 handle_command_reply: tid=0 2023-10-18T12:29:06.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 client.15966 shutdown 2023-10-18T12:29:06.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 msgr2=0x5629a5656590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a5656590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c880caea0 tx=0x7f6c8809e120 comp rx=0 tx=0).stop 2023-10-18T12:29:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 shutdown_connections 2023-10-18T12:29:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a5663950 0x5629a5665d40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a5650500 0x5629a5653f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a5656590 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/4094930770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a564fbe0 0x5629a5653810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 >> 172.21.15.70:0/4094930770 conn(0x5629a55be720 msgr2=0x5629a5649080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:06.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6c77fff700 0 client.15966 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:06.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 shutdown_connections 2023-10-18T12:29:06.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/4094930770 wait complete. 2023-10-18T12:29:06.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.912+0000 7f6cb2b1cb80 20 client.15966 trim_cache size 0 max 16384 2023-10-18T12:29:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.932+0000 7f6cb2b1cb80 1 Processor -- start 2023-10-18T12:29:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.932+0000 7f6cb2b1cb80 1 -- start start 2023-10-18T12:29:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a565aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a565f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5645130 con 0x5629a564a5d0 2023-10-18T12:29:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a563faf0 con 0x5629a564fbe0 2023-10-18T12:29:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a541d920 con 0x5629a5650500 2023-10-18T12:29:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a565aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a565f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c9d79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a565aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39098/0 (socket says 172.21.15.70:39098) 2023-10-18T12:29:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a565f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58038/0 (socket says 172.21.15.70:58038) 2023-10-18T12:29:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c9cf9b700 1 -- 172.21.15.70:0/1860896172 learned_addr learned my addr 172.21.15.70:0/1860896172 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 -- 172.21.15.70:0/1860896172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 msgr2=0x5629a56463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 -- 172.21.15.70:0/1860896172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 msgr2=0x5629a565aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a565aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.933+0000 7f6c97fff700 1 -- 172.21.15.70:0/1860896172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629a53b0c60 con 0x5629a564fbe0 2023-10-18T12:29:06.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.934+0000 7f6c97fff700 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a565f600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b2e0 tx=0x7f6c8c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ccd0e05e17a7be server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:06.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.934+0000 7f6c7f7fe700 1 -- 172.21.15.70:0/1860896172 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c032270 con 0x5629a564fbe0 2023-10-18T12:29:06.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.934+0000 7f6c7f7fe700 1 -- 172.21.15.70:0/1860896172 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c8c02a040 con 0x5629a564fbe0 2023-10-18T12:29:06.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.934+0000 7f6c7f7fe700 1 -- 172.21.15.70:0/1860896172 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c004760 con 0x5629a564fbe0 2023-10-18T12:29:06.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1860896172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 msgr2=0x5629a565f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a565f600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b2e0 tx=0x7f6c8c020000 comp rx=0 tx=0).stop 2023-10-18T12:29:06.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1860896172 shutdown_connections 2023-10-18T12:29:06.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564a5d0 0x5629a565aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/1860896172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564fbe0 0x5629a565f600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1860896172 >> 172.21.15.70:0/1860896172 conn(0x5629a55be720 msgr2=0x5629a5649990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:06.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1860896172 shutdown_connections 2023-10-18T12:29:06.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/1860896172 wait complete. 2023-10-18T12:29:06.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 -- start start 2023-10-18T12:29:06.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.935+0000 7f6cb2b1cb80 1 Processor -- start 2023-10-18T12:29:06.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564a5d0 0x5629a565a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 0x5629a565a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a56b6c30 con 0x5629a564fbe0 2023-10-18T12:29:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a56b6d70 con 0x5629a564a5d0 2023-10-18T12:29:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a56b6eb0 con 0x5629a5650500 2023-10-18T12:29:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6cb2b1cb80 10 client.0 _ll_register_callbacks cb 0x5629a5483a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 0x5629a565a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 0x5629a565a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39106/0 (socket says 172.21.15.70:39106) 2023-10-18T12:29:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c9cf9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564a5d0 0x5629a565a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c97fff700 1 -- 172.21.15.70:0/2800115581 learned_addr learned my addr 172.21.15.70:0/2800115581 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:06.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c9d79c700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c97fff700 1 -- 172.21.15.70:0/2800115581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 msgr2=0x5629a56b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.936+0000 7f6c97fff700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c97fff700 1 -- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564a5d0 msgr2=0x5629a565a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:06.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c97fff700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564a5d0 0x5629a565a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c97fff700 1 -- 172.21.15.70:0/2800115581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c98110120 con 0x5629a564fbe0 2023-10-18T12:29:06.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564a5d0 0x5629a565a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:06.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c97fff700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 0x5629a565a800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0098e0 tx=0x7f6c8c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c774871e4c7ec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:06.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c77fff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:29:06.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c77fff700 1 -- 172.21.15.70:0/2800115581 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c00bc80 con 0x5629a564fbe0 2023-10-18T12:29:06.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c77fff700 1 -- 172.21.15.70:0/2800115581 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6c8c004500 con 0x5629a564fbe0 2023-10-18T12:29:06.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c8809e300 con 0x5629a564fbe0 2023-10-18T12:29:06.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.937+0000 7f6c77fff700 1 -- 172.21.15.70:0/2800115581 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c8c00bde0 con 0x5629a564fbe0 2023-10-18T12:29:06.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 10 client.15972 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:06.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5629a53ee250 con 0x5629a564fbe0 2023-10-18T12:29:06.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6c77fff700 1 -- 172.21.15.70:0/2800115581 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c8809e300 con 0x5629a564fbe0 2023-10-18T12:29:06.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6c77fff700 1 -- 172.21.15.70:0/2800115581 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c8c00f760 con 0x5629a564fbe0 2023-10-18T12:29:06.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 10 client.15972 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:06.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:06.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 20 client.15972 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:06.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a56b82b0 0x5629a56ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:06.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 4 client.15972 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:29:06.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5629a5595e50 con 0x5629a56b82b0 2023-10-18T12:29:06.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.938+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a56b82b0 0x5629a56ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:06.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.939+0000 7f6c9cf9b700 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a56b82b0 0x5629a56ba6d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e120b11ccf536a server_cookie=6c276406eb943635 in_seq=0 out_seq=0 2023-10-18T12:29:06.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.939+0000 7f6c77fff700 10 client.15972 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:06.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6c77fff700 1 -- 172.21.15.70:0/2800115581 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5629a5595e50 con 0x5629a56b82b0 2023-10-18T12:29:06.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6c77fff700 10 client.15972 handle_command_reply: tid=0 2023-10-18T12:29:06.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 client.15972 shutdown 2023-10-18T12:29:06.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 msgr2=0x5629a565a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 0x5629a565a800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0098e0 tx=0x7f6c8c0058e0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 shutdown_connections 2023-10-18T12:29:06.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5629a56b82b0 0x5629a56ba6d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629a5650500 0x5629a56b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6c77fff700 0 client.15972 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:06.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629a564a5d0 0x5629a565a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 --2- 172.21.15.70:0/2800115581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5629a564fbe0 0x5629a565a800 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 >> 172.21.15.70:0/2800115581 conn(0x5629a55be720 msgr2=0x5629a565f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:06.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 shutdown_connections 2023-10-18T12:29:06.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 1 -- 172.21.15.70:0/2800115581 wait complete. 2023-10-18T12:29:06.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.941+0000 7f6cb2b1cb80 20 client.15972 trim_cache size 0 max 16384 2023-10-18T12:29:06.957 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:29:06.957 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:29:06.957 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:29:06.957 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:29:06.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c800a1670 msgr2=0x7f6c800a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c800a1670 0x7f6c800a3b30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c9819a2e0 tx=0x7f6c8c005fd0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 msgr2=0x7f6c98198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:06.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98198de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800e9a0 tx=0x7f6c8800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 shutdown_connections 2023-10-18T12:29:06.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c98108960 0x7f6c9819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6c800a1670 0x7f6c800a3b30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c98102950 0x7f6c98198de0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 --2- 172.21.15.70:0/3611295559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c98103310 0x7f6c98199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:06.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 >> 172.21.15.70:0/3611295559 conn(0x7f6c980fe400 msgr2=0x7f6c980fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:06.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 shutdown_connections 2023-10-18T12:29:06.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:06.942+0000 7f6c9f1ff700 1 -- 172.21.15.70:0/3611295559 wait complete. 2023-10-18T12:29:06.964 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:29:06.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:29:07.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/1609612641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a241116b0 msgr2=0x7f6a24113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/1609612641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a241116b0 0x7f6a24113aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00b0a0 tx=0x7f6a1c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/1609612641 shutdown_connections 2023-10-18T12:29:07.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/1609612641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a241116b0 0x7f6a24113aa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/1609612641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a24072b90 0x7f6a24072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/1609612641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a240734b0 0x7f6a24111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/1609612641 >> 172.21.15.70:0/1609612641 conn(0x7f6a240fc1d0 msgr2=0x7f6a240fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/1609612641 shutdown_connections 2023-10-18T12:29:07.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.176+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/1609612641 wait complete. 2023-10-18T12:29:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 Processor -- start 2023-10-18T12:29:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 -- start start 2023-10-18T12:29:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24072b90 0x7f6a2419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 0x7f6a2419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a241116b0 0x7f6a241a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241a22a0 con 0x7f6a241116b0 2023-10-18T12:29:07.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241a2410 con 0x7f6a240734b0 2023-10-18T12:29:07.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.177+0000 7f6a2a9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241a25d0 con 0x7f6a24072b90 2023-10-18T12:29:07.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 0x7f6a2419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a28f9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a241116b0 0x7f6a241a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 0x7f6a2419d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58068/0 (socket says 172.21.15.70:58068) 2023-10-18T12:29:07.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 -- 172.21.15.70:0/4260051854 learned_addr learned my addr 172.21.15.70:0/4260051854 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:07.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a23fff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24072b90 0x7f6a2419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 -- 172.21.15.70:0/4260051854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24072b90 msgr2=0x7f6a2419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24072b90 0x7f6a2419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 -- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a241116b0 msgr2=0x7f6a241a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a241116b0 0x7f6a241a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 -- 172.21.15.70:0/4260051854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c009d00 con 0x7f6a240734b0 2023-10-18T12:29:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a23fff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24072b90 0x7f6a2419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a237fe700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 0x7f6a2419d8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800e9a0 tx=0x7f6a1800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.178+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18009790 con 0x7f6a240734b0 2023-10-18T12:29:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.179+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a241a28b0 con 0x7f6a240734b0 2023-10-18T12:29:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.179+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a18010430 con 0x7f6a240734b0 2023-10-18T12:29:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.179+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18018680 con 0x7f6a240734b0 2023-10-18T12:29:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.179+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a241aa0f0 con 0x7f6a240734b0 2023-10-18T12:29:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.180+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a24103bd0 con 0x7f6a240734b0 2023-10-18T12:29:07.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.182+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6a180105a0 con 0x7f6a240734b0 2023-10-18T12:29:07.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.183+0000 7f6a217fa700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6a0c0a15f0 0x7f6a0c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.183+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6a18014070 con 0x7f6a240734b0 2023-10-18T12:29:07.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.183+0000 7f6a23fff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6a0c0a15f0 0x7f6a0c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.184+0000 7f6a23fff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6a0c0a15f0 0x7f6a0c0a3ab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6a14011490 tx=0x7f6a14009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.187+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a18096560 con 0x7f6a240734b0 2023-10-18T12:29:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:07 smithi070 ceph-mon[98639]: pgmap v407: 129 pgs: 129 active+clean; 129 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2023-10-18T12:29:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.484+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a2419e680 con 0x7f6a240734b0 2023-10-18T12:29:07.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.484+0000 7f6a217fa700 1 -- 172.21.15.70:0/4260051854 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f6a18099b80 con 0x7f6a240734b0 2023-10-18T12:29:07.487 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:29:07.488 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:29:07.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6a0c0a15f0 msgr2=0x7f6a0c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6a0c0a15f0 0x7f6a0c0a3ab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6a14011490 tx=0x7f6a14009260 comp rx=0 tx=0).stop 2023-10-18T12:29:07.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 msgr2=0x7f6a2419d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 0x7f6a2419d8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800e9a0 tx=0x7f6a1800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 shutdown_connections 2023-10-18T12:29:07.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a241116b0 0x7f6a241a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a24072b90 0x7f6a2419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6a0c0a15f0 0x7f6a0c0a3ab0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 --2- 172.21.15.70:0/4260051854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a240734b0 0x7f6a2419d8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 >> 172.21.15.70:0/4260051854 conn(0x7f6a240fc1d0 msgr2=0x7f6a240fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 shutdown_connections 2023-10-18T12:29:07.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.487+0000 7f6a2a9ff700 1 -- 172.21.15.70:0/4260051854 wait complete. 2023-10-18T12:29:07.493 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:29:07.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:29:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:07 smithi152 ceph-mon[134767]: pgmap v407: 129 pgs: 129 active+clean; 129 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2023-10-18T12:29:07.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:07 smithi186 ceph-mon[135253]: pgmap v407: 129 pgs: 129 active+clean; 129 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2023-10-18T12:29:07.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 -- 172.21.15.70:0/4233901477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a00734e0 msgr2=0x7f68a010ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 --2- 172.21.15.70:0/4233901477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a00734e0 0x7f68a010ca70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6894009990 tx=0x7f6894009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 -- 172.21.15.70:0/4233901477 shutdown_connections 2023-10-18T12:29:07.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 --2- 172.21.15.70:0/4233901477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a010cfb0 0x7f68a010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 --2- 172.21.15.70:0/4233901477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 --2- 172.21.15.70:0/4233901477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a00734e0 0x7f68a010ca70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 -- 172.21.15.70:0/4233901477 >> 172.21.15.70:0/4233901477 conn(0x7f68a00fc200 msgr2=0x7f68a00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 -- 172.21.15.70:0/4233901477 shutdown_connections 2023-10-18T12:29:07.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.730+0000 7f68a67ff700 1 -- 172.21.15.70:0/4233901477 wait complete. 2023-10-18T12:29:07.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.731+0000 7f68a67ff700 1 Processor -- start 2023-10-18T12:29:07.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.731+0000 7f68a67ff700 1 -- start start 2023-10-18T12:29:07.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a67ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a67ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 0x7f68a0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a67ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a010cfb0 0x7f68a019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a67ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a019dce0 con 0x7f68a00734e0 2023-10-18T12:29:07.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a67ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a019de50 con 0x7f68a010cfb0 2023-10-18T12:29:07.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a67ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a019e010 con 0x7f68a0072bc0 2023-10-18T12:29:07.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 0x7f68a0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 0x7f68a0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39136/0 (socket says 172.21.15.70:39136) 2023-10-18T12:29:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a4d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a010cfb0 0x7f68a019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:44388/0 (socket says 172.21.15.70:44388) 2023-10-18T12:29:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 -- 172.21.15.70:0/2966479823 learned_addr learned my addr 172.21.15.70:0/2966479823 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 -- 172.21.15.70:0/2966479823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 msgr2=0x7f68a0198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 -- 172.21.15.70:0/2966479823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a010cfb0 msgr2=0x7f68a019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a010cfb0 0x7f68a019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689f7fe700 1 -- 172.21.15.70:0/2966479823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890009520 con 0x7f68a00734e0 2023-10-18T12:29:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f689ffff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.732+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a010cfb0 0x7f68a019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:07.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.733+0000 7f689f7fe700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 0x7f68a0199300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6894000c00 tx=0x7f6894004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.733+0000 7f689d7fa700 1 -- 172.21.15.70:0/2966479823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689401d070 con 0x7f68a00734e0 2023-10-18T12:29:07.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.733+0000 7f689d7fa700 1 -- 172.21.15.70:0/2966479823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6894022470 con 0x7f68a00734e0 2023-10-18T12:29:07.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.733+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68940095f0 con 0x7f68a00734e0 2023-10-18T12:29:07.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.733+0000 7f689d7fa700 1 -- 172.21.15.70:0/2966479823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894017650 con 0x7f68a00734e0 2023-10-18T12:29:07.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.733+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a01a5950 con 0x7f68a00734e0 2023-10-18T12:29:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.736+0000 7f689d7fa700 1 -- 172.21.15.70:0/2966479823 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f68940225e0 con 0x7f68a00734e0 2023-10-18T12:29:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f689d7fa700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f68880a15f0 0x7f68880a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f689d7fa700 1 -- 172.21.15.70:0/2966479823 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f68940ce600 con 0x7f68a00734e0 2023-10-18T12:29:07.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 Processor -- start 2023-10-18T12:29:07.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f689ffff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f68880a15f0 0x7f68880a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 -- start start 2023-10-18T12:29:07.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4787fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4782090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4782a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b46f56e0 con 0x5579b4788500 2023-10-18T12:29:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b46dbc60 con 0x5579b47825d0 2023-10-18T12:29:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.738+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b46d9580 con 0x5579b4787be0 2023-10-18T12:29:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4782a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4782090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f689ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4787fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4782a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58094/0 (socket says 172.21.15.70:58094) 2023-10-18T12:29:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 -- 172.21.15.70:0/2311405725 learned_addr learned my addr 172.21.15.70:0/2311405725 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f689ffff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f68880a15f0 0x7f68880a3ab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6890009fd0 tx=0x7f6890009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 -- 172.21.15.70:0/2311405725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 msgr2=0x5579b4787fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4787fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 -- 172.21.15.70:0/2311405725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 msgr2=0x5579b4782090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4782090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 -- 172.21.15.70:0/2311405725 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579b46923f0 con 0x5579b47825d0 2023-10-18T12:29:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4782a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6898003e40 tx=0x7f6898009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc4c6caa29edb09 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68867fc700 1 -- 172.21.15.70:0/2311405725 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68980092d0 con 0x5579b47825d0 2023-10-18T12:29:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68867fc700 1 -- 172.21.15.70:0/2311405725 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689800f040 con 0x5579b47825d0 2023-10-18T12:29:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.739+0000 7f68867fc700 1 -- 172.21.15.70:0/2311405725 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800e7e0 con 0x5579b47825d0 2023-10-18T12:29:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/2311405725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 msgr2=0x5579b4782a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4782a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6898003e40 tx=0x7f6898009920 comp rx=0 tx=0).stop 2023-10-18T12:29:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/2311405725 shutdown_connections 2023-10-18T12:29:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4782a50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4787fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/2311405725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4782090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/2311405725 >> 172.21.15.70:0/2311405725 conn(0x5579b46f6720 msgr2=0x5579b477cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/2311405725 shutdown_connections 2023-10-18T12:29:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.740+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/2311405725 wait complete. 2023-10-18T12:29:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.741+0000 7f68ba1b8b80 1 -- start start 2023-10-18T12:29:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.741+0000 7f68ba1b8b80 1 Processor -- start 2023-10-18T12:29:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b478e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 0x5579b478b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4788500 0x5579b478bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b46d4e70 con 0x5579b4787be0 2023-10-18T12:29:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b4671710 con 0x5579b47825d0 2023-10-18T12:29:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b46713b0 con 0x5579b4788500 2023-10-18T12:29:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68ba1b8b80 10 client.0 _ll_register_callbacks cb 0x5579b45bba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 0x5579b478b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 0x5579b478b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39146/0 (socket says 172.21.15.70:39146) 2023-10-18T12:29:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 -- 172.21.15.70:0/3313399372 learned_addr learned my addr 172.21.15.70:0/3313399372 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689ffff700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b478e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4788500 0x5579b478bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 -- 172.21.15.70:0/3313399372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4788500 msgr2=0x5579b478bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4788500 0x5579b478bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 -- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 msgr2=0x5579b478e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b478e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 -- 172.21.15.70:0/3313399372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6898003b20 con 0x5579b4787be0 2023-10-18T12:29:07.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f689f7fe700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 0x5579b478b760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68940cba50 tx=0x7f689409ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6beb04dc77238 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.742+0000 7f687f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:29:07.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f687f7fe700 1 -- 172.21.15.70:0/3313399372 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940ce440 con 0x5579b4787be0 2023-10-18T12:29:07.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f687f7fe700 1 -- 172.21.15.70:0/3313399372 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f68940a3610 con 0x5579b4787be0 2023-10-18T12:29:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579b46d9ef0 con 0x5579b4787be0 2023-10-18T12:29:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f687f7fe700 1 -- 172.21.15.70:0/3313399372 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f689409b570 con 0x5579b4787be0 2023-10-18T12:29:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 10 client.15990 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5579b44ffde0 con 0x5579b4787be0 2023-10-18T12:29:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f687f7fe700 1 -- 172.21.15.70:0/3313399372 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579b46d9ef0 con 0x5579b4787be0 2023-10-18T12:29:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f687f7fe700 1 -- 172.21.15.70:0/3313399372 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f68940a3180 con 0x5579b4787be0 2023-10-18T12:29:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 20 client.15990 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b479bb20 0x5579b479df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 4 client.15990 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:07.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.743+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579b47518b0 con 0x5579b479bb20 2023-10-18T12:29:07.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.744+0000 7f689ffff700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b479bb20 0x5579b479df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.744+0000 7f689ffff700 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b479bb20 0x5579b479df10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431490e6ea2530 server_cookie=91d6b72307caff4 in_seq=0 out_seq=0 2023-10-18T12:29:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.744+0000 7f687f7fe700 10 client.15990 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f687f7fe700 1 -- 172.21.15.70:0/3313399372 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5579b47518b0 con 0x5579b479bb20 2023-10-18T12:29:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f687f7fe700 10 client.15990 handle_command_reply: tid=0 2023-10-18T12:29:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 client.15990 shutdown 2023-10-18T12:29:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 msgr2=0x5579b478b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 0x5579b478b760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68940cba50 tx=0x7f689409ba90 comp rx=0 tx=0).stop 2023-10-18T12:29:07.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 shutdown_connections 2023-10-18T12:29:07.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b479bb20 0x5579b479df10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4788500 0x5579b478bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b478e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3313399372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4787be0 0x5579b478b760 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f687f7fe700 0 client.15990 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:07.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 >> 172.21.15.70:0/3313399372 conn(0x5579b46f6720 msgr2=0x5579b4781080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 shutdown_connections 2023-10-18T12:29:07.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3313399372 wait complete. 2023-10-18T12:29:07.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.745+0000 7f68ba1b8b80 20 client.15990 trim_cache size 0 max 16384 2023-10-18T12:29:07.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.760+0000 7f68ba1b8b80 1 Processor -- start 2023-10-18T12:29:07.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.760+0000 7f68ba1b8b80 1 -- start start 2023-10-18T12:29:07.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 0x5579b4792eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4781930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b4788500 0x5579b478e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b477d9b0 con 0x5579b47825d0 2023-10-18T12:29:07.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b4555920 con 0x5579b4788500 2023-10-18T12:29:07.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b47933f0 con 0x5579b4787be0 2023-10-18T12:29:07.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4781930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 0x5579b4792eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4781930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:44420/0 (socket says 172.21.15.70:44420) 2023-10-18T12:29:07.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 0x5579b4792eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39162/0 (socket says 172.21.15.70:39162) 2023-10-18T12:29:07.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 -- 172.21.15.70:0/3809775141 learned_addr learned my addr 172.21.15.70:0/3809775141 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:07.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b4788500 0x5579b478e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 -- 172.21.15.70:0/3809775141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b4788500 msgr2=0x5579b478e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b4788500 0x5579b478e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 -- 172.21.15.70:0/3809775141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 msgr2=0x5579b4792eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.761+0000 7f689f7fe700 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 0x5579b4792eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.762+0000 7f689f7fe700 1 -- 172.21.15.70:0/3809775141 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579b44e8c60 con 0x5579b4787be0 2023-10-18T12:29:07.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.762+0000 7f689ffff700 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 0x5579b4792eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:07.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.762+0000 7f689f7fe700 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4781930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6894097d80 tx=0x7f6894009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf26d737c15f0184 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.762+0000 7f6886ffd700 1 -- 172.21.15.70:0/3809775141 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940971f0 con 0x5579b4787be0 2023-10-18T12:29:07.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.762+0000 7f6886ffd700 1 -- 172.21.15.70:0/3809775141 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68940cd070 con 0x5579b4787be0 2023-10-18T12:29:07.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.762+0000 7f6886ffd700 1 -- 172.21.15.70:0/3809775141 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689409a800 con 0x5579b4787be0 2023-10-18T12:29:07.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3809775141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 msgr2=0x5579b4781930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4781930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6894097d80 tx=0x7f6894009310 comp rx=0 tx=0).stop 2023-10-18T12:29:07.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3809775141 shutdown_connections 2023-10-18T12:29:07.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b4788500 0x5579b478e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b47825d0 0x5579b4792eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/3809775141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4781930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3809775141 >> 172.21.15.70:0/3809775141 conn(0x5579b46f6720 msgr2=0x5579b4781080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3809775141 shutdown_connections 2023-10-18T12:29:07.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/3809775141 wait complete. 2023-10-18T12:29:07.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 -- start start 2023-10-18T12:29:07.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.763+0000 7f68ba1b8b80 1 Processor -- start 2023-10-18T12:29:07.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4791ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4792420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4792960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b47efad0 con 0x5579b4788500 2023-10-18T12:29:07.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b47efc40 con 0x5579b47825d0 2023-10-18T12:29:07.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b47efe00 con 0x5579b4787be0 2023-10-18T12:29:07.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68ba1b8b80 10 client.0 _ll_register_callbacks cb 0x5579b4783040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:07.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4792420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4791ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4792420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:44430/0 (socket says 172.21.15.70:44430) 2023-10-18T12:29:07.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 -- 172.21.15.70:0/1752879207 learned_addr learned my addr 172.21.15.70:0/1752879207 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:07.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f68a4d9c700 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4792960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 -- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 msgr2=0x5579b4791ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4791ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 -- 172.21.15.70:0/1752879207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 msgr2=0x5579b4792960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:07.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.764+0000 7f689f7fe700 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4792960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f689f7fe700 1 -- 172.21.15.70:0/1752879207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890017050 con 0x5579b4787be0 2023-10-18T12:29:07.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f689f7fe700 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4792420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f68940a30e0 tx=0x7f689409ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0684337788be8bf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:07.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f687f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:29:07.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f687f7fe700 1 -- 172.21.15.70:0/1752879207 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894084070 con 0x5579b4787be0 2023-10-18T12:29:07.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f689800f040 con 0x5579b4787be0 2023-10-18T12:29:07.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f687f7fe700 1 -- 172.21.15.70:0/1752879207 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6894097450 con 0x5579b4787be0 2023-10-18T12:29:07.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f687f7fe700 1 -- 172.21.15.70:0/1752879207 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68940971f0 con 0x5579b4787be0 2023-10-18T12:29:07.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f68ba1b8b80 10 client.26011 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:07.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5579b44ffde0 con 0x5579b4787be0 2023-10-18T12:29:07.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f687f7fe700 1 -- 172.21.15.70:0/1752879207 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689800f040 con 0x5579b4787be0 2023-10-18T12:29:07.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f687f7fe700 1 -- 172.21.15.70:0/1752879207 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f689409b570 con 0x5579b4787be0 2023-10-18T12:29:07.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f68ba1b8b80 10 client.26011 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:07.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f68ba1b8b80 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:07.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.765+0000 7f68ba1b8b80 20 client.26011 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:07.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b47f10d0 0x5579b47f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:07.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f68ba1b8b80 4 client.26011 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:29:07.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5579b46cde50 con 0x5579b47f10d0 2023-10-18T12:29:07.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f689ffff700 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b47f10d0 0x5579b47f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:07.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f689ffff700 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b47f10d0 0x5579b47f34c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df062f13a1e44d0a server_cookie=6885150b503e69a2 in_seq=0 out_seq=0 2023-10-18T12:29:07.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f687f7fe700 10 client.26011 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:07.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f687f7fe700 1 -- 172.21.15.70:0/1752879207 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5579b46cde50 con 0x5579b47f10d0 2023-10-18T12:29:07.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f687f7fe700 10 client.26011 handle_command_reply: tid=0 2023-10-18T12:29:07.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.766+0000 7f68ba1b8b80 1 client.26011 shutdown 2023-10-18T12:29:07.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 msgr2=0x5579b4792420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4792420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f68940a30e0 tx=0x7f689409ba90 comp rx=0 tx=0).stop 2023-10-18T12:29:07.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 shutdown_connections 2023-10-18T12:29:07.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5579b47f10d0 0x5579b47f34c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5579b4788500 0x5579b4792960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f687f7fe700 0 client.26011 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:07.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5579b47825d0 0x5579b4791ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 --2- 172.21.15.70:0/1752879207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5579b4787be0 0x5579b4792420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 >> 172.21.15.70:0/1752879207 conn(0x5579b46f6720 msgr2=0x5579b4785640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 shutdown_connections 2023-10-18T12:29:07.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 1 -- 172.21.15.70:0/1752879207 wait complete. 2023-10-18T12:29:07.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.767+0000 7f68ba1b8b80 20 client.26011 trim_cache size 0 max 16384 2023-10-18T12:29:07.786 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:29:07.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f68880a15f0 msgr2=0x7f68880a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f68880a15f0 0x7f68880a3ab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6890009fd0 tx=0x7f6890009260 comp rx=0 tx=0).stop 2023-10-18T12:29:07.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 msgr2=0x7f68a0199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:07.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 0x7f68a0199300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6894000c00 tx=0x7f6894004730 comp rx=0 tx=0).stop 2023-10-18T12:29:07.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 shutdown_connections 2023-10-18T12:29:07.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f68880a15f0 0x7f68880a3ab0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68a010cfb0 0x7f68a019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a0072bc0 0x7f68a0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 --2- 172.21.15.70:0/2966479823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00734e0 0x7f68a0199300 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:07.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 >> 172.21.15.70:0/2966479823 conn(0x7f68a00fc200 msgr2=0x7f68a00fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:07.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 shutdown_connections 2023-10-18T12:29:07.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:07.768+0000 7f68a67ff700 1 -- 172.21.15.70:0/2966479823 wait complete. 2023-10-18T12:29:07.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 250420 14.22 MB/sec warmup 96 sec latency 10.235 ms 2023-10-18T12:29:08.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:08 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4260051854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:29:08.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:08 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4260051854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:29:08.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:08 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4260051854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:29:08.791 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:29:08.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:29:08.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 252671 14.22 MB/sec warmup 97 sec latency 7.095 ms 2023-10-18T12:29:09.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 -- 172.21.15.70:0/3584410586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 msgr2=0x7f189410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 --2- 172.21.15.70:0/3584410586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 0x7f189410ca20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1888009990 tx=0x7f1888009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 -- 172.21.15.70:0/3584410586 shutdown_connections 2023-10-18T12:29:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 --2- 172.21.15.70:0/3584410586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f189410cf60 0x7f189410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 --2- 172.21.15.70:0/3584410586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 --2- 172.21.15.70:0/3584410586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 0x7f189410ca20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 -- 172.21.15.70:0/3584410586 >> 172.21.15.70:0/3584410586 conn(0x7f18940fc1b0 msgr2=0x7f18940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 -- 172.21.15.70:0/3584410586 shutdown_connections 2023-10-18T12:29:09.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.008+0000 7f18995ff700 1 -- 172.21.15.70:0/3584410586 wait complete. 2023-10-18T12:29:09.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 Processor -- start 2023-10-18T12:29:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 -- start start 2023-10-18T12:29:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 0x7f1894199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f189410cf60 0x7f189419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419dd40 con 0x7f1894073540 2023-10-18T12:29:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419deb0 con 0x7f189410cf60 2023-10-18T12:29:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.009+0000 7f18995ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419e070 con 0x7f1894072c20 2023-10-18T12:29:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f18937fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f189410cf60 0x7f189419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53850/0 (socket says 172.21.15.70:53850) 2023-10-18T12:29:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 -- 172.21.15.70:0/863630552 learned_addr learned my addr 172.21.15.70:0/863630552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f18927fc700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 0x7f1894199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 -- 172.21.15.70:0/863630552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f189410cf60 msgr2=0x7f189419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f189410cf60 0x7f189419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 -- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 msgr2=0x7f1894199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 0x7f1894199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 -- 172.21.15.70:0/863630552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18880095f0 con 0x7f1894072c20 2023-10-18T12:29:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.010+0000 7f1892ffd700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894198e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f188400e9a0 tx=0x7f188400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.011+0000 7f187bfff700 1 -- 172.21.15.70:0/863630552 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884009790 con 0x7f1894072c20 2023-10-18T12:29:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.011+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189419e350 con 0x7f1894072c20 2023-10-18T12:29:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.011+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941a5b90 con 0x7f1894072c20 2023-10-18T12:29:09.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.011+0000 7f187bfff700 1 -- 172.21.15.70:0/863630552 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1884010430 con 0x7f1894072c20 2023-10-18T12:29:09.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.011+0000 7f187bfff700 1 -- 172.21.15.70:0/863630552 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884018680 con 0x7f1894072c20 2023-10-18T12:29:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.014+0000 7f187bfff700 1 -- 172.21.15.70:0/863630552 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f188400ca10 con 0x7f1894072c20 2023-10-18T12:29:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.015+0000 7f18ad01db80 1 Processor -- start 2023-10-18T12:29:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.015+0000 7f187bfff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f187c0a15f0 0x7f187c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.015+0000 7f187bfff700 1 -- 172.21.15.70:0/863630552 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1884014070 con 0x7f1894072c20 2023-10-18T12:29:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.015+0000 7f18ad01db80 1 -- start start 2023-10-18T12:29:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.015+0000 7f18937fe700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f187c0a15f0 0x7f187c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9944040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d993e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d993ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18ad01db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d98b1760 con 0x5575d9943c60 2023-10-18T12:29:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18ad01db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d9897ce0 con 0x5575d9944580 2023-10-18T12:29:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18ad01db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d9895600 con 0x5575d993e650 2023-10-18T12:29:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9944040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18927fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d993e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9944040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60322/0 (socket says 172.21.15.70:60322) 2023-10-18T12:29:09.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f1892ffd700 1 -- 172.21.15.70:0/78109486 learned_addr learned my addr 172.21.15.70:0/78109486 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18937fe700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f187c0a15f0 0x7f187c0a3ab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f188c00f520 tx=0x7f188c00b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18937fe700 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d993ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18927fc700 1 -- 172.21.15.70:0/78109486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 msgr2=0x5575d993ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18927fc700 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d993ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18927fc700 1 -- 172.21.15.70:0/78109486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 msgr2=0x5575d9944040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18927fc700 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9944040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f18927fc700 1 -- 172.21.15.70:0/78109486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575d984e470 con 0x5575d9944580 2023-10-18T12:29:09.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.016+0000 7f1892ffd700 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9944040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:09.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.017+0000 7f18927fc700 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d993e110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f188800b2e0 tx=0x7f1888005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0654b54d2f2367 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.017+0000 7f18797fa700 1 -- 172.21.15.70:0/78109486 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188801d070 con 0x5575d9944580 2023-10-18T12:29:09.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.017+0000 7f18797fa700 1 -- 172.21.15.70:0/78109486 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188801e040 con 0x5575d9944580 2023-10-18T12:29:09.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.017+0000 7f18797fa700 1 -- 172.21.15.70:0/78109486 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888004760 con 0x5575d9944580 2023-10-18T12:29:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 -- 172.21.15.70:0/78109486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 msgr2=0x5575d993e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d993e110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f188800b2e0 tx=0x7f1888005ec0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 -- 172.21.15.70:0/78109486 shutdown_connections 2023-10-18T12:29:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d993ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9944040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 --2- 172.21.15.70:0/78109486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d993e110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 -- 172.21.15.70:0/78109486 >> 172.21.15.70:0/78109486 conn(0x5575d98b27a0 msgr2=0x5575d9938be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 -- 172.21.15.70:0/78109486 shutdown_connections 2023-10-18T12:29:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.018+0000 7f18ad01db80 1 -- 172.21.15.70:0/78109486 wait complete. 2023-10-18T12:29:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 -- start start 2023-10-18T12:29:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 Processor -- start 2023-10-18T12:29:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d994a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9947870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d9947e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d9890ef0 con 0x5575d9943c60 2023-10-18T12:29:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d982d790 con 0x5575d9944580 2023-10-18T12:29:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d982d430 con 0x5575d993e650 2023-10-18T12:29:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f18ad01db80 10 client.0 _ll_register_callbacks cb 0x5575d9777b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.019+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d994a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d994a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53870/0 (socket says 172.21.15.70:53870) 2023-10-18T12:29:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 -- 172.21.15.70:0/1245353940 learned_addr learned my addr 172.21.15.70:0/1245353940 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f18937fe700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d9947e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f18927fc700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9947870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 -- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 msgr2=0x5575d9947e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d9947e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 -- 172.21.15.70:0/1245353940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 msgr2=0x5575d9947870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9947870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 -- 172.21.15.70:0/1245353940 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941a5b90 con 0x5575d993e650 2023-10-18T12:29:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f18937fe700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d9947e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1892ffd700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d994a5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18840ca900 tx=0x7f188409a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c58c55526a3d3c79 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1876ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:29:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1876ff5700 1 -- 172.21.15.70:0/1245353940 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188409a680 con 0x5575d993e650 2023-10-18T12:29:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575d9895f70 con 0x5575d993e650 2023-10-18T12:29:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.020+0000 7f1876ff5700 1 -- 172.21.15.70:0/1245353940 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f188409e370 con 0x5575d993e650 2023-10-18T12:29:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f1876ff5700 1 -- 172.21.15.70:0/1245353940 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f188409e170 con 0x5575d993e650 2023-10-18T12:29:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 10 client.26023 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f1894110090 con 0x5575d993e650 2023-10-18T12:29:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f1876ff5700 1 -- 172.21.15.70:0/1245353940 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1884063070 con 0x5575d993e650 2023-10-18T12:29:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f1876ff5700 1 -- 172.21.15.70:0/1245353940 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f18840a2120 con 0x5575d993e650 2023-10-18T12:29:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 10 client.26023 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 20 client.26023 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d9957e60 0x5575d995a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 4 client.26023 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575d990d930 con 0x5575d9957e60 2023-10-18T12:29:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.021+0000 7f18927fc700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d9957e60 0x5575d995a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.022+0000 7f1876ff5700 10 client.26023 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.022+0000 7f18927fc700 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d9957e60 0x5575d995a280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7856d8d7ba12fa1f server_cookie=2e9435a2ce4a9d32 in_seq=0 out_seq=0 2023-10-18T12:29:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f1876ff5700 1 -- 172.21.15.70:0/1245353940 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5575d990d930 con 0x5575d9957e60 2023-10-18T12:29:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f1876ff5700 10 client.26023 handle_command_reply: tid=0 2023-10-18T12:29:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 client.26023 shutdown 2023-10-18T12:29:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 msgr2=0x5575d994a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d994a5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18840ca900 tx=0x7f188409a920 comp rx=0 tx=0).stop 2023-10-18T12:29:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 shutdown_connections 2023-10-18T12:29:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d9957e60 0x5575d995a280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9944580 0x5575d9947e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d993e650 0x5575d994a5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 --2- 172.21.15.70:0/1245353940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d9947870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f1876ff5700 0 client.26023 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 >> 172.21.15.70:0/1245353940 conn(0x5575d98b27a0 msgr2=0x5575d993d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 shutdown_connections 2023-10-18T12:29:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.023+0000 7f18ad01db80 1 -- 172.21.15.70:0/1245353940 wait complete. 2023-10-18T12:29:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.024+0000 7f18ad01db80 20 client.26023 trim_cache size 0 max 16384 2023-10-18T12:29:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 Processor -- start 2023-10-18T12:29:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 -- start start 2023-10-18T12:29:09.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 0x5575d9941fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d994a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d9932530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d99391b0 con 0x5575d9943c60 2023-10-18T12:29:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d97119a0 con 0x5575d993e650 2023-10-18T12:29:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.043+0000 7f18ad01db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d9935360 con 0x5575d9944580 2023-10-18T12:29:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 0x5575d9941fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 0x5575d9941fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:53150/0 (socket says 172.21.15.70:53150) 2023-10-18T12:29:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 -- 172.21.15.70:0/601523439 learned_addr learned my addr 172.21.15.70:0/601523439 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f18927fc700 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d994a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f18937fe700 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d9932530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 -- 172.21.15.70:0/601523439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 msgr2=0x5575d9932530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d9932530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 -- 172.21.15.70:0/601523439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 msgr2=0x5575d994a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d994a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 -- 172.21.15.70:0/601523439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575d995eeb0 con 0x5575d993e650 2023-10-18T12:29:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.044+0000 7f1892ffd700 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 0x5575d9941fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f188400c510 tx=0x7f18840987d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=939d763e3f088c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f1879ffb700 1 -- 172.21.15.70:0/601523439 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188407b070 con 0x5575d993e650 2023-10-18T12:29:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f1879ffb700 1 -- 172.21.15.70:0/601523439 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1884063070 con 0x5575d993e650 2023-10-18T12:29:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f1879ffb700 1 -- 172.21.15.70:0/601523439 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884099710 con 0x5575d993e650 2023-10-18T12:29:09.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 -- 172.21.15.70:0/601523439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 msgr2=0x5575d9941fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 0x5575d9941fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f188400c510 tx=0x7f18840987d0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 -- 172.21.15.70:0/601523439 shutdown_connections 2023-10-18T12:29:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d9932530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d993e650 0x5575d9941fc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 --2- 172.21.15.70:0/601523439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d9943c60 0x5575d994a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 -- 172.21.15.70:0/601523439 >> 172.21.15.70:0/601523439 conn(0x5575d98b27a0 msgr2=0x5575d9953d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.045+0000 7f18ad01db80 1 -- 172.21.15.70:0/601523439 shutdown_connections 2023-10-18T12:29:09.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.046+0000 7f18ad01db80 1 -- 172.21.15.70:0/601523439 wait complete. 2023-10-18T12:29:09.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.046+0000 7f18ad01db80 1 -- start start 2023-10-18T12:29:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.046+0000 7f18ad01db80 1 Processor -- start 2023-10-18T12:29:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d993e650 0x5575d994dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9943c60 0x5575d994e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d99b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d994e980 con 0x5575d993e650 2023-10-18T12:29:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d99b49f0 con 0x5575d9943c60 2023-10-18T12:29:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d99b4bb0 con 0x5575d9944580 2023-10-18T12:29:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.048+0000 7f18ad01db80 10 client.0 _ll_register_callbacks cb 0x5575d9777b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d99b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d99b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53884/0 (socket says 172.21.15.70:53884) 2023-10-18T12:29:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 -- 172.21.15.70:0/3054255994 learned_addr learned my addr 172.21.15.70:0/3054255994 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 -- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9943c60 msgr2=0x5575d994e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9943c60 0x5575d994e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 -- 172.21.15.70:0/3054255994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d993e650 msgr2=0x5575d994dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d993e650 0x5575d994dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.049+0000 7f18937fe700 1 -- 172.21.15.70:0/3054255994 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894110090 con 0x5575d9944580 2023-10-18T12:29:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.050+0000 7f18937fe700 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d99b44b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f188c017440 tx=0x7f188c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db38a866dc90d421 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.050+0000 7f1876ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:29:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.050+0000 7f1876ff5700 1 -- 172.21.15.70:0/3054255994 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c01c040 con 0x5575d9944580 2023-10-18T12:29:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.051+0000 7f1876ff5700 1 -- 172.21.15.70:0/3054255994 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f188c004db0 con 0x5575d9944580 2023-10-18T12:29:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.051+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f188801e040 con 0x5575d9944580 2023-10-18T12:29:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.051+0000 7f1876ff5700 1 -- 172.21.15.70:0/3054255994 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f188c00e040 con 0x5575d9944580 2023-10-18T12:29:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.051+0000 7f18ad01db80 10 client.26029 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.051+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5575d96bbe60 con 0x5575d9944580 2023-10-18T12:29:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.053+0000 7f1876ff5700 1 -- 172.21.15.70:0/3054255994 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188801e040 con 0x5575d9944580 2023-10-18T12:29:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.053+0000 7f1876ff5700 1 -- 172.21.15.70:0/3054255994 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f188c024510 con 0x5575d9944580 2023-10-18T12:29:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f18ad01db80 10 client.26029 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f18ad01db80 10 client.26029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f18ad01db80 20 client.26029 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f18ad01db80 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d99b6410 0x5575d99b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f18ad01db80 4 client.26029 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:29:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5575d9889ed0 con 0x5575d99b6410 2023-10-18T12:29:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.055+0000 7f1892ffd700 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d99b6410 0x5575d99b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.060+0000 7f1892ffd700 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d99b6410 0x5575d99b8870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a5811248e741df server_cookie=b19a70955e47249 in_seq=0 out_seq=0 2023-10-18T12:29:09.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.060+0000 7f1876ff5700 10 client.26029 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.065+0000 7f1876ff5700 1 -- 172.21.15.70:0/3054255994 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5575d9889ed0 con 0x5575d99b6410 2023-10-18T12:29:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.065+0000 7f1876ff5700 10 client.26029 handle_command_reply: tid=0 2023-10-18T12:29:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.065+0000 7f18ad01db80 1 client.26029 shutdown 2023-10-18T12:29:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.065+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 msgr2=0x5575d99b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.065+0000 7f18ad01db80 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d99b44b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f188c017440 tx=0x7f188c004ad0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 shutdown_connections 2023-10-18T12:29:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f18ad01db80 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5575d99b6410 0x5575d99b8870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f1876ff5700 0 client.26029 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f18ad01db80 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5575d9944580 0x5575d99b44b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f18ad01db80 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575d993e650 0x5575d994dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f18ad01db80 1 --2- 172.21.15.70:0/3054255994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575d9943c60 0x5575d994e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.066+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 >> 172.21.15.70:0/3054255994 conn(0x5575d98b27a0 msgr2=0x5575d9932530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.068+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 shutdown_connections 2023-10-18T12:29:09.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.070+0000 7f18ad01db80 1 -- 172.21.15.70:0/3054255994 wait complete. 2023-10-18T12:29:09.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.073+0000 7f18ad01db80 20 client.26029 trim_cache size 0 max 16384 2023-10-18T12:29:09.077 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:29:09.078 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:29:09.078 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "5981d2fd-4e42-4435-aa42-e71211e43153", 2023-10-18T12:29:09.078 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:29:09.078 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:29:09.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.075+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f187c0a15f0 msgr2=0x7f187c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.075+0000 7f18995ff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f187c0a15f0 0x7f187c0a3ab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f188c00f520 tx=0x7f188c00b040 comp rx=0 tx=0).stop 2023-10-18T12:29:09.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.075+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 msgr2=0x7f1894198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.075+0000 7f18995ff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894198e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f188400e9a0 tx=0x7f188400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 shutdown_connections 2023-10-18T12:29:09.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f187c0a15f0 0x7f187c0a3ab0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f189410cf60 0x7f189419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1894072c20 0x7f1894198e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 --2- 172.21.15.70:0/863630552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894073540 0x7f1894199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 >> 172.21.15.70:0/863630552 conn(0x7f18940fc1b0 msgr2=0x7f18940fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 shutdown_connections 2023-10-18T12:29:09.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.077+0000 7f18995ff700 1 -- 172.21.15.70:0/863630552 wait complete. 2023-10-18T12:29:09.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:29:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 -- 172.21.15.70:0/1084335688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105b30 msgr2=0x7f9e4410d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 --2- 172.21.15.70:0/1084335688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105b30 0x7f9e4410d540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00b0a0 tx=0x7f9e3c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 -- 172.21.15.70:0/1084335688 shutdown_connections 2023-10-18T12:29:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 --2- 172.21.15.70:0/1084335688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105b30 0x7f9e4410d540 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 --2- 172.21.15.70:0/1084335688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e44067d40 0x7f9e44068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 --2- 172.21.15.70:0/1084335688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e44105170 0x7f9e441055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 -- 172.21.15.70:0/1084335688 >> 172.21.15.70:0/1084335688 conn(0x7f9e440767f0 msgr2=0x7f9e44076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 -- 172.21.15.70:0/1084335688 shutdown_connections 2023-10-18T12:29:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.341+0000 7f9e4abff700 1 -- 172.21.15.70:0/1084335688 wait complete. 2023-10-18T12:29:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.342+0000 7f9e4abff700 1 Processor -- start 2023-10-18T12:29:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.342+0000 7f9e4abff700 1 -- start start 2023-10-18T12:29:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.342+0000 7f9e4abff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e44067d40 0x7f9e44198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4abff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 0x7f9e441993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4abff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e44105b30 0x7f9e4419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4abff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4419ddb0 con 0x7f9e44105b30 2023-10-18T12:29:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4abff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4419df20 con 0x7f9e44105170 2023-10-18T12:29:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4abff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4419e0e0 con 0x7f9e44067d40 2023-10-18T12:29:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 0x7f9e441993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 0x7f9e441993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:53182/0 (socket says 172.21.15.70:53182) 2023-10-18T12:29:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 -- 172.21.15.70:0/307536210 learned_addr learned my addr 172.21.15.70:0/307536210 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4919c700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e44105b30 0x7f9e4419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e4899b700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e44067d40 0x7f9e44198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 -- 172.21.15.70:0/307536210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e44067d40 msgr2=0x7f9e44198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e44067d40 0x7f9e44198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 -- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e44105b30 msgr2=0x7f9e4419d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e44105b30 0x7f9e4419d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 -- 172.21.15.70:0/307536210 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009520 con 0x7f9e44105170 2023-10-18T12:29:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.343+0000 7f9e43fff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 0x7f9e441993d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000eb30 tx=0x7f9e3000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.344+0000 7f9e41ffb700 1 -- 172.21.15.70:0/307536210 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30009880 con 0x7f9e44105170 2023-10-18T12:29:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.344+0000 7f9e41ffb700 1 -- 172.21.15.70:0/307536210 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e300044e0 con 0x7f9e44105170 2023-10-18T12:29:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.344+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c009d00 con 0x7f9e44105170 2023-10-18T12:29:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.344+0000 7f9e41ffb700 1 -- 172.21.15.70:0/307536210 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30003d80 con 0x7f9e44105170 2023-10-18T12:29:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.344+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4419e5b0 con 0x7f9e44105170 2023-10-18T12:29:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.347+0000 7f9e41ffb700 1 -- 172.21.15.70:0/307536210 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9e30003f20 con 0x7f9e44105170 2023-10-18T12:29:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.348+0000 7f9e41ffb700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e340a1670 0x7f9e340a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.348+0000 7f9e5e558b80 1 Processor -- start 2023-10-18T12:29:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.348+0000 7f9e41ffb700 1 -- 172.21.15.70:0/307536210 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9e30014070 con 0x7f9e44105170 2023-10-18T12:29:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.348+0000 7f9e5e558b80 1 -- start start 2023-10-18T12:29:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.348+0000 7f9e4899b700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e340a1670 0x7f9e340a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e730699e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e73069dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e73069e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7306046e0 con 0x55e73069e500 2023-10-18T12:29:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7305eac60 con 0x55e73069a3b0 2023-10-18T12:29:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7305e8580 con 0x55e730699a90 2023-10-18T12:29:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e4899b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e730699e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e4899b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e730699e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53920/0 (socket says 172.21.15.70:53920) 2023-10-18T12:29:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e4899b700 1 -- 172.21.15.70:0/2634135404 learned_addr learned my addr 172.21.15.70:0/2634135404 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.349+0000 7f9e43fff700 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e73069dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4919c700 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e73069e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e340a1670 0x7f9e340a3b30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009cb0 tx=0x7f9e38009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 -- 172.21.15.70:0/2634135404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 msgr2=0x55e73069dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e73069dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 -- 172.21.15.70:0/2634135404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 msgr2=0x55e73069e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e73069e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 -- 172.21.15.70:0/2634135404 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7305a13f0 con 0x55e730699a90 2023-10-18T12:29:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e4899b700 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e730699e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009fd0 tx=0x7f9e3801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfe098eaa5350c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e2effd700 1 -- 172.21.15.70:0/2634135404 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3801ce30 con 0x55e730699a90 2023-10-18T12:29:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e2effd700 1 -- 172.21.15.70:0/2634135404 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e38004cf0 con 0x55e730699a90 2023-10-18T12:29:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.350+0000 7f9e2effd700 1 -- 172.21.15.70:0/2634135404 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38024420 con 0x55e730699a90 2023-10-18T12:29:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2634135404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 msgr2=0x55e730699e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e730699e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009fd0 tx=0x7f9e3801c000 comp rx=0 tx=0).stop 2023-10-18T12:29:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2634135404 shutdown_connections 2023-10-18T12:29:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e73069e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e730699e70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2634135404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e73069dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2634135404 >> 172.21.15.70:0/2634135404 conn(0x55e730605720 msgr2=0x55e73068bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2634135404 shutdown_connections 2023-10-18T12:29:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.351+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2634135404 wait complete. 2023-10-18T12:29:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.352+0000 7f9e5e558b80 1 -- start start 2023-10-18T12:29:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.352+0000 7f9e5e558b80 1 Processor -- start 2023-10-18T12:29:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069a3b0 0x55e7306a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7305e3e70 con 0x55e73069a3b0 2023-10-18T12:29:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e730580710 con 0x55e73069e500 2023-10-18T12:29:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7305803b0 con 0x55e730699a90 2023-10-18T12:29:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e5e558b80 10 client.0 _ll_register_callbacks cb 0x55e7304caa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e4919c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e4919c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:53210/0 (socket says 172.21.15.70:53210) 2023-10-18T12:29:09.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e4919c700 1 -- 172.21.15.70:0/1240172143 learned_addr learned my addr 172.21.15.70:0/1240172143 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.353+0000 7f9e4899b700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4919c700 1 -- 172.21.15.70:0/1240172143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 msgr2=0x55e7306a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4919c700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4919c700 1 -- 172.21.15.70:0/1240172143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069a3b0 msgr2=0x55e7306a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4919c700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069a3b0 0x55e7306a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4919c700 1 -- 172.21.15.70:0/1240172143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e38017050 con 0x55e73069e500 2023-10-18T12:29:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4899b700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:29:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e4919c700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306a3790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c015000 tx=0x7f9e3c007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f7090359067134 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e23fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:29:09.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e23fff700 1 -- 172.21.15.70:0/1240172143 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00e040 con 0x55e73069e500 2023-10-18T12:29:09.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7305e8ef0 con 0x55e73069e500 2023-10-18T12:29:09.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.354+0000 7f9e23fff700 1 -- 172.21.15.70:0/1240172143 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9e3c003c00 con 0x55e73069e500 2023-10-18T12:29:09.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e23fff700 1 -- 172.21.15.70:0/1240172143 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3c00be70 con 0x55e73069e500 2023-10-18T12:29:09.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 10 client.26042 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:09.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e73040ede0 con 0x55e73069e500 2023-10-18T12:29:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e23fff700 1 -- 172.21.15.70:0/1240172143 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7305e8ef0 con 0x55e73069e500 2023-10-18T12:29:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e23fff700 1 -- 172.21.15.70:0/1240172143 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9e3c019030 con 0x55e73069e500 2023-10-18T12:29:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 20 client.26042 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7306b36d0 0x55e7306b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 4 client.26042 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.355+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7306608b0 con 0x55e7306b36d0 2023-10-18T12:29:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.356+0000 7f9e4899b700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7306b36d0 0x55e7306b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.356+0000 7f9e4899b700 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7306b36d0 0x55e7306b5af0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1700958072ac51e4 server_cookie=15827a6afc5c7ba5 in_seq=0 out_seq=0 2023-10-18T12:29:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.356+0000 7f9e23fff700 10 client.26042 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e23fff700 1 -- 172.21.15.70:0/1240172143 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7306608b0 con 0x55e7306b36d0 2023-10-18T12:29:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e23fff700 10 client.26042 handle_command_reply: tid=0 2023-10-18T12:29:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 client.26042 shutdown 2023-10-18T12:29:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 msgr2=0x55e7306a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306a3790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c015000 tx=0x7f9e3c007f90 comp rx=0 tx=0).stop 2023-10-18T12:29:09.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 shutdown_connections 2023-10-18T12:29:09.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e7306b36d0 0x55e7306b5af0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306a3790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e23fff700 0 client.26042 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/1240172143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069a3b0 0x55e7306a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.357+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 >> 172.21.15.70:0/1240172143 conn(0x55e730605720 msgr2=0x55e730690080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.358+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 shutdown_connections 2023-10-18T12:29:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.358+0000 7f9e5e558b80 1 -- 172.21.15.70:0/1240172143 wait complete. 2023-10-18T12:29:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.358+0000 7f9e5e558b80 20 client.26042 trim_cache size 0 max 16384 2023-10-18T12:29:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.375+0000 7f9e5e558b80 1 Processor -- start 2023-10-18T12:29:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.375+0000 7f9e5e558b80 1 -- start start 2023-10-18T12:29:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 0x55e7306a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e73069a3b0 0x55e730690970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e730464920 con 0x55e730699a90 2023-10-18T12:29:09.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7306a2e50 con 0x55e73069e500 2023-10-18T12:29:09.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e730690ee0 con 0x55e73069a3b0 2023-10-18T12:29:09.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 0x55e7306a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 0x55e7306a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60376/0 (socket says 172.21.15.70:60376) 2023-10-18T12:29:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4899b700 1 -- 172.21.15.70:0/2325506515 learned_addr learned my addr 172.21.15.70:0/2325506515 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4919c700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e43fff700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e73069a3b0 0x55e730690970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4919c700 1 -- 172.21.15.70:0/2325506515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e73069a3b0 msgr2=0x55e730690970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4919c700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e73069a3b0 0x55e730690970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4919c700 1 -- 172.21.15.70:0/2325506515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 msgr2=0x55e7306a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4919c700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 0x55e7306a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4919c700 1 -- 172.21.15.70:0/2325506515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7303f7c60 con 0x55e73069e500 2023-10-18T12:29:09.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e43fff700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e73069a3b0 0x55e730690970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:29:09.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.376+0000 7f9e4899b700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 0x55e7306a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:09.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.377+0000 7f9e4919c700 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306aa6c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c01cb00 tx=0x7f9e3c00b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe509534bfbc530 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.377+0000 7f9e2f7fe700 1 -- 172.21.15.70:0/2325506515 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00e040 con 0x55e73069e500 2023-10-18T12:29:09.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.377+0000 7f9e2f7fe700 1 -- 172.21.15.70:0/2325506515 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e3c01c640 con 0x55e73069e500 2023-10-18T12:29:09.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.377+0000 7f9e2f7fe700 1 -- 172.21.15.70:0/2325506515 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0050e0 con 0x55e73069e500 2023-10-18T12:29:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2325506515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 msgr2=0x55e7306aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306aa6c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c01cb00 tx=0x7f9e3c00b7c0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2325506515 shutdown_connections 2023-10-18T12:29:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069e500 0x55e7306aa6c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e730699a90 0x55e7306a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/2325506515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e73069a3b0 0x55e730690970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2325506515 >> 172.21.15.70:0/2325506515 conn(0x55e730605720 msgr2=0x55e7306ba450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2325506515 shutdown_connections 2023-10-18T12:29:09.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 -- 172.21.15.70:0/2325506515 wait complete. 2023-10-18T12:29:09.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 -- start start 2023-10-18T12:29:09.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.378+0000 7f9e5e558b80 1 Processor -- start 2023-10-18T12:29:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306aa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e7306aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e7306a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7306a6f80 con 0x55e73069e500 2023-10-18T12:29:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7306a70f0 con 0x55e73069a3b0 2023-10-18T12:29:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7306a72b0 con 0x55e730699a90 2023-10-18T12:29:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e5e558b80 10 client.0 _ll_register_callbacks cb 0x55e7304caa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e7306aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306aa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53946/0 (socket says 172.21.15.70:53946) 2023-10-18T12:29:09.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 -- 172.21.15.70:0/475807925 learned_addr learned my addr 172.21.15.70:0/475807925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:09.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4919c700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e7306a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 -- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 msgr2=0x55e7306aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e7306aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 -- 172.21.15.70:0/475807925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 msgr2=0x55e7306a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:09.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e7306a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e4899b700 1 -- 172.21.15.70:0/475807925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7305a13f0 con 0x55e730699a90 2023-10-18T12:29:09.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.379+0000 7f9e43fff700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e7306aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:09.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e4899b700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306aa190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009fd0 tx=0x7f9e38004e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1405a558ad3e6b45 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:09.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e23fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:29:09.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e23fff700 1 -- 172.21.15.70:0/475807925 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38005000 con 0x55e730699a90 2023-10-18T12:29:09.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e38004cf0 con 0x55e730699a90 2023-10-18T12:29:09.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e23fff700 1 -- 172.21.15.70:0/475807925 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9e38005160 con 0x55e730699a90 2023-10-18T12:29:09.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e23fff700 1 -- 172.21.15.70:0/475807925 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e38005320 con 0x55e730699a90 2023-10-18T12:29:09.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e5e558b80 10 client.26047 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:09.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e73040ede0 con 0x55e730699a90 2023-10-18T12:29:09.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e23fff700 1 -- 172.21.15.70:0/475807925 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e38004cf0 con 0x55e730699a90 2023-10-18T12:29:09.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.380+0000 7f9e23fff700 1 -- 172.21.15.70:0/475807925 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9e3801c400 con 0x55e730699a90 2023-10-18T12:29:09.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e5e558b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:09.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e5e558b80 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:09.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e5e558b80 20 client.26047 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:09.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e730708830 0x55e73070ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:09.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e5e558b80 4 client.26047 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:29:09.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7305dce50 con 0x55e730708830 2023-10-18T12:29:09.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e43fff700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e730708830 0x55e73070ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:09.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e23fff700 10 client.26047 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.381+0000 7f9e43fff700 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e730708830 0x55e73070ac50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1066b5694cba0fd server_cookie=88f338828c3b753e in_seq=0 out_seq=0 2023-10-18T12:29:09.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e23fff700 1 -- 172.21.15.70:0/475807925 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e7305dce50 con 0x55e730708830 2023-10-18T12:29:09.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e23fff700 10 client.26047 handle_command_reply: tid=0 2023-10-18T12:29:09.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 client.26047 shutdown 2023-10-18T12:29:09.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 msgr2=0x55e7306aa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306aa190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009fd0 tx=0x7f9e38004e80 comp rx=0 tx=0).stop 2023-10-18T12:29:09.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 shutdown_connections 2023-10-18T12:29:09.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e730708830 0x55e73070ac50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73069e500 0x55e7306a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e730699a90 0x55e7306aa190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 --2- 172.21.15.70:0/475807925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e73069a3b0 0x55e7306aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e23fff700 0 client.26047 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:09.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 >> 172.21.15.70:0/475807925 conn(0x55e730605720 msgr2=0x55e7306a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 shutdown_connections 2023-10-18T12:29:09.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 1 -- 172.21.15.70:0/475807925 wait complete. 2023-10-18T12:29:09.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.382+0000 7f9e5e558b80 20 client.26047 trim_cache size 0 max 16384 2023-10-18T12:29:09.400 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:29:09.400 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:29:09.400 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:29:09.400 INFO:teuthology.orchestra.run.smithi070.stdout: "5981d2fd-4e42-4435-aa42-e71211e43153": { 2023-10-18T12:29:09.400 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:29:09.400 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "5981d2fd-4e42-4435-aa42-e71211e43153", 2023-10-18T12:29:09.401 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:29:09.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:29:09.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:29:09.401 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:29:09.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e340a1670 msgr2=0x7f9e340a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e340a1670 0x7f9e340a3b30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009cb0 tx=0x7f9e38009260 comp rx=0 tx=0).stop 2023-10-18T12:29:09.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 msgr2=0x7f9e441993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:09.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 0x7f9e441993d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000eb30 tx=0x7f9e3000ee40 comp rx=0 tx=0).stop 2023-10-18T12:29:09.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 shutdown_connections 2023-10-18T12:29:09.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e44105b30 0x7f9e4419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e44067d40 0x7f9e44198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9e340a1670 0x7f9e340a3b30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 --2- 172.21.15.70:0/307536210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e44105170 0x7f9e441993d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:09.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 >> 172.21.15.70:0/307536210 conn(0x7f9e440767f0 msgr2=0x7f9e44100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:09.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 shutdown_connections 2023-10-18T12:29:09.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:09.383+0000 7f9e4abff700 1 -- 172.21.15.70:0/307536210 wait complete. 2023-10-18T12:29:09.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:09 smithi070 ceph-mon[98639]: pgmap v408: 129 pgs: 129 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2023-10-18T12:29:09.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:09 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:29:09.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:09 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:09.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:09 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:29:09.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:09 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:09.406 INFO:tasks.cephfs.filesystem:scrub status for tag:5981d2fd-4e42-4435-aa42-e71211e43153 - {'path': '/', 'tag': '5981d2fd-4e42-4435-aa42-e71211e43153', 'options': 'recursive,force'} 2023-10-18T12:29:09.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:09 smithi152 ceph-mon[134767]: pgmap v408: 129 pgs: 129 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2023-10-18T12:29:09.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:09 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:29:09.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:09 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:09.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:09 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:29:09.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:09 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:09.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:09 smithi186 ceph-mon[135253]: pgmap v408: 129 pgs: 129 active+clean; 132 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2023-10-18T12:29:09.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:09 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:29:09.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:09 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:09.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:09 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:29:09.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:09 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:09.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 255204 14.21 MB/sec warmup 98 sec latency 65.624 ms 2023-10-18T12:29:10.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 257823 14.19 MB/sec warmup 99 sec latency 10.008 ms 2023-10-18T12:29:11.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:11 smithi070 ceph-mon[98639]: pgmap v409: 129 pgs: 129 active+clean; 136 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 321 op/s 2023-10-18T12:29:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:11 smithi152 ceph-mon[134767]: pgmap v409: 129 pgs: 129 active+clean; 136 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 321 op/s 2023-10-18T12:29:11.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:11 smithi186 ceph-mon[135253]: pgmap v409: 129 pgs: 129 active+clean; 136 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 321 op/s 2023-10-18T12:29:11.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 260007 14.19 MB/sec warmup 100 sec latency 15.978 ms 2023-10-18T12:29:12.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 262946 14.22 MB/sec warmup 101 sec latency 23.390 ms 2023-10-18T12:29:13.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:13 smithi070 ceph-mon[98639]: pgmap v410: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 293 op/s 2023-10-18T12:29:13.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:13 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:29:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:13 smithi152 ceph-mon[134767]: pgmap v410: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 293 op/s 2023-10-18T12:29:13.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:13 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:29:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:13 smithi186 ceph-mon[135253]: pgmap v410: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 293 op/s 2023-10-18T12:29:13.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:13 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:29:13.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 265524 14.20 MB/sec warmup 102 sec latency 41.968 ms 2023-10-18T12:29:14.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 267053 14.14 MB/sec warmup 103 sec latency 69.716 ms 2023-10-18T12:29:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:15 smithi070 ceph-mon[98639]: pgmap v411: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 287 op/s 2023-10-18T12:29:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:15 smithi152 ceph-mon[134767]: pgmap v411: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 287 op/s 2023-10-18T12:29:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:15 smithi186 ceph-mon[135253]: pgmap v411: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 287 op/s 2023-10-18T12:29:15.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 270080 14.18 MB/sec warmup 104 sec latency 8.366 ms 2023-10-18T12:29:16.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 273255 14.21 MB/sec warmup 105 sec latency 7.380 ms 2023-10-18T12:29:17.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:17 smithi070 ceph-mon[98639]: pgmap v412: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2023-10-18T12:29:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:17 smithi152 ceph-mon[134767]: pgmap v412: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2023-10-18T12:29:17.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:17 smithi186 ceph-mon[135253]: pgmap v412: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2023-10-18T12:29:17.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 275630 14.16 MB/sec warmup 106 sec latency 21.820 ms 2023-10-18T12:29:18.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 277817 14.15 MB/sec warmup 107 sec latency 42.901 ms 2023-10-18T12:29:19.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:19 smithi070 ceph-mon[98639]: pgmap v413: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 257 op/s 2023-10-18T12:29:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:19 smithi152 ceph-mon[134767]: pgmap v413: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 257 op/s 2023-10-18T12:29:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:19 smithi186 ceph-mon[135253]: pgmap v413: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 257 op/s 2023-10-18T12:29:19.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 280483 14.18 MB/sec warmup 108 sec latency 114.834 ms 2023-10-18T12:29:20.841 INFO:tasks.workunit.client.0.smithi070.stdout: 1 282849 14.13 MB/sec warmup 109 sec latency 8.314 ms 2023-10-18T12:29:21.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:21 smithi070 ceph-mon[98639]: pgmap v414: 129 pgs: 129 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 257 op/s 2023-10-18T12:29:21.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:21 smithi152 ceph-mon[134767]: pgmap v414: 129 pgs: 129 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 257 op/s 2023-10-18T12:29:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:21 smithi186 ceph-mon[135253]: pgmap v414: 129 pgs: 129 active+clean; 148 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 257 op/s 2023-10-18T12:29:21.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 285007 14.12 MB/sec warmup 110 sec latency 16.246 ms 2023-10-18T12:29:22.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 287787 14.14 MB/sec warmup 111 sec latency 76.027 ms 2023-10-18T12:29:23.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:23 smithi070 ceph-mon[98639]: pgmap v415: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 261 op/s 2023-10-18T12:29:23.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:23 smithi152 ceph-mon[134767]: pgmap v415: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 261 op/s 2023-10-18T12:29:23.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:23 smithi186 ceph-mon[135253]: pgmap v415: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 261 op/s 2023-10-18T12:29:23.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 290889 14.17 MB/sec warmup 112 sec latency 27.824 ms 2023-10-18T12:29:24.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 291934 14.08 MB/sec warmup 113 sec latency 158.422 ms 2023-10-18T12:29:25.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:25 smithi186 ceph-mon[135253]: pgmap v416: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 239 op/s 2023-10-18T12:29:25.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:25 smithi070 ceph-mon[98639]: pgmap v416: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 239 op/s 2023-10-18T12:29:25.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:25 smithi152 ceph-mon[134767]: pgmap v416: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 239 op/s 2023-10-18T12:29:25.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 294839 14.11 MB/sec warmup 114 sec latency 14.718 ms 2023-10-18T12:29:26.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 297452 14.09 MB/sec warmup 115 sec latency 71.444 ms 2023-10-18T12:29:27.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:27 smithi186 ceph-mon[135253]: pgmap v417: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2023-10-18T12:29:27.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:27 smithi070 ceph-mon[98639]: pgmap v417: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2023-10-18T12:29:27.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:27 smithi152 ceph-mon[134767]: pgmap v417: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2023-10-18T12:29:27.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 299677 14.07 MB/sec warmup 116 sec latency 7.013 ms 2023-10-18T12:29:28.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 302124 14.07 MB/sec warmup 117 sec latency 19.293 ms 2023-10-18T12:29:29.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:29 smithi186 ceph-mon[135253]: pgmap v418: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 300 op/s 2023-10-18T12:29:29.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:29 smithi070 ceph-mon[98639]: pgmap v418: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 300 op/s 2023-10-18T12:29:29.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:29 smithi152 ceph-mon[134767]: pgmap v418: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 300 op/s 2023-10-18T12:29:29.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 304710 14.06 MB/sec warmup 118 sec latency 68.320 ms 2023-10-18T12:29:30.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 306705 14.04 MB/sec warmup 119 sec latency 56.194 ms 2023-10-18T12:29:31.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:31 smithi186 ceph-mon[135253]: pgmap v419: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 275 op/s 2023-10-18T12:29:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:31 smithi070 ceph-mon[98639]: pgmap v419: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 275 op/s 2023-10-18T12:29:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:31 smithi152 ceph-mon[134767]: pgmap v419: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 275 op/s 2023-10-18T12:29:32.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 312435 18.46 MB/sec execute 1 sec latency 13.735 ms 2023-10-18T12:29:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:33 smithi070 ceph-mon[98639]: pgmap v420: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2023-10-18T12:29:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:33 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:29:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:33 smithi152 ceph-mon[134767]: pgmap v420: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2023-10-18T12:29:33.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:33 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:29:33.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:33 smithi186 ceph-mon[135253]: pgmap v420: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 272 op/s 2023-10-18T12:29:33.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:33 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:29:33.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 315070 15.64 MB/sec execute 2 sec latency 8.678 ms 2023-10-18T12:29:34.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:29:34.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:29:34.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:34 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:29:34.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:29:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:29:34.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:34 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:29:34.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:29:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:29:34.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:34 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:29:34.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 316773 13.27 MB/sec execute 3 sec latency 60.958 ms 2023-10-18T12:29:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:35 smithi070 ceph-mon[98639]: pgmap v421: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 236 op/s 2023-10-18T12:29:35.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:35 smithi152 ceph-mon[134767]: pgmap v421: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 236 op/s 2023-10-18T12:29:35.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:35 smithi186 ceph-mon[135253]: pgmap v421: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 236 op/s 2023-10-18T12:29:35.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 319642 14.25 MB/sec execute 4 sec latency 21.325 ms 2023-10-18T12:29:36.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 322756 14.88 MB/sec execute 5 sec latency 6.604 ms 2023-10-18T12:29:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:37 smithi070 ceph-mon[98639]: pgmap v422: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 267 op/s 2023-10-18T12:29:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:37 smithi152 ceph-mon[134767]: pgmap v422: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 267 op/s 2023-10-18T12:29:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:37 smithi186 ceph-mon[135253]: pgmap v422: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 267 op/s 2023-10-18T12:29:37.842 INFO:tasks.workunit.client.0.smithi070.stdout: 1 325273 13.98 MB/sec execute 6 sec latency 7.013 ms 2023-10-18T12:29:38.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 327501 13.90 MB/sec execute 7 sec latency 20.926 ms 2023-10-18T12:29:39.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:29:39.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:39 smithi070 ceph-mon[98639]: pgmap v423: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 265 op/s 2023-10-18T12:29:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 -- 172.21.15.70:0/108384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c04100710 msgr2=0x7f4c04100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 --2- 172.21.15.70:0/108384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c04100710 0x7f4c04100b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009990 tx=0x7f4bf8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 -- 172.21.15.70:0/108384928 shutdown_connections 2023-10-18T12:29:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 --2- 172.21.15.70:0/108384928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c041010d0 0x7f4c0410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 --2- 172.21.15.70:0/108384928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c04106720 0x7f4c04106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 --2- 172.21.15.70:0/108384928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c04100710 0x7f4c04100b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 -- 172.21.15.70:0/108384928 >> 172.21.15.70:0/108384928 conn(0x7f4c040fc200 msgr2=0x7f4c040fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.630+0000 7f4c08dff700 1 -- 172.21.15.70:0/108384928 shutdown_connections 2023-10-18T12:29:39.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.631+0000 7f4c08dff700 1 -- 172.21.15.70:0/108384928 wait complete. 2023-10-18T12:29:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 Processor -- start 2023-10-18T12:29:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 -- start start 2023-10-18T12:29:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c04100710 0x7f4c04198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c041010d0 0x7f4c04199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106720 0x7f4c0419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419dce0 con 0x7f4c04106720 2023-10-18T12:29:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419de50 con 0x7f4c041010d0 2023-10-18T12:29:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c08dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419e010 con 0x7f4c04100710 2023-10-18T12:29:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.632+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c041010d0 0x7f4c04199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c041010d0 0x7f4c04199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60042/0 (socket says 172.21.15.70:60042) 2023-10-18T12:29:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c01d9b700 1 -- 172.21.15.70:0/4003018982 learned_addr learned my addr 172.21.15.70:0/4003018982 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106720 0x7f4c0419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c0259c700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c04100710 0x7f4c04198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 -- 172.21.15.70:0/4003018982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c04100710 msgr2=0x7f4c04198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c04100710 0x7f4c04198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 -- 172.21.15.70:0/4003018982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c041010d0 msgr2=0x7f4c04199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c041010d0 0x7f4c04199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 -- 172.21.15.70:0/4003018982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc009c40 con 0x7f4c04106720 2023-10-18T12:29:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.633+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106720 0x7f4c0419d6a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00efb0 tx=0x7f4bfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.634+0000 7f4bf37fe700 1 -- 172.21.15.70:0/4003018982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc003a00 con 0x7f4c04106720 2023-10-18T12:29:39.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.634+0000 7f4bf37fe700 1 -- 172.21.15.70:0/4003018982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bfc00f040 con 0x7f4c04106720 2023-10-18T12:29:39.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.634+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf4009520 con 0x7f4c04106720 2023-10-18T12:29:39.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.634+0000 7f4bf37fe700 1 -- 172.21.15.70:0/4003018982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc004540 con 0x7f4c04106720 2023-10-18T12:29:39.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.634+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf80095f0 con 0x7f4c04106720 2023-10-18T12:29:39.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.637+0000 7f4bf37fe700 1 -- 172.21.15.70:0/4003018982 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4bfc003bf0 con 0x7f4c04106720 2023-10-18T12:29:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.638+0000 7f4bf37fe700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4bec0a15f0 0x7f4bec0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.638+0000 7f4c1c656b80 1 Processor -- start 2023-10-18T12:29:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.638+0000 7f4bf37fe700 1 -- 172.21.15.70:0/4003018982 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4bfc0cd0b0 con 0x7f4c04106720 2023-10-18T12:29:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c0259c700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4bec0a15f0 0x7f4bec0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 -- start start 2023-10-18T12:29:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d353876fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353877500 0x55d353871090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d353871a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3537e46e0 con 0x55d353877500 2023-10-18T12:29:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3537cac60 con 0x55d353876be0 2023-10-18T12:29:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3537c8580 con 0x55d3538715d0 2023-10-18T12:29:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c0259c700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4bec0a15f0 0x7f4bec0a3ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009d70 tx=0x7f4bf4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d353876fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.639+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353877500 0x55d353871090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d353876fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60052/0 (socket says 172.21.15.70:60052) 2023-10-18T12:29:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 -- 172.21.15.70:0/1760438642 learned_addr learned my addr 172.21.15.70:0/1760438642 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d353871a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 -- 172.21.15.70:0/1760438642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 msgr2=0x55d353871a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d353871a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 -- 172.21.15.70:0/1760438642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353877500 msgr2=0x55d353871090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353877500 0x55d353871090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 -- 172.21.15.70:0/1760438642 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3537813f0 con 0x55d353876be0 2023-10-18T12:29:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4c0259c700 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d353876fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e5b0 tx=0x7f4bf4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d71f49f93515b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4bf0ff9700 1 -- 172.21.15.70:0/1760438642 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400ed50 con 0x55d353876be0 2023-10-18T12:29:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4bf0ff9700 1 -- 172.21.15.70:0/1760438642 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bf401e8e0 con 0x55d353876be0 2023-10-18T12:29:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.640+0000 7f4bf0ff9700 1 -- 172.21.15.70:0/1760438642 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf401d7d0 con 0x55d353876be0 2023-10-18T12:29:39.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 -- 172.21.15.70:0/1760438642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 msgr2=0x55d353876fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d353876fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e5b0 tx=0x7f4bf4014000 comp rx=0 tx=0).stop 2023-10-18T12:29:39.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 -- 172.21.15.70:0/1760438642 shutdown_connections 2023-10-18T12:29:39.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d353871a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d353876fc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/1760438642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353877500 0x55d353871090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 -- 172.21.15.70:0/1760438642 >> 172.21.15.70:0/1760438642 conn(0x55d3537e5720 msgr2=0x55d35386bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 -- 172.21.15.70:0/1760438642 shutdown_connections 2023-10-18T12:29:39.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.641+0000 7f4c1c656b80 1 -- 172.21.15.70:0/1760438642 wait complete. 2023-10-18T12:29:39.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.642+0000 7f4c1c656b80 1 -- start start 2023-10-18T12:29:39.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.642+0000 7f4c1c656b80 1 Processor -- start 2023-10-18T12:29:39.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3538715d0 0x55d35387a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d35387adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d35387b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3537c3e70 con 0x55d3538715d0 2023-10-18T12:29:39.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d353760710 con 0x55d353876be0 2023-10-18T12:29:39.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3537603b0 con 0x55d353877500 2023-10-18T12:29:39.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c1c656b80 10 client.0 _ll_register_callbacks cb 0x55d3536aaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:39.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d35387adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3538715d0 0x55d35387a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d35387adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60060/0 (socket says 172.21.15.70:60060) 2023-10-18T12:29:39.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c01d9b700 1 -- 172.21.15.70:0/3681908482 learned_addr learned my addr 172.21.15.70:0/3681908482 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:39.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d35387b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c0259c700 1 -- 172.21.15.70:0/3681908482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 msgr2=0x55d35387b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.643+0000 7f4c0259c700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d35387b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c0259c700 1 -- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 msgr2=0x55d35387adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c0259c700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d35387adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c0259c700 1 -- 172.21.15.70:0/3681908482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf4017070 con 0x55d3538715d0 2023-10-18T12:29:39.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c01d9b700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d35387adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:39.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c0259c700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3538715d0 0x55d35387a870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e580 tx=0x7f4bf400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751d2ebf495d423b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4be9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:29:39.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4be9ffb700 1 -- 172.21.15.70:0/3681908482 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402a040 con 0x55d3538715d0 2023-10-18T12:29:39.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3537c8ef0 con 0x55d3538715d0 2023-10-18T12:29:39.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4be9ffb700 1 -- 172.21.15.70:0/3681908482 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4bf401e6f0 con 0x55d3538715d0 2023-10-18T12:29:39.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4be9ffb700 1 -- 172.21.15.70:0/3681908482 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bf4004220 con 0x55d3538715d0 2023-10-18T12:29:39.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c1c656b80 10 client.16035 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:39.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.644+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d3535eede0 con 0x55d3538715d0 2023-10-18T12:29:39.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4be9ffb700 1 -- 172.21.15.70:0/3681908482 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3537c8ef0 con 0x55d3538715d0 2023-10-18T12:29:39.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4be9ffb700 1 -- 172.21.15.70:0/3681908482 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4bf4024020 con 0x55d3538715d0 2023-10-18T12:29:39.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c1c656b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:39.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c1c656b80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:39.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c1c656b80 20 client.16035 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:39.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d35388aeb0 0x55d35388d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c1c656b80 4 client.16035 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:39.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3538408b0 con 0x55d35388aeb0 2023-10-18T12:29:39.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.645+0000 7f4c01d9b700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d35388aeb0 0x55d35388d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.646+0000 7f4c01d9b700 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d35388aeb0 0x55d35388d2d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f186764b9f85ccf server_cookie=db86e57f7c720706 in_seq=0 out_seq=0 2023-10-18T12:29:39.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.646+0000 7f4be9ffb700 10 client.16035 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:39.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4be9ffb700 1 -- 172.21.15.70:0/3681908482 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d3538408b0 con 0x55d35388aeb0 2023-10-18T12:29:39.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4be9ffb700 10 client.16035 handle_command_reply: tid=0 2023-10-18T12:29:39.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 client.16035 shutdown 2023-10-18T12:29:39.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3538715d0 msgr2=0x55d35387a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3538715d0 0x55d35387a870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400e580 tx=0x7f4bf400ed50 comp rx=0 tx=0).stop 2023-10-18T12:29:39.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 shutdown_connections 2023-10-18T12:29:39.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d35388aeb0 0x55d35388d2d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d35387b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3538715d0 0x55d35387a870 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4be9ffb700 0 client.16035 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:39.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3681908482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353876be0 0x55d35387adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 >> 172.21.15.70:0/3681908482 conn(0x55d3537e5720 msgr2=0x55d353870080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 shutdown_connections 2023-10-18T12:29:39.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3681908482 wait complete. 2023-10-18T12:29:39.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.647+0000 7f4c1c656b80 20 client.16035 trim_cache size 0 max 16384 2023-10-18T12:29:39.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.668+0000 7f4c1c656b80 1 Processor -- start 2023-10-18T12:29:39.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.668+0000 7f4c1c656b80 1 -- start start 2023-10-18T12:29:39.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:39 smithi152 ceph-mon[134767]: pgmap v423: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 265 op/s 2023-10-18T12:29:39.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 0x55d3538761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d353881eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d353874dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d353644920 con 0x55d353876be0 2023-10-18T12:29:39.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d353876fc0 con 0x55d3538715d0 2023-10-18T12:29:39.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35387be70 con 0x55d353877500 2023-10-18T12:29:39.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 0x55d3538761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c02d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d353874dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 0x55d3538761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60068/0 (socket says 172.21.15.70:60068) 2023-10-18T12:29:39.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d353881eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 -- 172.21.15.70:0/3782302547 learned_addr learned my addr 172.21.15.70:0/3782302547 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:39.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 -- 172.21.15.70:0/3782302547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 msgr2=0x55d353874dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d353874dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 -- 172.21.15.70:0/3782302547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 msgr2=0x55d353881eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d353881eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c0259c700 1 -- 172.21.15.70:0/3782302547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3535d7c60 con 0x55d3538715d0 2023-10-18T12:29:39.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.669+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d353874dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:39.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.670+0000 7f4c0259c700 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 0x55d3538761e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009870 tx=0x7f4bf4005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ce63536c7d7a1d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.670+0000 7f4bf17fa700 1 -- 172.21.15.70:0/3782302547 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402a040 con 0x55d3538715d0 2023-10-18T12:29:39.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.670+0000 7f4bf17fa700 1 -- 172.21.15.70:0/3782302547 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bf401e770 con 0x55d3538715d0 2023-10-18T12:29:39.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.670+0000 7f4bf17fa700 1 -- 172.21.15.70:0/3782302547 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4005280 con 0x55d3538715d0 2023-10-18T12:29:39.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3782302547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 msgr2=0x55d3538761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 0x55d3538761e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009870 tx=0x7f4bf4005b80 comp rx=0 tx=0).stop 2023-10-18T12:29:39.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3782302547 shutdown_connections 2023-10-18T12:29:39.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d353877500 0x55d353874dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3538715d0 0x55d3538761e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/3782302547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d353881eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3782302547 >> 172.21.15.70:0/3782302547 conn(0x55d3537e5720 msgr2=0x55d3538746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3782302547 shutdown_connections 2023-10-18T12:29:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 -- 172.21.15.70:0/3782302547 wait complete. 2023-10-18T12:29:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.671+0000 7f4c1c656b80 1 -- start start 2023-10-18T12:29:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 Processor -- start 2023-10-18T12:29:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d35387f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d35387fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353877500 0x55d3538ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3538806c0 con 0x55d353876be0 2023-10-18T12:29:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3538de220 con 0x55d353877500 2023-10-18T12:29:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3538de360 con 0x55d3538715d0 2023-10-18T12:29:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c1c656b80 10 client.0 _ll_register_callbacks cb 0x55d3536aaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d35387fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c01d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d35387fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43130/0 (socket says 172.21.15.70:43130) 2023-10-18T12:29:39.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c01d9b700 1 -- 172.21.15.70:0/282174797 learned_addr learned my addr 172.21.15.70:0/282174797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:39.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353877500 0x55d3538ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.672+0000 7f4c0259c700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d35387f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c02d9d700 1 -- 172.21.15.70:0/282174797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 msgr2=0x55d35387f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d35387f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c02d9d700 1 -- 172.21.15.70:0/282174797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 msgr2=0x55d35387fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d35387fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c02d9d700 1 -- 172.21.15.70:0/282174797 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf80095f0 con 0x55d353877500 2023-10-18T12:29:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c01d9b700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d35387fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c02d9d700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353877500 0x55d3538ddce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0ca4a0 tx=0x7f4bfc0997f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56a0fcc16c0abd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4be9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:29:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4be9ffb700 1 -- 172.21.15.70:0/282174797 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0048a0 con 0x55d353877500 2023-10-18T12:29:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4be9ffb700 1 -- 172.21.15.70:0/282174797 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4bfc09df10 con 0x55d353877500 2023-10-18T12:29:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d353872040 con 0x55d353877500 2023-10-18T12:29:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.673+0000 7f4be9ffb700 1 -- 172.21.15.70:0/282174797 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bfc0a20c0 con 0x55d353877500 2023-10-18T12:29:39.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 10 client.26066 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:39.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d3535eede0 con 0x55d353877500 2023-10-18T12:29:39.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4be9ffb700 1 -- 172.21.15.70:0/282174797 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d353872040 con 0x55d353877500 2023-10-18T12:29:39.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4be9ffb700 1 -- 172.21.15.70:0/282174797 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4bfc004bd0 con 0x55d353877500 2023-10-18T12:29:39.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 10 client.26066 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:39.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 20 client.26066 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d3538dfb00 0x55d3538e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 4 client.26066 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:29:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f4bf401e8e0 con 0x55d3538dfb00 2023-10-18T12:29:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.674+0000 7f4c0259c700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d3538dfb00 0x55d3538e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.675+0000 7f4c0259c700 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d3538dfb00 0x55d3538e1f20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e072b5f8144a3a0d server_cookie=3de3b31468760435 in_seq=0 out_seq=0 2023-10-18T12:29:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.675+0000 7f4be9ffb700 10 client.26066 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.675+0000 7f4be9ffb700 1 -- 172.21.15.70:0/282174797 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f4bf401e8e0 con 0x55d3538dfb00 2023-10-18T12:29:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.675+0000 7f4be9ffb700 10 client.26066 handle_command_reply: tid=0 2023-10-18T12:29:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.675+0000 7f4c1c656b80 1 client.26066 shutdown 2023-10-18T12:29:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353877500 msgr2=0x55d3538ddce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353877500 0x55d3538ddce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0ca4a0 tx=0x7f4bfc0997f0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 shutdown_connections 2023-10-18T12:29:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d3538dfb00 0x55d3538e1f20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d353877500 0x55d3538ddce0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d3538715d0 0x55d35387f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 --2- 172.21.15.70:0/282174797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d353876be0 0x55d35387fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 >> 172.21.15.70:0/282174797 conn(0x55d3537e5720 msgr2=0x55d35387b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4be9ffb700 0 client.26066 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 shutdown_connections 2023-10-18T12:29:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 1 -- 172.21.15.70:0/282174797 wait complete. 2023-10-18T12:29:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.676+0000 7f4c1c656b80 20 client.26066 trim_cache size 0 max 16384 2023-10-18T12:29:39.693 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:29:39.694 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:29:39.694 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:29:39.694 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:29:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4bec0a15f0 msgr2=0x7f4bec0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4bec0a15f0 0x7f4bec0a3ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009d70 tx=0x7f4bf4009260 comp rx=0 tx=0).stop 2023-10-18T12:29:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106720 msgr2=0x7f4c0419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106720 0x7f4c0419d6a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00efb0 tx=0x7f4bfc00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 shutdown_connections 2023-10-18T12:29:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106720 0x7f4c0419d6a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c04100710 0x7f4c04198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4bec0a15f0 0x7f4bec0a3ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 --2- 172.21.15.70:0/4003018982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c041010d0 0x7f4c04199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 >> 172.21.15.70:0/4003018982 conn(0x7f4c040fc200 msgr2=0x7f4c0410be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 shutdown_connections 2023-10-18T12:29:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.677+0000 7f4c08dff700 1 -- 172.21.15.70:0/4003018982 wait complete. 2023-10-18T12:29:39.699 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:29:39.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:29:39.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:39 smithi186 ceph-mon[135253]: pgmap v423: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 265 op/s 2023-10-18T12:29:39.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 330360 14.29 MB/sec execute 8 sec latency 52.106 ms 2023-10-18T12:29:39.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 -- 172.21.15.70:0/2537337699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 msgr2=0x7fd1700ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 --2- 172.21.15.70:0/2537337699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd1700ff620 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd160009990 tx=0x7fd160009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 -- 172.21.15.70:0/2537337699 shutdown_connections 2023-10-18T12:29:39.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 --2- 172.21.15.70:0/2537337699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd17010cdf0 0x7fd17010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 --2- 172.21.15.70:0/2537337699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd1700ff620 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 --2- 172.21.15.70:0/2537337699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17010c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 -- 172.21.15.70:0/2537337699 >> 172.21.15.70:0/2537337699 conn(0x7fd1700fae80 msgr2=0x7fd1700fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 -- 172.21.15.70:0/2537337699 shutdown_connections 2023-10-18T12:29:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.971+0000 7fd16ffff700 1 -- 172.21.15.70:0/2537337699 wait complete. 2023-10-18T12:29:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.972+0000 7fd16ffff700 1 Processor -- start 2023-10-18T12:29:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 -- start start 2023-10-18T12:29:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd17019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd17010cdf0 0x7fd170198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170198fc0 con 0x7fd1700ff240 2023-10-18T12:29:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170199130 con 0x7fd17010cdf0 2023-10-18T12:29:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701992f0 con 0x7fd1700ffbf0 2023-10-18T12:29:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd17019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53774/0 (socket says 172.21.15.70:53774) 2023-10-18T12:29:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16e7fc700 1 -- 172.21.15.70:0/555856171 learned_addr learned my addr 172.21.15.70:0/555856171 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.973+0000 7fd16f7fe700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd17010cdf0 0x7fd170198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16e7fc700 1 -- 172.21.15.70:0/555856171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd17010cdf0 msgr2=0x7fd170198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16e7fc700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd17010cdf0 0x7fd170198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16e7fc700 1 -- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 msgr2=0x7fd17019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16e7fc700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd17019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16e7fc700 1 -- 172.21.15.70:0/555856171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009c40 con 0x7fd1700ffbf0 2023-10-18T12:29:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16effd700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd17019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16e7fc700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17019ed70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e9a0 tx=0x7fd16400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164009790 con 0x7fd1700ffbf0 2023-10-18T12:29:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1600095f0 con 0x7fd1700ffbf0 2023-10-18T12:29:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd164010430 con 0x7fd1700ffbf0 2023-10-18T12:29:39.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164018680 con 0x7fd1700ffbf0 2023-10-18T12:29:39.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.974+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1701997c0 con 0x7fd1700ffbf0 2023-10-18T12:29:39.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.975+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170108010 con 0x7fd1700ffbf0 2023-10-18T12:29:39.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.977+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd1640105a0 con 0x7fd1700ffbf0 2023-10-18T12:29:39.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.979+0000 7fd1749fd700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1580a15f0 0x7fd1580a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:39.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.979+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd164014070 con 0x7fd1700ffbf0 2023-10-18T12:29:39.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.979+0000 7fd16effd700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1580a15f0 0x7fd1580a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:39.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.979+0000 7fd16effd700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1580a15f0 0x7fd1580a3ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1600098e0 tx=0x7fd160005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:39.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:39.981+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd1640965f0 con 0x7fd1700ffbf0 2023-10-18T12:29:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.280+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd170199e20 con 0x7fd1700ffbf0 2023-10-18T12:29:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.281+0000 7fd1749fd700 1 -- 172.21.15.70:0/555856171 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fd1640cc090 con 0x7fd1700ffbf0 2023-10-18T12:29:40.284 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:29:40.284 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:29:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1580a15f0 msgr2=0x7fd1580a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1580a15f0 0x7fd1580a3ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1600098e0 tx=0x7fd160005960 comp rx=0 tx=0).stop 2023-10-18T12:29:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 msgr2=0x7fd17019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17019ed70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e9a0 tx=0x7fd16400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 shutdown_connections 2023-10-18T12:29:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1580a15f0 0x7fd1580a3ab0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd17010cdf0 0x7fd170198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1700ff240 0x7fd17019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 --2- 172.21.15.70:0/555856171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1700ffbf0 0x7fd17019ed70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 >> 172.21.15.70:0/555856171 conn(0x7fd1700fae80 msgr2=0x7fd1700fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 shutdown_connections 2023-10-18T12:29:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.284+0000 7fd16ffff700 1 -- 172.21.15.70:0/555856171 wait complete. 2023-10-18T12:29:40.290 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:29:40.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:29:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.531+0000 7f93bcbff700 1 -- 172.21.15.70:0/1544914878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8106730 msgr2=0x7f93b8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.531+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1544914878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8106730 0x7f93b8106b10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009990 tx=0x7f93a8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 -- 172.21.15.70:0/1544914878 shutdown_connections 2023-10-18T12:29:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1544914878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1544914878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8106730 0x7f93b8106b10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1544914878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b8100720 0x7f93b8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 -- 172.21.15.70:0/1544914878 >> 172.21.15.70:0/1544914878 conn(0x7f93b80fc1b0 msgr2=0x7f93b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 -- 172.21.15.70:0/1544914878 shutdown_connections 2023-10-18T12:29:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.532+0000 7f93bcbff700 1 -- 172.21.15.70:0/1544914878 wait complete. 2023-10-18T12:29:40.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 Processor -- start 2023-10-18T12:29:40.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 -- start start 2023-10-18T12:29:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8100720 0x7f93b8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b8106730 0x7f93b819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819dc90 con 0x7f93b8100720 2023-10-18T12:29:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819de00 con 0x7f93b81010e0 2023-10-18T12:29:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93bcbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b819dfc0 con 0x7f93b8106730 2023-10-18T12:29:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93b5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93b5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60122/0 (socket says 172.21.15.70:60122) 2023-10-18T12:29:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.533+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1286422657 learned_addr learned my addr 172.21.15.70:0/1286422657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b659c700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8100720 0x7f93b8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b6d9d700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b8106730 0x7f93b819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1286422657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b8106730 msgr2=0x7f93b819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b8106730 0x7f93b819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8100720 msgr2=0x7f93b8198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8100720 0x7f93b8198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1286422657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80095f0 con 0x7f93b81010e0 2023-10-18T12:29:40.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b8199300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9a0 tx=0x7f93ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:40.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.534+0000 7f93a77fe700 1 -- 172.21.15.70:0/1286422657 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac009790 con 0x7f93b81010e0 2023-10-18T12:29:40.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.535+0000 7f93a77fe700 1 -- 172.21.15.70:0/1286422657 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac00cbd0 con 0x7f93b81010e0 2023-10-18T12:29:40.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.535+0000 7f93a77fe700 1 -- 172.21.15.70:0/1286422657 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac018680 con 0x7f93b81010e0 2023-10-18T12:29:40.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.535+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b819e2a0 con 0x7f93b81010e0 2023-10-18T12:29:40.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.535+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81a5ae0 con 0x7f93b81010e0 2023-10-18T12:29:40.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.539+0000 7f93a77fe700 1 -- 172.21.15.70:0/1286422657 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f93ac00cd40 con 0x7f93b81010e0 2023-10-18T12:29:40.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.540+0000 7f93a77fe700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93a00a1670 0x7f93a00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.540+0000 7f93d046ab80 1 Processor -- start 2023-10-18T12:29:40.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.540+0000 7f93a77fe700 1 -- 172.21.15.70:0/1286422657 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93ac014070 con 0x7f93b81010e0 2023-10-18T12:29:40.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 -- start start 2023-10-18T12:29:40.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93b659c700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93a00a1670 0x7f93a00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 0x563548a0cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a075d0 0x563548a07a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56354897a6e0 con 0x563548a0d500 2023-10-18T12:29:40.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548960c60 con 0x563548a0cbe0 2023-10-18T12:29:40.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93d046ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56354895e580 con 0x563548a075d0 2023-10-18T12:29:40.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93b659c700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93a00a1670 0x7f93a00a3b30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b2e0 tx=0x7f93a8005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:40.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93b659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 0x563548a0cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93b659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 0x563548a0cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60130/0 (socket says 172.21.15.70:60130) 2023-10-18T12:29:40.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.541+0000 7f93b659c700 1 -- 172.21.15.70:0/2403579763 learned_addr learned my addr 172.21.15.70:0/2403579763 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:40.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b6d9d700 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a075d0 0x563548a07a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b659c700 1 -- 172.21.15.70:0/2403579763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a075d0 msgr2=0x563548a07a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b659c700 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a075d0 0x563548a07a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b659c700 1 -- 172.21.15.70:0/2403579763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 msgr2=0x563548a07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b659c700 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a07090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b659c700 1 -- 172.21.15.70:0/2403579763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635489173f0 con 0x563548a0cbe0 2023-10-18T12:29:40.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:29:40.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93b659c700 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 0x563548a0cfc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005640 tx=0x7f93a8020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76fca2b92c4db86b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:40.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93a4ff9700 1 -- 172.21.15.70:0/2403579763 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004500 con 0x563548a0cbe0 2023-10-18T12:29:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93a4ff9700 1 -- 172.21.15.70:0/2403579763 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a801e040 con 0x563548a0cbe0 2023-10-18T12:29:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.542+0000 7f93a4ff9700 1 -- 172.21.15.70:0/2403579763 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800fad0 con 0x563548a0cbe0 2023-10-18T12:29:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 -- 172.21.15.70:0/2403579763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 msgr2=0x563548a0cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 0x563548a0cfc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005640 tx=0x7f93a8020000 comp rx=0 tx=0).stop 2023-10-18T12:29:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 -- 172.21.15.70:0/2403579763 shutdown_connections 2023-10-18T12:29:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a075d0 0x563548a07a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a0cbe0 0x563548a0cfc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 --2- 172.21.15.70:0/2403579763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 -- 172.21.15.70:0/2403579763 >> 172.21.15.70:0/2403579763 conn(0x56354897b720 msgr2=0x563548a01b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 -- 172.21.15.70:0/2403579763 shutdown_connections 2023-10-18T12:29:40.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.543+0000 7f93d046ab80 1 -- 172.21.15.70:0/2403579763 wait complete. 2023-10-18T12:29:40.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.544+0000 7f93d046ab80 1 -- start start 2023-10-18T12:29:40.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.544+0000 7f93d046ab80 1 Processor -- start 2023-10-18T12:29:40.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a13420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a10d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548959e70 con 0x563548a0d500 2023-10-18T12:29:40.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635488f6710 con 0x563548a075d0 2023-10-18T12:29:40.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635488f63b0 con 0x563548a0cbe0 2023-10-18T12:29:40.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93d046ab80 10 client.0 _ll_register_callbacks cb 0x563548840a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:40.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93b6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a10d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93b659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a13420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93b5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93b6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a10d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43184/0 (socket says 172.21.15.70:43184) 2023-10-18T12:29:40.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93b659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a13420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60136/0 (socket says 172.21.15.70:60136) 2023-10-18T12:29:40.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.545+0000 7f93b6d9d700 1 -- 172.21.15.70:0/3719153835 learned_addr learned my addr 172.21.15.70:0/3719153835 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:40.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b659c700 1 -- 172.21.15.70:0/3719153835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 msgr2=0x563548a10710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b659c700 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a10710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b659c700 1 -- 172.21.15.70:0/3719153835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 msgr2=0x563548a10d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b659c700 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a10d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b659c700 1 -- 172.21.15.70:0/3719153835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b810def0 con 0x563548a075d0 2023-10-18T12:29:40.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a10710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:40.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f93b659c700 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a13420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93a8031040 tx=0x7f93a800fb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9268ddac7bc1bb16 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:40.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f939dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:29:40.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.546+0000 7f939dffb700 1 -- 172.21.15.70:0/3719153835 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004700 con 0x563548a075d0 2023-10-18T12:29:40.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f939dffb700 1 -- 172.21.15.70:0/3719153835 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93a8004860 con 0x563548a075d0 2023-10-18T12:29:40.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56354895eef0 con 0x563548a075d0 2023-10-18T12:29:40.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f939dffb700 1 -- 172.21.15.70:0/3719153835 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a8004a40 con 0x563548a075d0 2023-10-18T12:29:40.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 10 client.26084 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:40.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563548784de0 con 0x563548a075d0 2023-10-18T12:29:40.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f939dffb700 1 -- 172.21.15.70:0/3719153835 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56354895eef0 con 0x563548a075d0 2023-10-18T12:29:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f939dffb700 1 -- 172.21.15.70:0/3719153835 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93a8038c10 con 0x563548a075d0 2023-10-18T12:29:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 10 client.26084 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 20 client.26084 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a207e0 0x563548a22bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 4 client.26084 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:40.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.547+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635489d68b0 con 0x563548a207e0 2023-10-18T12:29:40.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.548+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a207e0 0x563548a22bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.548+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a207e0 0x563548a22bd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3911540e5d067587 server_cookie=f577549f84cfe287 in_seq=0 out_seq=0 2023-10-18T12:29:40.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.548+0000 7f939dffb700 10 client.26084 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:40.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f939dffb700 1 -- 172.21.15.70:0/3719153835 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635489d68b0 con 0x563548a207e0 2023-10-18T12:29:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f939dffb700 10 client.26084 handle_command_reply: tid=0 2023-10-18T12:29:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 client.26084 shutdown 2023-10-18T12:29:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 msgr2=0x563548a13420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a13420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93a8031040 tx=0x7f93a800fb70 comp rx=0 tx=0).stop 2023-10-18T12:29:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 shutdown_connections 2023-10-18T12:29:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a207e0 0x563548a22bd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a10d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a13420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 --2- 172.21.15.70:0/3719153835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a10710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f939dffb700 0 client.26084 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.549+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 >> 172.21.15.70:0/3719153835 conn(0x56354897b720 msgr2=0x563548a06080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.550+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 shutdown_connections 2023-10-18T12:29:40.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.550+0000 7f93d046ab80 1 -- 172.21.15.70:0/3719153835 wait complete. 2023-10-18T12:29:40.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.550+0000 7f93d046ab80 20 client.26084 trim_cache size 0 max 16384 2023-10-18T12:29:40.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 Processor -- start 2023-10-18T12:29:40.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 -- start start 2023-10-18T12:29:40.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a17df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635487da920 con 0x563548a0d500 2023-10-18T12:29:40.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548a02100 con 0x563548a075d0 2023-10-18T12:29:40.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.564+0000 7f93d046ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548a18380 con 0x563548a0cbe0 2023-10-18T12:29:40.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43188/0 (socket says 172.21.15.70:43188) 2023-10-18T12:29:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b6d9d700 1 -- 172.21.15.70:0/1979307197 learned_addr learned my addr 172.21.15.70:0/1979307197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a17df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1979307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 msgr2=0x563548a03740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a03740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1979307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 msgr2=0x563548a079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1979307197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56354876dc60 con 0x563548a0cbe0 2023-10-18T12:29:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b6d9d700 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a17df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0cac50 tx=0x7f93ac099ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfab7d3568dc169 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.565+0000 7f93a57fa700 1 -- 172.21.15.70:0/1979307197 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac09abe0 con 0x563548a0cbe0 2023-10-18T12:29:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93a57fa700 1 -- 172.21.15.70:0/1979307197 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac09a4e0 con 0x563548a0cbe0 2023-10-18T12:29:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93a57fa700 1 -- 172.21.15.70:0/1979307197 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac09a9a0 con 0x563548a0cbe0 2023-10-18T12:29:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 -- 172.21.15.70:0/1979307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 msgr2=0x563548a17df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a17df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0cac50 tx=0x7f93ac099ee0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 -- 172.21.15.70:0/1979307197 shutdown_connections 2023-10-18T12:29:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0d500 0x563548a079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1979307197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0cbe0 0x563548a17df0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 -- 172.21.15.70:0/1979307197 >> 172.21.15.70:0/1979307197 conn(0x56354897b720 msgr2=0x563548a06080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 -- 172.21.15.70:0/1979307197 shutdown_connections 2023-10-18T12:29:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.566+0000 7f93d046ab80 1 -- 172.21.15.70:0/1979307197 wait complete. 2023-10-18T12:29:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.567+0000 7f93d046ab80 1 -- start start 2023-10-18T12:29:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.567+0000 7f93d046ab80 1 Processor -- start 2023-10-18T12:29:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.567+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.567+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 0x563548a0e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.567+0000 7f93d046ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0d500 0x563548a735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93d046ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548a73ae0 con 0x563548a0cbe0 2023-10-18T12:29:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93d046ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548a73c20 con 0x563548a075d0 2023-10-18T12:29:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93d046ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563548a73d90 con 0x563548a0d500 2023-10-18T12:29:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93d046ab80 10 client.0 _ll_register_callbacks cb 0x563548840a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 0x563548a0e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 0x563548a0e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43190/0 (socket says 172.21.15.70:43190) 2023-10-18T12:29:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1623669198 learned_addr learned my addr 172.21.15.70:0/1623669198 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b6d9d700 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0d500 0x563548a735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1623669198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0d500 msgr2=0x563548a735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0d500 0x563548a735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 msgr2=0x563548a0dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 -- 172.21.15.70:0/1623669198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b0008450 con 0x563548a0cbe0 2023-10-18T12:29:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f93b5d9b700 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 0x563548a0e010 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0a2340 tx=0x7f93ac09abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69ade2648755740 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.568+0000 7f939dffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:29:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f939dffb700 1 -- 172.21.15.70:0/1623669198 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac098880 con 0x563548a0cbe0 2023-10-18T12:29:40.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a801e040 con 0x563548a0cbe0 2023-10-18T12:29:40.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f939dffb700 1 -- 172.21.15.70:0/1623669198 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93ac0999d0 con 0x563548a0cbe0 2023-10-18T12:29:40.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f939dffb700 1 -- 172.21.15.70:0/1623669198 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ac099b90 con 0x563548a0cbe0 2023-10-18T12:29:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 10 client.16065 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f93b81a5ae0 con 0x563548a0cbe0 2023-10-18T12:29:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f939dffb700 1 -- 172.21.15.70:0/1623669198 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a801e040 con 0x563548a0cbe0 2023-10-18T12:29:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f939dffb700 1 -- 172.21.15.70:0/1623669198 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93ac0a2500 con 0x563548a0cbe0 2023-10-18T12:29:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 10 client.16065 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 20 client.16065 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a75360 0x563548a77780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 4 client.16065 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:29:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.569+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563548952e50 con 0x563548a75360 2023-10-18T12:29:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.570+0000 7f93b659c700 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a75360 0x563548a77780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.570+0000 7f939dffb700 10 client.16065 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.570+0000 7f93b659c700 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a75360 0x563548a77780 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ab9496f0a593e6 server_cookie=5a72576aa7ca694e in_seq=0 out_seq=0 2023-10-18T12:29:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f939dffb700 1 -- 172.21.15.70:0/1623669198 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563548952e50 con 0x563548a75360 2023-10-18T12:29:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f939dffb700 10 client.16065 handle_command_reply: tid=0 2023-10-18T12:29:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 client.16065 shutdown 2023-10-18T12:29:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 msgr2=0x563548a0e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 0x563548a0e010 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0a2340 tx=0x7f93ac09abe0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 shutdown_connections 2023-10-18T12:29:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563548a75360 0x563548a77780 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563548a0d500 0x563548a735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563548a075d0 0x563548a0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 --2- 172.21.15.70:0/1623669198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563548a0cbe0 0x563548a0e010 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f939dffb700 0 client.16065 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 >> 172.21.15.70:0/1623669198 conn(0x56354897b720 msgr2=0x563548a079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 shutdown_connections 2023-10-18T12:29:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 1 -- 172.21.15.70:0/1623669198 wait complete. 2023-10-18T12:29:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.572+0000 7f93d046ab80 20 client.16065 trim_cache size 0 max 16384 2023-10-18T12:29:40.587 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:29:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93a00a1670 msgr2=0x7f93a00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93a00a1670 0x7f93a00a3b30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b2e0 tx=0x7f93a8005fd0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 msgr2=0x7f93b8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b8199300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e9a0 tx=0x7f93ac00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 shutdown_connections 2023-10-18T12:29:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b8106730 0x7f93b819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8100720 0x7f93b8198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93a00a1670 0x7f93a00a3b30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 --2- 172.21.15.70:0/1286422657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93b81010e0 0x7f93b8199300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 >> 172.21.15.70:0/1286422657 conn(0x7f93b80fc1b0 msgr2=0x7f93b80fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:40.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 shutdown_connections 2023-10-18T12:29:40.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:40.573+0000 7f93bcbff700 1 -- 172.21.15.70:0/1286422657 wait complete. 2023-10-18T12:29:40.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 332476 13.60 MB/sec execute 9 sec latency 58.865 ms 2023-10-18T12:29:41.597 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:29:41.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:29:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:41 smithi070 ceph-mon[98639]: pgmap v424: 129 pgs: 129 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2023-10-18T12:29:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/555856171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:29:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:41 smithi152 ceph-mon[134767]: pgmap v424: 129 pgs: 129 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2023-10-18T12:29:41.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/555856171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:29:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:41 smithi186 ceph-mon[135253]: pgmap v424: 129 pgs: 129 active+clean; 180 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2023-10-18T12:29:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/555856171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:29:41.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 334853 13.85 MB/sec execute 10 sec latency 33.380 ms 2023-10-18T12:29:41.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 -- 172.21.15.70:0/319616154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af8100000 msgr2=0x7f0af810c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/319616154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af8100000 0x7f0af810c4a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009f90 tx=0x7f0af000b330 comp rx=0 tx=0).stop 2023-10-18T12:29:41.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 -- 172.21.15.70:0/319616154 shutdown_connections 2023-10-18T12:29:41.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/319616154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af8100000 0x7f0af810c4a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/319616154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af81056e0 0x7f0af8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/319616154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af80ff640 0x7f0af80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 -- 172.21.15.70:0/319616154 >> 172.21.15.70:0/319616154 conn(0x7f0af80767f0 msgr2=0x7f0af8076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:41.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 -- 172.21.15.70:0/319616154 shutdown_connections 2023-10-18T12:29:41.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.843+0000 7f0aff5ff700 1 -- 172.21.15.70:0/319616154 wait complete. 2023-10-18T12:29:41.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.844+0000 7f0aff5ff700 1 Processor -- start 2023-10-18T12:29:41.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 -- start start 2023-10-18T12:29:41.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af80ff640 0x7f0af8198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 0x7f0af8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af81056e0 0x7f0af819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af819ddf0 con 0x7f0af81056e0 2023-10-18T12:29:41.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af819df60 con 0x7f0af8100000 2023-10-18T12:29:41.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0aff5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af819e120 con 0x7f0af80ff640 2023-10-18T12:29:41.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 0x7f0af8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0afdb9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af81056e0 0x7f0af819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 0x7f0af8199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60156/0 (socket says 172.21.15.70:60156) 2023-10-18T12:29:41.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0afcb9a700 1 -- 172.21.15.70:0/1048563500 learned_addr learned my addr 172.21.15.70:0/1048563500 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:41.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.845+0000 7f0afd39b700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af80ff640 0x7f0af8198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afcb9a700 1 -- 172.21.15.70:0/1048563500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af80ff640 msgr2=0x7f0af8198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af80ff640 0x7f0af8198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afcb9a700 1 -- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af81056e0 msgr2=0x7f0af819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af81056e0 0x7f0af819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afcb9a700 1 -- 172.21.15.70:0/1048563500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009520 con 0x7f0af8100000 2023-10-18T12:29:41.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afd39b700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af80ff640 0x7f0af8198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:29:41.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 0x7f0af8199410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800eb30 tx=0x7f0ae800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:41.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0aee7fc700 1 -- 172.21.15.70:0/1048563500 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8009880 con 0x7f0af8100000 2023-10-18T12:29:41.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0aee7fc700 1 -- 172.21.15.70:0/1048563500 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae80044e0 con 0x7f0af8100000 2023-10-18T12:29:41.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af40095f0 con 0x7f0af8100000 2023-10-18T12:29:41.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0aee7fc700 1 -- 172.21.15.70:0/1048563500 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8003e90 con 0x7f0af8100000 2023-10-18T12:29:41.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.846+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af0009c40 con 0x7f0af8100000 2023-10-18T12:29:41.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.850+0000 7f0aee7fc700 1 -- 172.21.15.70:0/1048563500 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0ae8004000 con 0x7f0af8100000 2023-10-18T12:29:41.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0b12fc1b80 1 Processor -- start 2023-10-18T12:29:41.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0aee7fc700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0ae40a15f0 0x7f0ae40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0b12fc1b80 1 -- start start 2023-10-18T12:29:41.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0aee7fc700 1 -- 172.21.15.70:0/1048563500 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ae8014070 con 0x7f0af8100000 2023-10-18T12:29:41.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0afdb9c700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0ae40a15f0 0x7f0ae40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f47025110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.851+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47025ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f98760 con 0x564f47025650 2023-10-18T12:29:41.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f7ece0 con 0x564f4702ac60 2023-10-18T12:29:41.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f7c600 con 0x564f4702b580 2023-10-18T12:29:41.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60170/0 (socket says 172.21.15.70:60170) 2023-10-18T12:29:41.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 -- 172.21.15.70:0/177016041 learned_addr learned my addr 172.21.15.70:0/177016041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:41.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f47025110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 -- 172.21.15.70:0/177016041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 msgr2=0x564f47025110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f47025110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 -- 172.21.15.70:0/177016041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 msgr2=0x564f47025ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47025ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.852+0000 7f0afd39b700 1 -- 172.21.15.70:0/177016041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f46f35470 con 0x564f4702ac60 2023-10-18T12:29:41.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.853+0000 7f0afdb9c700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0ae40a15f0 0x7f0ae40a3ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009790 tx=0x7f0af00075e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:41.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.853+0000 7f0afd39b700 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702b040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009f70 tx=0x7f0af40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850bb09ff0c54244 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:41.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.853+0000 7f0ae37fe700 1 -- 172.21.15.70:0/177016041 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4011ee0 con 0x564f4702ac60 2023-10-18T12:29:41.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.853+0000 7f0ae37fe700 1 -- 172.21.15.70:0/177016041 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af4004cf0 con 0x564f4702ac60 2023-10-18T12:29:41.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.853+0000 7f0ae37fe700 1 -- 172.21.15.70:0/177016041 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40126b0 con 0x564f4702ac60 2023-10-18T12:29:41.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/177016041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 msgr2=0x564f4702b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702b040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009f70 tx=0x7f0af40116c0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/177016041 shutdown_connections 2023-10-18T12:29:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47025ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702b040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/177016041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f47025110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/177016041 >> 172.21.15.70:0/177016041 conn(0x564f46f997a0 msgr2=0x564f4701fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/177016041 shutdown_connections 2023-10-18T12:29:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/177016041 wait complete. 2023-10-18T12:29:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.854+0000 7f0b12fc1b80 1 -- start start 2023-10-18T12:29:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 Processor -- start 2023-10-18T12:29:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47031450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4702ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f77ef0 con 0x564f47025650 2023-10-18T12:29:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f14790 con 0x564f4702ac60 2023-10-18T12:29:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f14430 con 0x564f4702b580 2023-10-18T12:29:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.855+0000 7f0b12fc1b80 10 client.0 _ll_register_callbacks cb 0x564f46e5eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afd39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47031450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60182/0 (socket says 172.21.15.70:60182) 2023-10-18T12:29:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 -- 172.21.15.70:0/4176009284 learned_addr learned my addr 172.21.15.70:0/4176009284 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afdb9c700 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4702ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 -- 172.21.15.70:0/4176009284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 msgr2=0x564f4702ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4702ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 -- 172.21.15.70:0/4176009284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 msgr2=0x564f47031450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47031450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 -- 172.21.15.70:0/4176009284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af4009bd0 con 0x564f4702ac60 2023-10-18T12:29:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702e790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8097000 tx=0x7f0ae809a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb2f99de17b748b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0ae0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:29:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/4176009284 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae809a9f0 con 0x564f4702ac60 2023-10-18T12:29:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/4176009284 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ae800cc90 con 0x564f4702ac60 2023-10-18T12:29:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.856+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f46f7cf70 con 0x564f4702ac60 2023-10-18T12:29:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/4176009284 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae809a590 con 0x564f4702ac60 2023-10-18T12:29:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 10 client.26099 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564f46da2e60 con 0x564f4702ac60 2023-10-18T12:29:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/4176009284 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f46f7cf70 con 0x564f4702ac60 2023-10-18T12:29:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/4176009284 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ae80a2490 con 0x564f4702ac60 2023-10-18T12:29:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 10 client.26099 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 20 client.26099 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f4703ee50 0x564f47041270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 4 client.26099 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f46ff4930 con 0x564f4703ee50 2023-10-18T12:29:41.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.857+0000 7f0afd39b700 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f4703ee50 0x564f47041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.858+0000 7f0afd39b700 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f4703ee50 0x564f47041270 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c883e94d76877712 server_cookie=8a9c30387dd5338e in_seq=0 out_seq=0 2023-10-18T12:29:41.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.858+0000 7f0ae0ff9700 10 client.26099 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:41.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/4176009284 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f46ff4930 con 0x564f4703ee50 2023-10-18T12:29:41.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0ae0ff9700 10 client.26099 handle_command_reply: tid=0 2023-10-18T12:29:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 client.26099 shutdown 2023-10-18T12:29:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 msgr2=0x564f4702e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702e790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8097000 tx=0x7f0ae809a1c0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 shutdown_connections 2023-10-18T12:29:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f4703ee50 0x564f47041270 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4702ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47031450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/4176009284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f4702e790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0ae0ff9700 0 client.26099 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 >> 172.21.15.70:0/4176009284 conn(0x564f46f997a0 msgr2=0x564f47024100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 shutdown_connections 2023-10-18T12:29:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/4176009284 wait complete. 2023-10-18T12:29:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.859+0000 7f0b12fc1b80 20 client.26099 trim_cache size 0 max 16384 2023-10-18T12:29:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.879+0000 7f0b12fc1b80 1 Processor -- start 2023-10-18T12:29:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.879+0000 7f0b12fc1b80 1 -- start start 2023-10-18T12:29:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47028dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4703a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46df89a0 con 0x564f47025650 2023-10-18T12:29:41.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46f7cf70 con 0x564f4702ac60 2023-10-18T12:29:41.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4703c560 con 0x564f4702b580 2023-10-18T12:29:41.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afd39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47028dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60196/0 (socket says 172.21.15.70:60196) 2023-10-18T12:29:41.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 -- 172.21.15.70:0/3181341200 learned_addr learned my addr 172.21.15.70:0/3181341200 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:41.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afdb9c700 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4703a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 -- 172.21.15.70:0/3181341200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 msgr2=0x564f4703a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4703a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 -- 172.21.15.70:0/3181341200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 msgr2=0x564f47028dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47028dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afcb9a700 1 -- 172.21.15.70:0/3181341200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f46d8bce0 con 0x564f4702ac60 2023-10-18T12:29:41.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.880+0000 7f0afd39b700 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47028dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:41.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.881+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ae809a290 tx=0x7f0ae8098b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728ccd8e29e4d2ca server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:41.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.881+0000 7f0ae3fff700 1 -- 172.21.15.70:0/3181341200 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800cc90 con 0x564f4702ac60 2023-10-18T12:29:41.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.881+0000 7f0ae3fff700 1 -- 172.21.15.70:0/3181341200 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae80b6060 con 0x564f4702ac60 2023-10-18T12:29:41.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.881+0000 7f0ae3fff700 1 -- 172.21.15.70:0/3181341200 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae809a590 con 0x564f4702ac60 2023-10-18T12:29:41.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.881+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/3181341200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 msgr2=0x564f47035f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.881+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ae809a290 tx=0x7f0ae8098b40 comp rx=0 tx=0).stop 2023-10-18T12:29:41.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/3181341200 shutdown_connections 2023-10-18T12:29:41.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4703a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f47028dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/3181341200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/3181341200 >> 172.21.15.70:0/3181341200 conn(0x564f46f997a0 msgr2=0x564f470286c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:41.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/3181341200 shutdown_connections 2023-10-18T12:29:41.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/3181341200 wait complete. 2023-10-18T12:29:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 -- start start 2023-10-18T12:29:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.882+0000 7f0b12fc1b80 1 Processor -- start 2023-10-18T12:29:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f470353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4708cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4708d2f0 con 0x564f47025650 2023-10-18T12:29:41.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4708d460 con 0x564f4702ac60 2023-10-18T12:29:41.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4708d620 con 0x564f4702b580 2023-10-18T12:29:41.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0b12fc1b80 10 client.0 _ll_register_callbacks cb 0x564f46e5eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:41.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0afd39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f470353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0afcb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60200/0 (socket says 172.21.15.70:60200) 2023-10-18T12:29:41.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0afcb9a700 1 -- 172.21.15.70:0/2515815051 learned_addr learned my addr 172.21.15.70:0/2515815051 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:41.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.883+0000 7f0afdb9c700 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4708cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0afcb9a700 1 -- 172.21.15.70:0/2515815051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 msgr2=0x564f4708cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4708cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0afcb9a700 1 -- 172.21.15.70:0/2515815051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 msgr2=0x564f470353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:41.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f470353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0afcb9a700 1 -- 172.21.15.70:0/2515815051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af401e040 con 0x564f4702ac60 2023-10-18T12:29:41.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0afcb9a700 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80851a0 tx=0x7f0ae80a2800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d85953f63fd02ca server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:41.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0ae0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:29:41.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/2515815051 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800ce90 con 0x564f4702ac60 2023-10-18T12:29:41.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0af4004cf0 con 0x564f4702ac60 2023-10-18T12:29:41.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/2515815051 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ae800cc90 con 0x564f4702ac60 2023-10-18T12:29:41.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/2515815051 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae809e6e0 con 0x564f4702ac60 2023-10-18T12:29:41.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0b12fc1b80 10 client.26105 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:41.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.884+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564f46da2e60 con 0x564f4702ac60 2023-10-18T12:29:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/2515815051 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af4004cf0 con 0x564f4702ac60 2023-10-18T12:29:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/2515815051 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ae8096380 con 0x564f4702ac60 2023-10-18T12:29:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0b12fc1b80 10 client.26105 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0b12fc1b80 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0b12fc1b80 20 client.26105 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f470949a0 0x564f47096d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:41.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0b12fc1b80 4 client.26105 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:29:41.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564f46f70ed0 con 0x564f470949a0 2023-10-18T12:29:41.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.885+0000 7f0afd39b700 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f470949a0 0x564f47096d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:41.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.886+0000 7f0afd39b700 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f470949a0 0x564f47096d90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55869fec9cf39fb server_cookie=126d3d82ba0e256c in_seq=0 out_seq=0 2023-10-18T12:29:41.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.886+0000 7f0ae0ff9700 10 client.26105 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:41.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0ae0ff9700 1 -- 172.21.15.70:0/2515815051 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564f46f70ed0 con 0x564f470949a0 2023-10-18T12:29:41.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0ae0ff9700 10 client.26105 handle_command_reply: tid=0 2023-10-18T12:29:41.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 client.26105 shutdown 2023-10-18T12:29:41.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 msgr2=0x564f47035950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80851a0 tx=0x7f0ae80a2800 comp rx=0 tx=0).stop 2023-10-18T12:29:41.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 shutdown_connections 2023-10-18T12:29:41.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f470949a0 0x564f47096d90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f4702b580 0x564f4708cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0ae0ff9700 0 client.26105 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:41.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f47025650 0x564f470353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 --2- 172.21.15.70:0/2515815051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4702ac60 0x564f47035950 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.888+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 >> 172.21.15.70:0/2515815051 conn(0x564f46f997a0 msgr2=0x564f47019530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:41.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 shutdown_connections 2023-10-18T12:29:41.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0b12fc1b80 1 -- 172.21.15.70:0/2515815051 wait complete. 2023-10-18T12:29:41.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0b12fc1b80 20 client.26105 trim_cache size 0 max 16384 2023-10-18T12:29:41.905 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:29:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:29:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "e3c006b6-fec1-4f59-a97a-6f0f8d39e41f", 2023-10-18T12:29:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:29:41.905 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:29:41.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0ae40a15f0 msgr2=0x7f0ae40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0ae40a15f0 0x7f0ae40a3ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009790 tx=0x7f0af00075e0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 msgr2=0x7f0af8199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:41.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 0x7f0af8199410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800eb30 tx=0x7f0ae800ee40 comp rx=0 tx=0).stop 2023-10-18T12:29:41.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 shutdown_connections 2023-10-18T12:29:41.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0af81056e0 0x7f0af819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0af80ff640 0x7f0af8198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0ae40a15f0 0x7f0ae40a3ab0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 --2- 172.21.15.70:0/1048563500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0af8100000 0x7f0af8199410 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:41.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.889+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 >> 172.21.15.70:0/1048563500 conn(0x7f0af80767f0 msgr2=0x7f0af80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:41.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.890+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 shutdown_connections 2023-10-18T12:29:41.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:41.890+0000 7f0aff5ff700 1 -- 172.21.15.70:0/1048563500 wait complete. 2023-10-18T12:29:41.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:29:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 -- 172.21.15.70:0/3436474346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c1088a0 msgr2=0x7fcd8c108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 --2- 172.21.15.70:0/3436474346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c1088a0 0x7fcd8c108c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009990 tx=0x7fcd80009ca0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 -- 172.21.15.70:0/3436474346 shutdown_connections 2023-10-18T12:29:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 --2- 172.21.15.70:0/3436474346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c103250 0x7fcd8c10f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 --2- 172.21.15.70:0/3436474346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c1088a0 0x7fcd8c108c80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 --2- 172.21.15.70:0/3436474346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 0x7fcd8c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.179+0000 7fcd937ff700 1 -- 172.21.15.70:0/3436474346 >> 172.21.15.70:0/3436474346 conn(0x7fcd8c0fe360 msgr2=0x7fcd8c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.180+0000 7fcd937ff700 1 -- 172.21.15.70:0/3436474346 shutdown_connections 2023-10-18T12:29:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.180+0000 7fcd937ff700 1 -- 172.21.15.70:0/3436474346 wait complete. 2023-10-18T12:29:42.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 Processor -- start 2023-10-18T12:29:42.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 -- start start 2023-10-18T12:29:42.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 0x7fcd8c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 0x7fcd8c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c1088a0 0x7fcd8c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c199120 con 0x7fcd8c102890 2023-10-18T12:29:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c199290 con 0x7fcd8c103250 2023-10-18T12:29:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd937ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c199450 con 0x7fcd8c1088a0 2023-10-18T12:29:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd90d9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 0x7fcd8c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd90d9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 0x7fcd8c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60210/0 (socket says 172.21.15.70:60210) 2023-10-18T12:29:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd90d9a700 1 -- 172.21.15.70:0/2485669652 learned_addr learned my addr 172.21.15.70:0/2485669652 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 0x7fcd8c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.181+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c1088a0 0x7fcd8c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd90d9a700 1 -- 172.21.15.70:0/2485669652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c1088a0 msgr2=0x7fcd8c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c1088a0 0x7fcd8c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd90d9a700 1 -- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 msgr2=0x7fcd8c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 0x7fcd8c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd90d9a700 1 -- 172.21.15.70:0/2485669652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800095f0 con 0x7fcd8c103250 2023-10-18T12:29:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 0x7fcd8c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.182+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 0x7fcd8c19eed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e9a0 tx=0x7fcd7800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.183+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/2485669652 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78009790 con 0x7fcd8c103250 2023-10-18T12:29:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.183+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/2485669652 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd78010430 con 0x7fcd8c103250 2023-10-18T12:29:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.183+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/2485669652 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78018680 con 0x7fcd8c103250 2023-10-18T12:29:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.183+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c199730 con 0x7fcd8c103250 2023-10-18T12:29:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.183+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1a5980 con 0x7fcd8c103250 2023-10-18T12:29:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.186+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/2485669652 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcd780105a0 con 0x7fcd8c103250 2023-10-18T12:29:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.188+0000 7fcda7179b80 1 Processor -- start 2023-10-18T12:29:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.188+0000 7fcda7179b80 1 -- start start 2023-10-18T12:29:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd8a7fc700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd7c0a1660 0x7fcd7c0a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bb6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb7500 0x55aa97bb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcda7179b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97b246e0 con 0x55aa97bb7500 2023-10-18T12:29:42.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcda7179b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97b0ac60 con 0x55aa97bb6be0 2023-10-18T12:29:42.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcda7179b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97b08580 con 0x55aa97bb15d0 2023-10-18T12:29:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd8a7fc700 1 -- 172.21.15.70:0/2485669652 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcd78014070 con 0x7fcd8c103250 2023-10-18T12:29:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd90d9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb7500 0x55aa97bb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bb6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bb6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60222/0 (socket says 172.21.15.70:60222) 2023-10-18T12:29:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd9159b700 1 -- 172.21.15.70:0/3215005757 learned_addr learned my addr 172.21.15.70:0/3215005757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:42.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3215005757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 msgr2=0x55aa97bb1a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T12:29:42.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3215005757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 msgr2=0x55aa97bb6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bb6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3215005757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa97ac13f0 con 0x55aa97bb7500 2023-10-18T12:29:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.189+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd7c0a1660 0x7fcd7c0a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.190+0000 7fcd9159b700 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bb6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.190+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb7500 0x55aa97bb1090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ca9b0 tx=0x7fcd78099ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf70a3ab5a89726 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:42.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.190+0000 7fcd777fe700 1 -- 172.21.15.70:0/3215005757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7809a940 con 0x55aa97bb7500 2023-10-18T12:29:42.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.190+0000 7fcd777fe700 1 -- 172.21.15.70:0/3215005757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd7809a240 con 0x55aa97bb7500 2023-10-18T12:29:42.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.190+0000 7fcd777fe700 1 -- 172.21.15.70:0/3215005757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd780cd1f0 con 0x55aa97bb7500 2023-10-18T12:29:42.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 -- 172.21.15.70:0/3215005757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb7500 msgr2=0x55aa97bb1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb7500 0x55aa97bb1090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ca9b0 tx=0x7fcd78099ba0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd7c0a1660 0x7fcd7c0a3b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400f4e0 tx=0x7fcd8400b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:42.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 -- 172.21.15.70:0/3215005757 shutdown_connections 2023-10-18T12:29:42.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bb6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3215005757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb7500 0x55aa97bb1090 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 -- 172.21.15.70:0/3215005757 >> 172.21.15.70:0/3215005757 conn(0x55aa97b25720 msgr2=0x55aa97babb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 -- 172.21.15.70:0/3215005757 shutdown_connections 2023-10-18T12:29:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.191+0000 7fcda7179b80 1 -- 172.21.15.70:0/3215005757 wait complete. 2023-10-18T12:29:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.192+0000 7fcda7179b80 1 -- start start 2023-10-18T12:29:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.192+0000 7fcda7179b80 1 Processor -- start 2023-10-18T12:29:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.192+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb15d0 0x55aa97bbd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.192+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bbb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcda7179b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97b03e70 con 0x55aa97bb15d0 2023-10-18T12:29:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcda7179b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97aa0710 con 0x55aa97bb6be0 2023-10-18T12:29:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcda7179b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97aa03b0 con 0x55aa97bb7500 2023-10-18T12:29:42.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcda7179b80 10 client.0 _ll_register_callbacks cb 0x55aa979eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:42.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb15d0 0x55aa97bbd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60230/0 (socket says 172.21.15.70:60230) 2023-10-18T12:29:42.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3361269675 learned_addr learned my addr 172.21.15.70:0/3361269675 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:42.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bbb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3361269675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 msgr2=0x55aa97bbb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bbb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3361269675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb15d0 msgr2=0x55aa97bbd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb15d0 0x55aa97bbd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd90d9a700 1 -- 172.21.15.70:0/3361269675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c110060 con 0x55aa97bb6be0 2023-10-18T12:29:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.193+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bbb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bba8e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd780caa10 tx=0x7fcd780c4670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305e357330960127 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:29:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd74ff9700 1 -- 172.21.15.70:0/3361269675 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd780c4930 con 0x55aa97bb6be0 2023-10-18T12:29:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa97b08ef0 con 0x55aa97bb6be0 2023-10-18T12:29:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd74ff9700 1 -- 172.21.15.70:0/3361269675 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcd780a25d0 con 0x55aa97bb6be0 2023-10-18T12:29:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd74ff9700 1 -- 172.21.15.70:0/3361269675 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd780cac30 con 0x55aa97bb6be0 2023-10-18T12:29:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcda7179b80 10 client.26117 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fcd8c1a5980 con 0x55aa97bb6be0 2023-10-18T12:29:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd74ff9700 1 -- 172.21.15.70:0/3361269675 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa97b08ef0 con 0x55aa97bb6be0 2023-10-18T12:29:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcd74ff9700 1 -- 172.21.15.70:0/3361269675 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd780bd050 con 0x55aa97bb6be0 2023-10-18T12:29:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcda7179b80 10 client.26117 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.194+0000 7fcda7179b80 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcda7179b80 20 client.26117 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97bcaaf0 0x55aa97bccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcda7179b80 4 client.26117 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:29:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa97b808b0 con 0x55aa97bcaaf0 2023-10-18T12:29:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcd9159b700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97bcaaf0 0x55aa97bccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcd9159b700 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97bcaaf0 0x55aa97bccf10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee73a8d01d0eb5c server_cookie=2dba43a98877fdb3 in_seq=0 out_seq=0 2023-10-18T12:29:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.195+0000 7fcd74ff9700 10 client.26117 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.196+0000 7fcd74ff9700 1 -- 172.21.15.70:0/3361269675 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa97b808b0 con 0x55aa97bcaaf0 2023-10-18T12:29:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.196+0000 7fcd74ff9700 10 client.26117 handle_command_reply: tid=0 2023-10-18T12:29:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.196+0000 7fcda7179b80 1 client.26117 shutdown 2023-10-18T12:29:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 msgr2=0x55aa97bba8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bba8e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd780caa10 tx=0x7fcd780c4670 comp rx=0 tx=0).stop 2023-10-18T12:29:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 shutdown_connections 2023-10-18T12:29:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97bcaaf0 0x55aa97bccf10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bbb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb15d0 0x55aa97bbd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 --2- 172.21.15.70:0/3361269675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb6be0 0x55aa97bba8e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 >> 172.21.15.70:0/3361269675 conn(0x55aa97b25720 msgr2=0x55aa97bb0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcd74ff9700 0 client.26117 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 shutdown_connections 2023-10-18T12:29:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 1 -- 172.21.15.70:0/3361269675 wait complete. 2023-10-18T12:29:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.197+0000 7fcda7179b80 20 client.26117 trim_cache size 0 max 16384 2023-10-18T12:29:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 Processor -- start 2023-10-18T12:29:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 -- start start 2023-10-18T12:29:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bc1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97984920 con 0x55aa97bb6be0 2023-10-18T12:29:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97ba82e0 con 0x55aa97bb15d0 2023-10-18T12:29:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.217+0000 7fcda7179b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97bb53e0 con 0x55aa97bb7500 2023-10-18T12:29:42.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:60238/0 (socket says 172.21.15.70:60238) 2023-10-18T12:29:42.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 -- 172.21.15.70:0/2543254388 learned_addr learned my addr 172.21.15.70:0/2543254388 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:42.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bc1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 -- 172.21.15.70:0/2543254388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 msgr2=0x55aa97bc1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bc1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 -- 172.21.15.70:0/2543254388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 msgr2=0x55aa97bba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 -- 172.21.15.70:0/2543254388 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa9792ede0 con 0x55aa97bb15d0 2023-10-18T12:29:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.218+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b6f0 tx=0x7fcd80005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c5bec643cb3291 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.219+0000 7fcd77fff700 1 -- 172.21.15.70:0/2543254388 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8001d040 con 0x55aa97bb15d0 2023-10-18T12:29:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.219+0000 7fcd77fff700 1 -- 172.21.15.70:0/2543254388 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd8001e040 con 0x55aa97bb15d0 2023-10-18T12:29:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.219+0000 7fcd77fff700 1 -- 172.21.15.70:0/2543254388 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd800056b0 con 0x55aa97bb15d0 2023-10-18T12:29:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.219+0000 7fcda7179b80 1 -- 172.21.15.70:0/2543254388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 msgr2=0x55aa97bb1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.219+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b6f0 tx=0x7fcd80005960 comp rx=0 tx=0).stop 2023-10-18T12:29:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 -- 172.21.15.70:0/2543254388 shutdown_connections 2023-10-18T12:29:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb7500 0x55aa97bc1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb15d0 0x55aa97bb1090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2543254388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 -- 172.21.15.70:0/2543254388 >> 172.21.15.70:0/2543254388 conn(0x55aa97b25720 msgr2=0x55aa97bad3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 -- 172.21.15.70:0/2543254388 shutdown_connections 2023-10-18T12:29:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 -- 172.21.15.70:0/2543254388 wait complete. 2023-10-18T12:29:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 -- start start 2023-10-18T12:29:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.220+0000 7fcda7179b80 1 Processor -- start 2023-10-18T12:29:42.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bc0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bc05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb7500 0x55aa97c1d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97bc0d20 con 0x55aa97bb6be0 2023-10-18T12:29:42.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97c1de60 con 0x55aa97bb7500 2023-10-18T12:29:42.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa97c1e020 con 0x55aa97bb15d0 2023-10-18T12:29:42.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcda7179b80 10 client.0 _ll_register_callbacks cb 0x55aa979eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:29:42.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bc0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcd9159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bc0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53928/0 (socket says 172.21.15.70:53928) 2023-10-18T12:29:42.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcd9159b700 1 -- 172.21.15.70:0/2151469552 learned_addr learned my addr 172.21.15.70:0/2151469552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:29:42.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bc05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.221+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb7500 0x55aa97c1d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd9159b700 1 -- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb7500 msgr2=0x55aa97c1d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb7500 0x55aa97c1d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd9159b700 1 -- 172.21.15.70:0/2151469552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 msgr2=0x55aa97bc05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:29:42.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bc05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd9159b700 1 -- 172.21.15.70:0/2151469552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8000bb00 con 0x55aa97bb15d0 2023-10-18T12:29:42.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd91d9c700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb7500 0x55aa97c1d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:42.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bc05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:29:42.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd9159b700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bc0080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005ac0 tx=0x7fcd80009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cdd6afbe5c1a92b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:29:42.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:29:42.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd74ff9700 1 -- 172.21.15.70:0/2151469552 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8001d070 con 0x55aa97bb15d0 2023-10-18T12:29:42.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd7809a240 con 0x55aa97bb15d0 2023-10-18T12:29:42.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.222+0000 7fcd74ff9700 1 -- 172.21.15.70:0/2151469552 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcd80029e40 con 0x55aa97bb15d0 2023-10-18T12:29:42.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcd74ff9700 1 -- 172.21.15.70:0/2151469552 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd80029380 con 0x55aa97bb15d0 2023-10-18T12:29:42.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 10 client.26122 fetch_fsmap learned FSMap version 20 2023-10-18T12:29:42.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55aa97917c60 con 0x55aa97bb15d0 2023-10-18T12:29:42.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcd74ff9700 1 -- 172.21.15.70:0/2151469552 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd7809a240 con 0x55aa97bb15d0 2023-10-18T12:29:42.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcd74ff9700 1 -- 172.21.15.70:0/2151469552 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd80004b80 con 0x55aa97bb15d0 2023-10-18T12:29:42.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:29:42.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:29:42.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 20 client.26122 populate_metadata read hostname 'smithi070' 2023-10-18T12:29:42.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97c1f300 0x55aa97c216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:29:42.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 4 client.26122 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:29:42.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aa97afce50 con 0x55aa97c1f300 2023-10-18T12:29:42.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.223+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97c1f300 0x55aa97c216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:29:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.224+0000 7fcd90d9a700 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97c1f300 0x55aa97c216f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8631372c79780d8e server_cookie=34f474e2e67aac3c in_seq=0 out_seq=0 2023-10-18T12:29:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.224+0000 7fcd74ff9700 10 client.26122 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcd74ff9700 1 -- 172.21.15.70:0/2151469552 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aa97afce50 con 0x55aa97c1f300 2023-10-18T12:29:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcd74ff9700 10 client.26122 handle_command_reply: tid=0 2023-10-18T12:29:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 client.26122 shutdown 2023-10-18T12:29:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 msgr2=0x55aa97bc0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bc0080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005ac0 tx=0x7fcd80009340 comp rx=0 tx=0).stop 2023-10-18T12:29:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 shutdown_connections 2023-10-18T12:29:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa97c1f300 0x55aa97c216f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa97bb7500 0x55aa97c1d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa97bb15d0 0x55aa97bc0080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcd74ff9700 0 client.26122 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:29:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 --2- 172.21.15.70:0/2151469552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa97bb6be0 0x55aa97bc05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 >> 172.21.15.70:0/2151469552 conn(0x55aa97b25720 msgr2=0x55aa97bab810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 shutdown_connections 2023-10-18T12:29:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 1 -- 172.21.15.70:0/2151469552 wait complete. 2023-10-18T12:29:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.225+0000 7fcda7179b80 20 client.26122 trim_cache size 0 max 16384 2023-10-18T12:29:42.240 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:29:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:29:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:29:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "e3c006b6-fec1-4f59-a97a-6f0f8d39e41f": { 2023-10-18T12:29:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:29:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "e3c006b6-fec1-4f59-a97a-6f0f8d39e41f", 2023-10-18T12:29:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:29:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:29:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:29:42.241 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:29:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd7c0a1660 msgr2=0x7fcd7c0a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd7c0a1660 0x7fcd7c0a3b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400f4e0 tx=0x7fcd8400b040 comp rx=0 tx=0).stop 2023-10-18T12:29:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 msgr2=0x7fcd8c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:29:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 0x7fcd8c19eed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e9a0 tx=0x7fcd7800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 shutdown_connections 2023-10-18T12:29:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd8c1088a0 0x7fcd8c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd8c102890 0x7fcd8c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcd7c0a1660 0x7fcd7c0a3b20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 --2- 172.21.15.70:0/2485669652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd8c103250 0x7fcd8c19eed0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:29:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 >> 172.21.15.70:0/2485669652 conn(0x7fcd8c0fe360 msgr2=0x7fcd8c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:29:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 shutdown_connections 2023-10-18T12:29:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:29:42.226+0000 7fcd937ff700 1 -- 172.21.15.70:0/2485669652 wait complete. 2023-10-18T12:29:42.250 INFO:tasks.cephfs.filesystem:scrub status for tag:e3c006b6-fec1-4f59-a97a-6f0f8d39e41f - {'path': '/', 'tag': 'e3c006b6-fec1-4f59-a97a-6f0f8d39e41f', 'options': 'recursive,force'} 2023-10-18T12:29:42.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:42 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:29:42.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:42 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:42.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:42 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:29:42.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:42 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:42.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:42 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:29:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:42 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:42 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:29:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:42 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:42.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:42 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:29:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:42 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:42 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:29:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:42 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:29:42.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 337576 13.87 MB/sec execute 11 sec latency 6.209 ms 2023-10-18T12:29:43.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:43 smithi070 ceph-mon[98639]: pgmap v425: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2023-10-18T12:29:43.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:43 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:29:43.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:43 smithi152 ceph-mon[134767]: pgmap v425: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2023-10-18T12:29:43.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:43 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:29:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:43 smithi186 ceph-mon[135253]: pgmap v425: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2023-10-18T12:29:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:43 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:29:43.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 340762 14.28 MB/sec execute 12 sec latency 9.999 ms 2023-10-18T12:29:44.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 343439 14.17 MB/sec execute 13 sec latency 22.197 ms 2023-10-18T12:29:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:45 smithi070 ceph-mon[98639]: pgmap v426: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-10-18T12:29:45.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:45 smithi152 ceph-mon[134767]: pgmap v426: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-10-18T12:29:45.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:45 smithi186 ceph-mon[135253]: pgmap v426: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-10-18T12:29:45.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 344933 13.68 MB/sec execute 14 sec latency 38.977 ms 2023-10-18T12:29:46.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 347937 13.98 MB/sec execute 15 sec latency 7.655 ms 2023-10-18T12:29:47.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:47 smithi070 ceph-mon[98639]: pgmap v427: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-10-18T12:29:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:47 smithi152 ceph-mon[134767]: pgmap v427: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-10-18T12:29:47.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:47 smithi186 ceph-mon[135253]: pgmap v427: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-10-18T12:29:47.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 350600 13.91 MB/sec execute 16 sec latency 11.041 ms 2023-10-18T12:29:48.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 353019 13.79 MB/sec execute 17 sec latency 7.840 ms 2023-10-18T12:29:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:49 smithi070 ceph-mon[98639]: pgmap v428: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 261 op/s 2023-10-18T12:29:49.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:49 smithi152 ceph-mon[134767]: pgmap v428: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 261 op/s 2023-10-18T12:29:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:49 smithi186 ceph-mon[135253]: pgmap v428: 129 pgs: 129 active+clean; 187 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 261 op/s 2023-10-18T12:29:49.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 355411 13.80 MB/sec execute 18 sec latency 24.251 ms 2023-10-18T12:29:50.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:50 smithi070 ceph-mon[98639]: pgmap v429: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2023-10-18T12:29:50.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:50 smithi152 ceph-mon[134767]: pgmap v429: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2023-10-18T12:29:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:50 smithi186 ceph-mon[135253]: pgmap v429: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2023-10-18T12:29:50.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 357721 13.72 MB/sec execute 19 sec latency 93.250 ms 2023-10-18T12:29:51.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 359674 13.62 MB/sec execute 20 sec latency 96.803 ms 2023-10-18T12:29:52.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 362524 13.66 MB/sec execute 21 sec latency 6.478 ms 2023-10-18T12:29:53.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:52 smithi070 ceph-mon[98639]: pgmap v430: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2023-10-18T12:29:53.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:52 smithi152 ceph-mon[134767]: pgmap v430: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2023-10-18T12:29:53.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:52 smithi186 ceph-mon[135253]: pgmap v430: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2023-10-18T12:29:53.843 INFO:tasks.workunit.client.0.smithi070.stdout: 1 365633 13.87 MB/sec execute 22 sec latency 7.497 ms 2023-10-18T12:29:54.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 368337 13.83 MB/sec execute 23 sec latency 23.708 ms 2023-10-18T12:29:55.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:55 smithi186 ceph-mon[135253]: pgmap v431: 129 pgs: 129 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2023-10-18T12:29:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:55 smithi070 ceph-mon[98639]: pgmap v431: 129 pgs: 129 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2023-10-18T12:29:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:55 smithi152 ceph-mon[134767]: pgmap v431: 129 pgs: 129 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2023-10-18T12:29:55.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 369767 13.56 MB/sec execute 24 sec latency 68.282 ms 2023-10-18T12:29:56.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 372819 13.75 MB/sec execute 25 sec latency 8.427 ms 2023-10-18T12:29:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:57 smithi070 ceph-mon[98639]: pgmap v432: 129 pgs: 129 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-10-18T12:29:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:57 smithi152 ceph-mon[134767]: pgmap v432: 129 pgs: 129 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-10-18T12:29:57.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:57 smithi186 ceph-mon[135253]: pgmap v432: 129 pgs: 129 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-10-18T12:29:57.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 375367 13.70 MB/sec execute 26 sec latency 34.634 ms 2023-10-18T12:29:58.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 377129 13.52 MB/sec execute 27 sec latency 63.467 ms 2023-10-18T12:29:59.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:29:59 smithi070 ceph-mon[98639]: pgmap v433: 129 pgs: 129 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2023-10-18T12:29:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:29:59 smithi152 ceph-mon[134767]: pgmap v433: 129 pgs: 129 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2023-10-18T12:29:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:29:59 smithi186 ceph-mon[135253]: pgmap v433: 129 pgs: 129 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2023-10-18T12:29:59.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 380095 13.65 MB/sec execute 28 sec latency 7.711 ms 2023-10-18T12:30:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:00 smithi070 ceph-mon[98639]: overall HEALTH_OK 2023-10-18T12:30:00.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:00 smithi152 ceph-mon[134767]: overall HEALTH_OK 2023-10-18T12:30:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:00 smithi186 ceph-mon[135253]: overall HEALTH_OK 2023-10-18T12:30:00.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 382145 13.45 MB/sec execute 29 sec latency 74.766 ms 2023-10-18T12:30:01.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:01 smithi070 ceph-mon[98639]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 261 op/s 2023-10-18T12:30:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:01 smithi152 ceph-mon[134767]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 261 op/s 2023-10-18T12:30:01.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:01 smithi186 ceph-mon[135253]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 261 op/s 2023-10-18T12:30:01.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 382830 13.22 MB/sec execute 30 sec latency 65.579 ms 2023-10-18T12:30:02.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 384947 13.12 MB/sec execute 31 sec latency 11.926 ms 2023-10-18T12:30:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:03 smithi070 ceph-mon[98639]: pgmap v435: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 262 op/s 2023-10-18T12:30:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:03 smithi152 ceph-mon[134767]: pgmap v435: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 262 op/s 2023-10-18T12:30:03.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:03 smithi186 ceph-mon[135253]: pgmap v435: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 262 op/s 2023-10-18T12:30:03.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 387276 13.14 MB/sec execute 32 sec latency 29.960 ms 2023-10-18T12:30:04.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 390360 13.28 MB/sec execute 33 sec latency 7.369 ms 2023-10-18T12:30:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:05 smithi070 ceph-mon[98639]: pgmap v436: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2023-10-18T12:30:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:05 smithi152 ceph-mon[134767]: pgmap v436: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2023-10-18T12:30:05.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:05 smithi186 ceph-mon[135253]: pgmap v436: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2023-10-18T12:30:05.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 391828 13.08 MB/sec execute 34 sec latency 105.411 ms 2023-10-18T12:30:06.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 394488 13.11 MB/sec execute 35 sec latency 16.553 ms 2023-10-18T12:30:07.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:07 smithi070 ceph-mon[98639]: pgmap v437: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2023-10-18T12:30:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:07 smithi152 ceph-mon[134767]: pgmap v437: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2023-10-18T12:30:07.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:07 smithi186 ceph-mon[135253]: pgmap v437: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2023-10-18T12:30:07.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 397572 13.26 MB/sec execute 36 sec latency 15.069 ms 2023-10-18T12:30:08.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 399885 13.13 MB/sec execute 37 sec latency 77.705 ms 2023-10-18T12:30:09.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:09 smithi070 ceph-mon[98639]: pgmap v438: 129 pgs: 129 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 310 op/s 2023-10-18T12:30:09.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:09 smithi152 ceph-mon[134767]: pgmap v438: 129 pgs: 129 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 310 op/s 2023-10-18T12:30:09.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:09 smithi186 ceph-mon[135253]: pgmap v438: 129 pgs: 129 active+clean; 211 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 310 op/s 2023-10-18T12:30:09.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 402079 13.13 MB/sec execute 38 sec latency 21.489 ms 2023-10-18T12:30:10.844 INFO:tasks.workunit.client.0.smithi070.stdout: 1 404704 13.22 MB/sec execute 39 sec latency 61.414 ms 2023-10-18T12:30:11.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:11 smithi070 ceph-mon[98639]: pgmap v439: 129 pgs: 129 active+clean; 217 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 301 op/s 2023-10-18T12:30:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:11 smithi152 ceph-mon[134767]: pgmap v439: 129 pgs: 129 active+clean; 217 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 301 op/s 2023-10-18T12:30:11.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:11 smithi186 ceph-mon[135253]: pgmap v439: 129 pgs: 129 active+clean; 217 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 301 op/s 2023-10-18T12:30:11.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 407158 13.17 MB/sec execute 40 sec latency 26.143 ms 2023-10-18T12:30:12.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:30:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 -- 172.21.15.70:0/2856484475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 msgr2=0x7fdbac10c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 --2- 172.21.15.70:0/2856484475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac10c360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b0d0 tx=0x7fdba400b3e0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 -- 172.21.15.70:0/2856484475 shutdown_connections 2023-10-18T12:30:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 --2- 172.21.15.70:0/2856484475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac10c360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 --2- 172.21.15.70:0/2856484475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbac1055a0 0x7fdbac105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 --2- 172.21.15.70:0/2856484475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac0ff4e0 0x7fdbac0ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 -- 172.21.15.70:0/2856484475 >> 172.21.15.70:0/2856484475 conn(0x7fdbac0fafb0 msgr2=0x7fdbac0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 -- 172.21.15.70:0/2856484475 shutdown_connections 2023-10-18T12:30:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.472+0000 7fdbb299d700 1 -- 172.21.15.70:0/2856484475 wait complete. 2023-10-18T12:30:12.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 Processor -- start 2023-10-18T12:30:12.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 -- start start 2023-10-18T12:30:12.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac0ff4e0 0x7fdbac198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbac1055a0 0x7fdbac19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac19cb50 con 0x7fdbac1055a0 2023-10-18T12:30:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac19ccc0 con 0x7fdbac0ffea0 2023-10-18T12:30:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.473+0000 7fdbb299d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac19ce80 con 0x7fdbac0ff4e0 2023-10-18T12:30:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56914/0 (socket says 172.21.15.70:56914) 2023-10-18T12:30:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 -- 172.21.15.70:0/21950439 learned_addr learned my addr 172.21.15.70:0/21950439 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb219c700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbac1055a0 0x7fdbac19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb199b700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac0ff4e0 0x7fdbac198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 -- 172.21.15.70:0/21950439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac0ff4e0 msgr2=0x7fdbac198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac0ff4e0 0x7fdbac198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 -- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbac1055a0 msgr2=0x7fdbac19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbac1055a0 0x7fdbac19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 -- 172.21.15.70:0/21950439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009520 con 0x7fdbac0ffea0 2023-10-18T12:30:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.474+0000 7fdbb119a700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac199260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00e9a0 tx=0x7fdb9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.475+0000 7fdba2ffd700 1 -- 172.21.15.70:0/21950439 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c009790 con 0x7fdbac0ffea0 2023-10-18T12:30:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.475+0000 7fdba2ffd700 1 -- 172.21.15.70:0/21950439 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb9c010430 con 0x7fdbac0ffea0 2023-10-18T12:30:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.475+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba4009d00 con 0x7fdbac0ffea0 2023-10-18T12:30:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.475+0000 7fdba2ffd700 1 -- 172.21.15.70:0/21950439 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c018680 con 0x7fdbac0ffea0 2023-10-18T12:30:12.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.475+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac1a58e0 con 0x7fdbac0ffea0 2023-10-18T12:30:12.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.478+0000 7fdba2ffd700 1 -- 172.21.15.70:0/21950439 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdb9c0105a0 con 0x7fdbac0ffea0 2023-10-18T12:30:12.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.479+0000 7fdbc7427b80 1 Processor -- start 2023-10-18T12:30:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdba2ffd700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb980a1670 0x7fdb980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 -- start start 2023-10-18T12:30:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdba2ffd700 1 -- 172.21.15.70:0/21950439 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdb9c014070 con 0x7fdbac0ffea0 2023-10-18T12:30:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbb219c700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb980a1670 0x7fdb980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361e362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e36c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361da96e0 con 0x559361e36820 2023-10-18T12:30:12.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361d8fc60 con 0x559361e3bd60 2023-10-18T12:30:12.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.480+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361d8d580 con 0x559361e3c680 2023-10-18T12:30:12.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e3c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56918/0 (socket says 172.21.15.70:56918) 2023-10-18T12:30:12.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 -- 172.21.15.70:0/1972034826 learned_addr learned my addr 172.21.15.70:0/1972034826 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:12.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb119a700 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361e362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb219c700 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e36c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 -- 172.21.15.70:0/1972034826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 msgr2=0x559361e362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361e362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 -- 172.21.15.70:0/1972034826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 msgr2=0x559361e36c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e36c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 -- 172.21.15.70:0/1972034826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559361d463f0 con 0x559361e3bd60 2023-10-18T12:30:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb219c700 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e36c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:30:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb219c700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb980a1670 0x7fdb980a3b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009fd0 tx=0x7fdba40093a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdbb199b700 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e3c140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009f10 tx=0x7fdba8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900334a544e0479b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdb97fff700 1 -- 172.21.15.70:0/1972034826 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8019630 con 0x559361e3bd60 2023-10-18T12:30:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.481+0000 7fdb97fff700 1 -- 172.21.15.70:0/1972034826 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba8019c50 con 0x559361e3bd60 2023-10-18T12:30:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdb97fff700 1 -- 172.21.15.70:0/1972034826 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800ece0 con 0x559361e3bd60 2023-10-18T12:30:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1972034826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 msgr2=0x559361e3c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e3c140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009f10 tx=0x7fdba8011720 comp rx=0 tx=0).stop 2023-10-18T12:30:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1972034826 shutdown_connections 2023-10-18T12:30:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e36c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e3c140 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1972034826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361e362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1972034826 >> 172.21.15.70:0/1972034826 conn(0x559361daa720 msgr2=0x559361e30cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1972034826 shutdown_connections 2023-10-18T12:30:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.482+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1972034826 wait complete. 2023-10-18T12:30:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.483+0000 7fdbc7427b80 1 -- start start 2023-10-18T12:30:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.483+0000 7fdbc7427b80 1 Processor -- start 2023-10-18T12:30:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 0x559361e3fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3bd60 0x559361e3ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3c680 0x559361e40500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361d88e70 con 0x559361e3c680 2023-10-18T12:30:12.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361d25710 con 0x559361e36820 2023-10-18T12:30:12.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361d253b0 con 0x559361e3bd60 2023-10-18T12:30:12.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbc7427b80 10 client.0 _ll_register_callbacks cb 0x559361c6fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:12.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 0x559361e3fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 0x559361e3fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56932/0 (socket says 172.21.15.70:56932) 2023-10-18T12:30:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb219c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3c680 0x559361e40500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 -- 172.21.15.70:0/1124214284 learned_addr learned my addr 172.21.15.70:0/1124214284 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb119a700 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3bd60 0x559361e3ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 -- 172.21.15.70:0/1124214284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3bd60 msgr2=0x559361e3ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3bd60 0x559361e3ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 -- 172.21.15.70:0/1124214284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3c680 msgr2=0x559361e40500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3c680 0x559361e40500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 -- 172.21.15.70:0/1124214284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba8009b70 con 0x559361e36820 2023-10-18T12:30:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdbb199b700 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 0x559361e3fa80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e860 tx=0x7fdba8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f909084b9cff9b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.484+0000 7fdb957fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdb957fa700 1 -- 172.21.15.70:0/1124214284 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8019430 con 0x559361e36820 2023-10-18T12:30:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559361d8def0 con 0x559361e36820 2023-10-18T12:30:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdb957fa700 1 -- 172.21.15.70:0/1124214284 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdba8019590 con 0x559361e36820 2023-10-18T12:30:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdb957fa700 1 -- 172.21.15.70:0/1124214284 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba8019750 con 0x559361e36820 2023-10-18T12:30:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdbc7427b80 10 client.26135 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:12.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559361bb3de0 con 0x559361e36820 2023-10-18T12:30:12.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdb957fa700 1 -- 172.21.15.70:0/1124214284 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559361d8def0 con 0x559361e36820 2023-10-18T12:30:12.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdb957fa700 1 -- 172.21.15.70:0/1124214284 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdba8004770 con 0x559361e36820 2023-10-18T12:30:12.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.485+0000 7fdbc7427b80 10 client.26135 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:12.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbc7427b80 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbc7427b80 20 client.26135 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361e4fea0 0x559361e522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbc7427b80 4 client.26135 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559361e058b0 con 0x559361e4fea0 2023-10-18T12:30:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbb119a700 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361e4fea0 0x559361e522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdbb119a700 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361e4fea0 0x559361e522c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a86c6e9e2da252 server_cookie=2f8cc75fc8b875d3 in_seq=0 out_seq=0 2023-10-18T12:30:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.486+0000 7fdb957fa700 10 client.26135 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:12.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdb957fa700 1 -- 172.21.15.70:0/1124214284 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559361e058b0 con 0x559361e4fea0 2023-10-18T12:30:12.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdb957fa700 10 client.26135 handle_command_reply: tid=0 2023-10-18T12:30:12.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 client.26135 shutdown 2023-10-18T12:30:12.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 msgr2=0x559361e3fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 0x559361e3fa80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e860 tx=0x7fdba8009920 comp rx=0 tx=0).stop 2023-10-18T12:30:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 shutdown_connections 2023-10-18T12:30:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361e4fea0 0x559361e522c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3c680 0x559361e40500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e36820 0x559361e3fa80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdb957fa700 0 client.26135 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/1124214284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3bd60 0x559361e3ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.487+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 >> 172.21.15.70:0/1124214284 conn(0x559361daa720 msgr2=0x559361e352d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.488+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 shutdown_connections 2023-10-18T12:30:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.488+0000 7fdbc7427b80 1 -- 172.21.15.70:0/1124214284 wait complete. 2023-10-18T12:30:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.488+0000 7fdbc7427b80 20 client.26135 trim_cache size 0 max 16384 2023-10-18T12:30:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.507+0000 7fdbc7427b80 1 Processor -- start 2023-10-18T12:30:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.507+0000 7fdbc7427b80 1 -- start start 2023-10-18T12:30:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e36820 0x559361e2af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3bd60 0x559361e47060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3c680 0x559361e3eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361c09920 con 0x559361e3bd60 2023-10-18T12:30:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361e32580 con 0x559361e3c680 2023-10-18T12:30:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361e3da70 con 0x559361e36820 2023-10-18T12:30:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e36820 0x559361e2af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3bd60 0x559361e47060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb219c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3c680 0x559361e3eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e36820 0x559361e2af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57982/0 (socket says 172.21.15.70:57982) 2023-10-18T12:30:12.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb199b700 1 -- 172.21.15.70:0/3698021320 learned_addr learned my addr 172.21.15.70:0/3698021320 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:12.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb219c700 1 -- 172.21.15.70:0/3698021320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e36820 msgr2=0x559361e2af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb219c700 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e36820 0x559361e2af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb219c700 1 -- 172.21.15.70:0/3698021320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3bd60 msgr2=0x559361e47060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb219c700 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3bd60 0x559361e47060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.508+0000 7fdbb219c700 1 -- 172.21.15.70:0/3698021320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559361b9cc60 con 0x559361e3c680 2023-10-18T12:30:12.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.509+0000 7fdbb219c700 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3c680 0x559361e3eaa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b0d0 tx=0x7fdba4024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e3299933f5b370 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.509+0000 7fdba0bff700 1 -- 172.21.15.70:0/3698021320 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400e040 con 0x559361e3c680 2023-10-18T12:30:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.509+0000 7fdba0bff700 1 -- 172.21.15.70:0/3698021320 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba40074a0 con 0x559361e3c680 2023-10-18T12:30:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.509+0000 7fdba0bff700 1 -- 172.21.15.70:0/3698021320 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40053e0 con 0x559361e3c680 2023-10-18T12:30:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3698021320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3c680 msgr2=0x559361e3eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3c680 0x559361e3eaa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b0d0 tx=0x7fdba4024000 comp rx=0 tx=0).stop 2023-10-18T12:30:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3698021320 shutdown_connections 2023-10-18T12:30:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3c680 0x559361e3eaa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e36820 0x559361e2af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3698021320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e3bd60 0x559361e47060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3698021320 >> 172.21.15.70:0/3698021320 conn(0x559361daa720 msgr2=0x559361e3e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3698021320 shutdown_connections 2023-10-18T12:30:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3698021320 wait complete. 2023-10-18T12:30:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 -- start start 2023-10-18T12:30:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.510+0000 7fdbc7427b80 1 Processor -- start 2023-10-18T12:30:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e45920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e45e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361ea3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361e46770 con 0x559361e36820 2023-10-18T12:30:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361ea4260 con 0x559361e3bd60 2023-10-18T12:30:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559361ea43a0 con 0x559361e3c680 2023-10-18T12:30:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbc7427b80 10 client.0 _ll_register_callbacks cb 0x559361c6fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e45920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbb199b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e45920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47620/0 (socket says 172.21.15.70:47620) 2023-10-18T12:30:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbb199b700 1 -- 172.21.15.70:0/3636688008 learned_addr learned my addr 172.21.15.70:0/3636688008 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.511+0000 7fdbb219c700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361ea3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb119a700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e45e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb199b700 1 -- 172.21.15.70:0/3636688008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 msgr2=0x559361ea3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb199b700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361ea3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb199b700 1 -- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 msgr2=0x559361e45e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb199b700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e45e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb199b700 1 -- 172.21.15.70:0/3636688008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac10cc60 con 0x559361e36820 2023-10-18T12:30:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbb199b700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e45920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e650 tx=0x7fdba8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd2c32ff8cf62b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdb957fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:30:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdb957fa700 1 -- 172.21.15.70:0/3636688008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800e800 con 0x559361e36820 2023-10-18T12:30:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdb957fa700 1 -- 172.21.15.70:0/3636688008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdba80048f0 con 0x559361e36820 2023-10-18T12:30:12.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.512+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdba8019c50 con 0x559361e36820 2023-10-18T12:30:12.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdb957fa700 1 -- 172.21.15.70:0/3636688008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba8019430 con 0x559361e36820 2023-10-18T12:30:12.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 10 client.16116 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:12.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559361bda250 con 0x559361e36820 2023-10-18T12:30:12.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdb957fa700 1 -- 172.21.15.70:0/3636688008 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba8019c50 con 0x559361e36820 2023-10-18T12:30:12.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdb957fa700 1 -- 172.21.15.70:0/3636688008 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdba80053e0 con 0x559361e36820 2023-10-18T12:30:12.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 10 client.16116 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:12.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:12.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 20 client.16116 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361ea5760 0x559361ea7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 4 client.16116 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:30:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559361d81e50 con 0x559361ea5760 2023-10-18T12:30:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.513+0000 7fdbb119a700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361ea5760 0x559361ea7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdb957fa700 10 client.16116 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbb119a700 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361ea5760 0x559361ea7b80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e5fb90f83841db server_cookie=ee8efe35a16ea2cd in_seq=0 out_seq=0 2023-10-18T12:30:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdb957fa700 1 -- 172.21.15.70:0/3636688008 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559361d81e50 con 0x559361ea5760 2023-10-18T12:30:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdb957fa700 10 client.16116 handle_command_reply: tid=0 2023-10-18T12:30:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 client.16116 shutdown 2023-10-18T12:30:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 msgr2=0x559361e45920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e45920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e650 tx=0x7fdba8005dc0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 shutdown_connections 2023-10-18T12:30:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559361ea5760 0x559361ea7b80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559361e3c680 0x559361ea3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559361e36820 0x559361e45920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 --2- 172.21.15.70:0/3636688008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559361e3bd60 0x559361e45e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdb957fa700 0 client.16116 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.514+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 >> 172.21.15.70:0/3636688008 conn(0x559361daa720 msgr2=0x559361e3a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 shutdown_connections 2023-10-18T12:30:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbc7427b80 1 -- 172.21.15.70:0/3636688008 wait complete. 2023-10-18T12:30:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbc7427b80 20 client.16116 trim_cache size 0 max 16384 2023-10-18T12:30:12.532 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:30:12.532 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:30:12.533 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:30:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:30:12.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb980a1670 msgr2=0x7fdb980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb980a1670 0x7fdb980a3b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009fd0 tx=0x7fdba40093a0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 msgr2=0x7fdbac199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac199260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00e9a0 tx=0x7fdb9c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 shutdown_connections 2023-10-18T12:30:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbac1055a0 0x7fdbac19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac0ff4e0 0x7fdbac198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fdb980a1670 0x7fdb980a3b30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 --2- 172.21.15.70:0/21950439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbac0ffea0 0x7fdbac199260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.515+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 >> 172.21.15.70:0/21950439 conn(0x7fdbac0fafb0 msgr2=0x7fdbac0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.516+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 shutdown_connections 2023-10-18T12:30:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.516+0000 7fdbb299d700 1 -- 172.21.15.70:0/21950439 wait complete. 2023-10-18T12:30:12.538 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:30:12.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:30:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 -- 172.21.15.70:0/499014873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01006a0 msgr2=0x7fbcd0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/499014873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01006a0 0x7fbcd0100b20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009990 tx=0x7fbcbc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 -- 172.21.15.70:0/499014873 shutdown_connections 2023-10-18T12:30:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/499014873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd0101060 0x7fbcd010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/499014873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01066b0 0x7fbcd0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/499014873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01006a0 0x7fbcd0100b20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.757+0000 7fbcd6bff700 1 -- 172.21.15.70:0/499014873 >> 172.21.15.70:0/499014873 conn(0x7fbcd00fc170 msgr2=0x7fbcd00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.758+0000 7fbcd6bff700 1 -- 172.21.15.70:0/499014873 shutdown_connections 2023-10-18T12:30:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.758+0000 7fbcd6bff700 1 -- 172.21.15.70:0/499014873 wait complete. 2023-10-18T12:30:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 Processor -- start 2023-10-18T12:30:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 -- start start 2023-10-18T12:30:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01006a0 0x7fbcd019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd0101060 0x7fbcd019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 0x7fbcd0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01992a0 con 0x7fbcd01006a0 2023-10-18T12:30:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0199410 con 0x7fbcd01066b0 2023-10-18T12:30:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd6bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01995d0 con 0x7fbcd0101060 2023-10-18T12:30:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd519c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 0x7fbcd0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01006a0 0x7fbcd019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd519c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 0x7fbcd0198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56976/0 (socket says 172.21.15.70:56976) 2023-10-18T12:30:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbcd519c700 1 -- 172.21.15.70:0/676036609 learned_addr learned my addr 172.21.15.70:0/676036609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.759+0000 7fbccffff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd0101060 0x7fbcd019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd519c700 1 -- 172.21.15.70:0/676036609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd0101060 msgr2=0x7fbcd019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd519c700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd0101060 0x7fbcd019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd519c700 1 -- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01006a0 msgr2=0x7fbcd019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd519c700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01006a0 0x7fbcd019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd519c700 1 -- 172.21.15.70:0/676036609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0095f0 con 0x7fbcd01066b0 2023-10-18T12:30:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd499b700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01006a0 0x7fbcd019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd519c700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 0x7fbcd0198ad0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800ef30 tx=0x7fbcc800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8018070 con 0x7fbcd01066b0 2023-10-18T12:30:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc800f040 con 0x7fbcd01066b0 2023-10-18T12:30:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0199850 con 0x7fbcd01066b0 2023-10-18T12:30:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800cd00 con 0x7fbcd01066b0 2023-10-18T12:30:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.760+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd01a59b0 con 0x7fbcd01066b0 2023-10-18T12:30:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.761+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0101de0 con 0x7fbcd01066b0 2023-10-18T12:30:12.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.763+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbcc8003a00 con 0x7fbcd01066b0 2023-10-18T12:30:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.765+0000 7fbccdffb700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcc00a1670 0x7fbcc00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.765+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fbcc80cd0d0 con 0x7fbcd01066b0 2023-10-18T12:30:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.765+0000 7fbcd499b700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcc00a1670 0x7fbcc00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.765+0000 7fbcd499b700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcc00a1670 0x7fbcc00a3b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009fd0 tx=0x7fbcc4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:12.768+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbcc8096120 con 0x7fbcd01066b0 2023-10-18T12:30:12.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 409455 13.17 MB/sec execute 41 sec latency 10.203 ms 2023-10-18T12:30:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.032+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcd0067530 con 0x7fbcd01066b0 2023-10-18T12:30:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.032+0000 7fbccdffb700 1 -- 172.21.15.70:0/676036609 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fbcc8099740 con 0x7fbcd01066b0 2023-10-18T12:30:13.036 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:30:13.036 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:30:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcc00a1670 msgr2=0x7fbcc00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcc00a1670 0x7fbcc00a3b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009fd0 tx=0x7fbcc4009260 comp rx=0 tx=0).stop 2023-10-18T12:30:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 msgr2=0x7fbcd0198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 0x7fbcd0198ad0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800ef30 tx=0x7fbcc800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 shutdown_connections 2023-10-18T12:30:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbcc00a1670 0x7fbcc00a3b30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcd01066b0 0x7fbcd0198ad0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd01006a0 0x7fbcd019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 --2- 172.21.15.70:0/676036609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd0101060 0x7fbcd019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 >> 172.21.15.70:0/676036609 conn(0x7fbcd00fc170 msgr2=0x7fbcd010bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 shutdown_connections 2023-10-18T12:30:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.035+0000 7fbcd6bff700 1 -- 172.21.15.70:0/676036609 wait complete. 2023-10-18T12:30:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:30:13.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:30:13.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 -- 172.21.15.70:0/14642398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc680734e0 msgr2=0x7fcc6810caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/14642398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc680734e0 0x7fcc6810caa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc500098f0 tx=0x7fcc50009c00 comp rx=0 tx=0).stop 2023-10-18T12:30:13.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 -- 172.21.15.70:0/14642398 shutdown_connections 2023-10-18T12:30:13.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/14642398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6810cfe0 0x7fcc6810f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/14642398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc68072bc0 0x7fcc68072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/14642398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc680734e0 0x7fcc6810caa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.275+0000 7fcc6ddff700 1 -- 172.21.15.70:0/14642398 >> 172.21.15.70:0/14642398 conn(0x7fcc680fc230 msgr2=0x7fcc680fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.276+0000 7fcc6ddff700 1 -- 172.21.15.70:0/14642398 shutdown_connections 2023-10-18T12:30:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.276+0000 7fcc6ddff700 1 -- 172.21.15.70:0/14642398 wait complete. 2023-10-18T12:30:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 Processor -- start 2023-10-18T12:30:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 -- start start 2023-10-18T12:30:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc68072bc0 0x7fcc68198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 0x7fcc681993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc6810cfe0 0x7fcc6819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819dda0 con 0x7fcc68072bc0 2023-10-18T12:30:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819df10 con 0x7fcc680734e0 2023-10-18T12:30:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc6ddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819e0d0 con 0x7fcc6810cfe0 2023-10-18T12:30:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 0x7fcc681993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 0x7fcc681993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56988/0 (socket says 172.21.15.70:56988) 2023-10-18T12:30:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc5ffff700 1 -- 172.21.15.70:0/697644386 learned_addr learned my addr 172.21.15.70:0/697644386 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc677fe700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc68072bc0 0x7fcc68198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.277+0000 7fcc67fff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc6810cfe0 0x7fcc6819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc5ffff700 1 -- 172.21.15.70:0/697644386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc6810cfe0 msgr2=0x7fcc6819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc6810cfe0 0x7fcc6819d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc5ffff700 1 -- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc68072bc0 msgr2=0x7fcc68198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc68072bc0 0x7fcc68198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc5ffff700 1 -- 172.21.15.70:0/697644386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50009520 con 0x7fcc680734e0 2023-10-18T12:30:13.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 0x7fcc681993c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005950 tx=0x7fcc5000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:13.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc657fa700 1 -- 172.21.15.70:0/697644386 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5001c070 con 0x7fcc680734e0 2023-10-18T12:30:13.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6819e350 con 0x7fcc680734e0 2023-10-18T12:30:13.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc657fa700 1 -- 172.21.15.70:0/697644386 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5000bb20 con 0x7fcc680734e0 2023-10-18T12:30:13.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc657fa700 1 -- 172.21.15.70:0/697644386 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500038e0 con 0x7fcc680734e0 2023-10-18T12:30:13.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.278+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc681a5bf0 con 0x7fcc680734e0 2023-10-18T12:30:13.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.282+0000 7fcc657fa700 1 -- 172.21.15.70:0/697644386 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcc50020030 con 0x7fcc680734e0 2023-10-18T12:30:13.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.283+0000 7fcc817a2b80 1 Processor -- start 2023-10-18T12:30:13.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.283+0000 7fcc817a2b80 1 -- start start 2023-10-18T12:30:13.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.283+0000 7fcc657fa700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc540a15f0 0x7fcc540a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc657fa700 1 -- 172.21.15.70:0/697644386 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcc500ce1d0 con 0x7fcc680734e0 2023-10-18T12:30:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d2da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7ca06e0 con 0x555af7d2d5d0 2023-10-18T12:30:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7c86c60 con 0x555af7d32be0 2023-10-18T12:30:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7c84580 con 0x555af7d33500 2023-10-18T12:30:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc67fff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc540a15f0 0x7fcc540a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d32fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56998/0 (socket says 172.21.15.70:56998) 2023-10-18T12:30:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 -- 172.21.15.70:0/891480037 learned_addr learned my addr 172.21.15.70:0/891480037 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc67fff700 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d2da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 -- 172.21.15.70:0/891480037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 msgr2=0x555af7d2d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d2d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 -- 172.21.15.70:0/891480037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 msgr2=0x555af7d2da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d2da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.284+0000 7fcc677fe700 1 -- 172.21.15.70:0/891480037 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555af7c3d3f0 con 0x555af7d32be0 2023-10-18T12:30:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.285+0000 7fcc677fe700 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d32fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009d60 tx=0x7fcc58011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb00f7760d2670a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.285+0000 7fcc67fff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc540a15f0 0x7fcc540a3ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcc68069e00 tx=0x7fcc6000fc00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.285+0000 7fcc5dffb700 1 -- 172.21.15.70:0/891480037 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58019630 con 0x555af7d32be0 2023-10-18T12:30:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.285+0000 7fcc5dffb700 1 -- 172.21.15.70:0/891480037 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc58019c50 con 0x555af7d32be0 2023-10-18T12:30:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.285+0000 7fcc5dffb700 1 -- 172.21.15.70:0/891480037 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800eba0 con 0x555af7d32be0 2023-10-18T12:30:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 -- 172.21.15.70:0/891480037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 msgr2=0x555af7d32fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d32fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009d60 tx=0x7fcc58011720 comp rx=0 tx=0).stop 2023-10-18T12:30:13.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 -- 172.21.15.70:0/891480037 shutdown_connections 2023-10-18T12:30:13.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d2da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d32fc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/891480037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 -- 172.21.15.70:0/891480037 >> 172.21.15.70:0/891480037 conn(0x555af7ca1720 msgr2=0x555af7d27b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 -- 172.21.15.70:0/891480037 shutdown_connections 2023-10-18T12:30:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 -- 172.21.15.70:0/891480037 wait complete. 2023-10-18T12:30:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.286+0000 7fcc817a2b80 1 -- start start 2023-10-18T12:30:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 Processor -- start 2023-10-18T12:30:13.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 0x555af7d39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d32be0 0x555af7d36710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 0x555af7d36cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7c7fe70 con 0x555af7d32be0 2023-10-18T12:30:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7c1c710 con 0x555af7d33500 2023-10-18T12:30:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.287+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7c1c3b0 con 0x555af7d2d5d0 2023-10-18T12:30:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc817a2b80 10 client.0 _ll_register_callbacks cb 0x555af7b66a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 0x555af7d39490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 0x555af7d36cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 0x555af7d36cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57004/0 (socket says 172.21.15.70:57004) 2023-10-18T12:30:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 0x555af7d39490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58046/0 (socket says 172.21.15.70:58046) 2023-10-18T12:30:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 -- 172.21.15.70:0/2167563979 learned_addr learned my addr 172.21.15.70:0/2167563979 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d32be0 0x555af7d36710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 -- 172.21.15.70:0/2167563979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 msgr2=0x555af7d39490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 0x555af7d39490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 -- 172.21.15.70:0/2167563979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d32be0 msgr2=0x555af7d36710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d32be0 0x555af7d36710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 -- 172.21.15.70:0/2167563979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc580099c0 con 0x555af7d33500 2023-10-18T12:30:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc677fe700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 0x555af7d39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc67fff700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 0x555af7d36cd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcc60018b80 tx=0x7fcc60004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef8f63299f407a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.288+0000 7fcc46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc46ffd700 1 -- 172.21.15.70:0/2167563979 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000e040 con 0x555af7d33500 2023-10-18T12:30:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555af7c84ef0 con 0x555af7d33500 2023-10-18T12:30:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc46ffd700 1 -- 172.21.15.70:0/2167563979 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcc60008db0 con 0x555af7d33500 2023-10-18T12:30:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc46ffd700 1 -- 172.21.15.70:0/2167563979 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc60018e90 con 0x555af7d33500 2023-10-18T12:30:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc817a2b80 10 client.26159 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555af7aaade0 con 0x555af7d33500 2023-10-18T12:30:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc46ffd700 1 -- 172.21.15.70:0/2167563979 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555af7c84ef0 con 0x555af7d33500 2023-10-18T12:30:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc46ffd700 1 -- 172.21.15.70:0/2167563979 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc6001a440 con 0x555af7d33500 2023-10-18T12:30:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc817a2b80 10 client.26159 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.289+0000 7fcc817a2b80 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc817a2b80 20 client.26159 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7d46eb0 0x555af7d492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc817a2b80 4 client.26159 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555af7cfc8b0 con 0x555af7d46eb0 2023-10-18T12:30:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc677fe700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7d46eb0 0x555af7d492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc677fe700 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7d46eb0 0x555af7d492d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc2f11951bd9347 server_cookie=5a73b541db348e75 in_seq=0 out_seq=0 2023-10-18T12:30:13.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.290+0000 7fcc46ffd700 10 client.26159 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:13.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.291+0000 7fcc46ffd700 1 -- 172.21.15.70:0/2167563979 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555af7cfc8b0 con 0x555af7d46eb0 2023-10-18T12:30:13.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.291+0000 7fcc46ffd700 10 client.26159 handle_command_reply: tid=0 2023-10-18T12:30:13.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.291+0000 7fcc817a2b80 1 client.26159 shutdown 2023-10-18T12:30:13.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 msgr2=0x555af7d36cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 0x555af7d36cd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcc60018b80 tx=0x7fcc60004b40 comp rx=0 tx=0).stop 2023-10-18T12:30:13.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 shutdown_connections 2023-10-18T12:30:13.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7d46eb0 0x555af7d492d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d33500 0x555af7d36cd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d2d5d0 0x555af7d39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/2167563979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d32be0 0x555af7d36710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 >> 172.21.15.70:0/2167563979 conn(0x555af7ca1720 msgr2=0x555af7d27ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc46ffd700 0 client.26159 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:13.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 shutdown_connections 2023-10-18T12:30:13.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 1 -- 172.21.15.70:0/2167563979 wait complete. 2023-10-18T12:30:13.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.292+0000 7fcc817a2b80 20 client.26159 trim_cache size 0 max 16384 2023-10-18T12:30:13.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 Processor -- start 2023-10-18T12:30:13.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 -- start start 2023-10-18T12:30:13.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d3deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d38fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7d28130 con 0x555af7d2d5d0 2023-10-18T12:30:13.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7b00920 con 0x555af7d32be0 2023-10-18T12:30:13.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.309+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7d1ffd0 con 0x555af7d33500 2023-10-18T12:30:13.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d38fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d3deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d38fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57010/0 (socket says 172.21.15.70:57010) 2023-10-18T12:30:13.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 -- 172.21.15.70:0/996061490 learned_addr learned my addr 172.21.15.70:0/996061490 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:13.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc67fff700 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 -- 172.21.15.70:0/996061490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 msgr2=0x555af7d366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 -- 172.21.15.70:0/996061490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 msgr2=0x555af7d3deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d3deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 -- 172.21.15.70:0/996061490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555af7d4df00 con 0x555af7d32be0 2023-10-18T12:30:13.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d38fc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc500973f0 tx=0x7fcc500a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4dbe030eb4af9ef server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:13.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5e7fc700 1 -- 172.21.15.70:0/996061490 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5009b640 con 0x555af7d32be0 2023-10-18T12:30:13.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5e7fc700 1 -- 172.21.15.70:0/996061490 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5009af40 con 0x555af7d32be0 2023-10-18T12:30:13.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.310+0000 7fcc5e7fc700 1 -- 172.21.15.70:0/996061490 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500cb8a0 con 0x555af7d32be0 2023-10-18T12:30:13.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 -- 172.21.15.70:0/996061490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 msgr2=0x555af7d38fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d38fc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc500973f0 tx=0x7fcc500a1040 comp rx=0 tx=0).stop 2023-10-18T12:30:13.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 -- 172.21.15.70:0/996061490 shutdown_connections 2023-10-18T12:30:13.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d33500 0x555af7d366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d2d5d0 0x555af7d3deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/996061490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d32be0 0x555af7d38fc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 -- 172.21.15.70:0/996061490 >> 172.21.15.70:0/996061490 conn(0x555af7ca1720 msgr2=0x555af7d30930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 -- 172.21.15.70:0/996061490 shutdown_connections 2023-10-18T12:30:13.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.311+0000 7fcc817a2b80 1 -- 172.21.15.70:0/996061490 wait complete. 2023-10-18T12:30:13.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.312+0000 7fcc817a2b80 1 -- start start 2023-10-18T12:30:13.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.312+0000 7fcc817a2b80 1 Processor -- start 2023-10-18T12:30:13.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 0x555af7d2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d32be0 0x555af7d2fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d33500 0x555af7d2e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7d2eb70 con 0x555af7d33500 2023-10-18T12:30:13.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7d2ece0 con 0x555af7d2d5d0 2023-10-18T12:30:13.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555af7d2ee50 con 0x555af7d32be0 2023-10-18T12:30:13.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc817a2b80 10 client.0 _ll_register_callbacks cb 0x555af7b66a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:13.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 0x555af7d2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 0x555af7d2f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57018/0 (socket says 172.21.15.70:57018) 2023-10-18T12:30:13.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 -- 172.21.15.70:0/3219164654 learned_addr learned my addr 172.21.15.70:0/3219164654 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:13.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d32be0 0x555af7d2fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc67fff700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d33500 0x555af7d2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 -- 172.21.15.70:0/3219164654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d32be0 msgr2=0x555af7d2fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d32be0 0x555af7d2fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 -- 172.21.15.70:0/3219164654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d33500 msgr2=0x555af7d2e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:13.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d33500 0x555af7d2e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc677fe700 1 -- 172.21.15.70:0/3219164654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc681a5bf0 con 0x555af7d2d5d0 2023-10-18T12:30:13.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d32be0 0x555af7d2fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:13.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.313+0000 7fcc67fff700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d33500 0x555af7d2e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:30:13.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc677fe700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 0x555af7d2f790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009990 tx=0x7fcc58005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b82d186e50b1115 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:13.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:13.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc46ffd700 1 -- 172.21.15.70:0/3219164654 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5802a040 con 0x555af7d2d5d0 2023-10-18T12:30:13.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc58019c50 con 0x555af7d2d5d0 2023-10-18T12:30:13.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc46ffd700 1 -- 172.21.15.70:0/3219164654 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcc58019a40 con 0x555af7d2d5d0 2023-10-18T12:30:13.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc46ffd700 1 -- 172.21.15.70:0/3219164654 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5802d590 con 0x555af7d2d5d0 2023-10-18T12:30:13.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc817a2b80 10 client.26165 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:13.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555af7a93c60 con 0x555af7d2d5d0 2023-10-18T12:30:13.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc46ffd700 1 -- 172.21.15.70:0/3219164654 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc58019c50 con 0x555af7d2d5d0 2023-10-18T12:30:13.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.314+0000 7fcc46ffd700 1 -- 172.21.15.70:0/3219164654 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc58005560 con 0x555af7d2d5d0 2023-10-18T12:30:13.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc817a2b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:13.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc817a2b80 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:13.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc817a2b80 20 client.26165 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:13.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7da42c0 0x555af7da66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:13.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc817a2b80 4 client.26165 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:30:13.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555af7c78e50 con 0x555af7da42c0 2023-10-18T12:30:13.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7da42c0 0x555af7da66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:13.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc5ffff700 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7da42c0 0x555af7da66e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf105c9a3a20f766 server_cookie=c1d2ea2374caee6b in_seq=0 out_seq=0 2023-10-18T12:30:13.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.315+0000 7fcc46ffd700 10 client.26165 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:13.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc46ffd700 1 -- 172.21.15.70:0/3219164654 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555af7c78e50 con 0x555af7da42c0 2023-10-18T12:30:13.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc46ffd700 10 client.26165 handle_command_reply: tid=0 2023-10-18T12:30:13.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 client.26165 shutdown 2023-10-18T12:30:13.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 msgr2=0x555af7d2f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 0x555af7d2f790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009990 tx=0x7fcc58005be0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 shutdown_connections 2023-10-18T12:30:13.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555af7da42c0 0x555af7da66e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555af7d33500 0x555af7d2e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555af7d2d5d0 0x555af7d2f790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 --2- 172.21.15.70:0/3219164654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555af7d32be0 0x555af7d2fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 >> 172.21.15.70:0/3219164654 conn(0x555af7ca1720 msgr2=0x555af7d366b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc46ffd700 0 client.26165 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:13.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 shutdown_connections 2023-10-18T12:30:13.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.316+0000 7fcc817a2b80 1 -- 172.21.15.70:0/3219164654 wait complete. 2023-10-18T12:30:13.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc817a2b80 20 client.26165 trim_cache size 0 max 16384 2023-10-18T12:30:13.333 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:30:13.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc540a15f0 msgr2=0x7fcc540a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc540a15f0 0x7fcc540a3ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcc68069e00 tx=0x7fcc6000fc00 comp rx=0 tx=0).stop 2023-10-18T12:30:13.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 msgr2=0x7fcc681993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:13.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 0x7fcc681993c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005950 tx=0x7fcc5000fa00 comp rx=0 tx=0).stop 2023-10-18T12:30:13.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 shutdown_connections 2023-10-18T12:30:13.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc6810cfe0 0x7fcc6819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc68072bc0 0x7fcc68198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc540a15f0 0x7fcc540a3ab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 --2- 172.21.15.70:0/697644386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc680734e0 0x7fcc681993c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:13.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.317+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 >> 172.21.15.70:0/697644386 conn(0x7fcc680fc230 msgr2=0x7fcc680fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:13.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.318+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 shutdown_connections 2023-10-18T12:30:13.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:13.318+0000 7fcc6ddff700 1 -- 172.21.15.70:0/697644386 wait complete. 2023-10-18T12:30:13.338 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:13 smithi070 ceph-mon[98639]: pgmap v440: 129 pgs: 129 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 285 op/s 2023-10-18T12:30:13.338 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:13 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/676036609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:30:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:13 smithi152 ceph-mon[134767]: pgmap v440: 129 pgs: 129 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 285 op/s 2023-10-18T12:30:13.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:13 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/676036609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:30:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:13 smithi186 ceph-mon[135253]: pgmap v440: 129 pgs: 129 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 285 op/s 2023-10-18T12:30:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:13 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/676036609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:30:13.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 412043 13.20 MB/sec execute 42 sec latency 10.431 ms 2023-10-18T12:30:14.341 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:30:14.341 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:30:14.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 -- 172.21.15.70:0/2134679815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944105170 msgr2=0x7fd9441055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/2134679815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944105170 0x7fd9441055f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd934009990 tx=0x7fd934009ca0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 -- 172.21.15.70:0/2134679815 shutdown_connections 2023-10-18T12:30:14.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/2134679815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd944105b30 0x7fd94410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/2134679815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944067d40 0x7fd944068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/2134679815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944105170 0x7fd9441055f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 -- 172.21.15.70:0/2134679815 >> 172.21.15.70:0/2134679815 conn(0x7fd9440767f0 msgr2=0x7fd944076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 -- 172.21.15.70:0/2134679815 shutdown_connections 2023-10-18T12:30:14.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.576+0000 7fd94b5ff700 1 -- 172.21.15.70:0/2134679815 wait complete. 2023-10-18T12:30:14.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.577+0000 7fd94b5ff700 1 Processor -- start 2023-10-18T12:30:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.577+0000 7fd94b5ff700 1 -- start start 2023-10-18T12:30:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94b5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 0x7fd944198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94b5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd944105170 0x7fd944199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94b5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944105b30 0x7fd94419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94b5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419dd20 con 0x7fd944105b30 2023-10-18T12:30:14.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94b5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419de90 con 0x7fd944067d40 2023-10-18T12:30:14.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94b5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419e050 con 0x7fd944105170 2023-10-18T12:30:14.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 0x7fd944198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 0x7fd944198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57034/0 (socket says 172.21.15.70:57034) 2023-10-18T12:30:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 -- 172.21.15.70:0/426852665 learned_addr learned my addr 172.21.15.70:0/426852665 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd949b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944105b30 0x7fd94419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd948b9a700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd944105170 0x7fd944199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 -- 172.21.15.70:0/426852665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd944105170 msgr2=0x7fd944199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd944105170 0x7fd944199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 -- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944105b30 msgr2=0x7fd94419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944105b30 0x7fd94419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd94939b700 1 -- 172.21.15.70:0/426852665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340095f0 con 0x7fd944067d40 2023-10-18T12:30:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.578+0000 7fd949b9c700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944105b30 0x7fd94419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.579+0000 7fd94939b700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 0x7fd944198e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd94000e9a0 tx=0x7fd94000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.579+0000 7fd93a7fc700 1 -- 172.21.15.70:0/426852665 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940009790 con 0x7fd944067d40 2023-10-18T12:30:14.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.579+0000 7fd93a7fc700 1 -- 172.21.15.70:0/426852665 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd940010430 con 0x7fd944067d40 2023-10-18T12:30:14.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.579+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94419e330 con 0x7fd944067d40 2023-10-18T12:30:14.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.579+0000 7fd93a7fc700 1 -- 172.21.15.70:0/426852665 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940018680 con 0x7fd944067d40 2023-10-18T12:30:14.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.579+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9441a5b70 con 0x7fd944067d40 2023-10-18T12:30:14.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.582+0000 7fd93a7fc700 1 -- 172.21.15.70:0/426852665 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd9400105a0 con 0x7fd944067d40 2023-10-18T12:30:14.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.583+0000 7fd93a7fc700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9300a1670 0x7fd9300a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 Processor -- start 2023-10-18T12:30:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd93a7fc700 1 -- 172.21.15.70:0/426852665 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd940014070 con 0x7fd944067d40 2023-10-18T12:30:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 -- start start 2023-10-18T12:30:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd948b9a700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9300a1670 0x7fd9300a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cabe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cab8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cab8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14ca2b760 con 0x55a14cabdc60 2023-10-18T12:30:14.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14ca11ce0 con 0x55a14cabe580 2023-10-18T12:30:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.584+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14ca0f600 con 0x55a14cab8650 2023-10-18T12:30:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cabe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cabe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47714/0 (socket says 172.21.15.70:47714) 2023-10-18T12:30:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 -- 172.21.15.70:0/1268995706 learned_addr learned my addr 172.21.15.70:0/1268995706 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd948b9a700 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cab8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd949b9c700 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cab8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd948b9a700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9300a1670 0x7fd9300a3b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd94419a350 tx=0x7fd93400b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 -- 172.21.15.70:0/1268995706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 msgr2=0x55a14cab8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cab8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 -- 172.21.15.70:0/1268995706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 msgr2=0x55a14cab8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cab8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 -- 172.21.15.70:0/1268995706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a14c9c8470 con 0x55a14cabdc60 2023-10-18T12:30:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd94939b700 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cabe040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd940096c80 tx=0x7fd940099e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cdcb565a8b1a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.585+0000 7fd92f7fe700 1 -- 172.21.15.70:0/1268995706 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94009e170 con 0x55a14cabdc60 2023-10-18T12:30:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.586+0000 7fd92f7fe700 1 -- 172.21.15.70:0/1268995706 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9400a25a0 con 0x55a14cabdc60 2023-10-18T12:30:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.586+0000 7fd92f7fe700 1 -- 172.21.15.70:0/1268995706 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940099700 con 0x55a14cabdc60 2023-10-18T12:30:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.586+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1268995706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 msgr2=0x55a14cabe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.586+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cabe040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd940096c80 tx=0x7fd940099e80 comp rx=0 tx=0).stop 2023-10-18T12:30:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1268995706 shutdown_connections 2023-10-18T12:30:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cab8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cabe040 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1268995706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cab8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1268995706 >> 172.21.15.70:0/1268995706 conn(0x55a14ca2c7a0 msgr2=0x55a14cab2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1268995706 shutdown_connections 2023-10-18T12:30:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1268995706 wait complete. 2023-10-18T12:30:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.587+0000 7fd95eed9b80 1 -- start start 2023-10-18T12:30:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 Processor -- start 2023-10-18T12:30:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 0x55a14cac4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cac1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cabe580 0x55a14cac1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14ca0aef0 con 0x55a14cabdc60 2023-10-18T12:30:14.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c9a7790 con 0x55a14cab8650 2023-10-18T12:30:14.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c9a7430 con 0x55a14cabe580 2023-10-18T12:30:14.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd95eed9b80 10 client.0 _ll_register_callbacks cb 0x55a14c8f1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:14.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd94939b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 0x55a14cac4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd948b9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cac1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd94939b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 0x55a14cac4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57048/0 (socket says 172.21.15.70:57048) 2023-10-18T12:30:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd94939b700 1 -- 172.21.15.70:0/1637696215 learned_addr learned my addr 172.21.15.70:0/1637696215 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.588+0000 7fd949b9c700 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cabe580 0x55a14cac1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd94939b700 1 -- 172.21.15.70:0/1637696215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cabe580 msgr2=0x55a14cac1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd94939b700 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cabe580 0x55a14cac1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd94939b700 1 -- 172.21.15.70:0/1637696215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 msgr2=0x55a14cac1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd94939b700 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cac1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd94939b700 1 -- 172.21.15.70:0/1637696215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9441a5b70 con 0x55a14cab8650 2023-10-18T12:30:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd94939b700 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 0x55a14cac4540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd940099f10 tx=0x7fd940010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f7a63b8ba725d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd92cff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd92cff9700 1 -- 172.21.15.70:0/1637696215 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9400b82a0 con 0x55a14cab8650 2023-10-18T12:30:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd92cff9700 1 -- 172.21.15.70:0/1637696215 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd9400b8400 con 0x55a14cab8650 2023-10-18T12:30:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a14ca0ff70 con 0x55a14cab8650 2023-10-18T12:30:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd92cff9700 1 -- 172.21.15.70:0/1637696215 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd94009a470 con 0x55a14cab8650 2023-10-18T12:30:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd95eed9b80 10 client.26177 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.589+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a14c835e60 con 0x55a14cab8650 2023-10-18T12:30:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd92cff9700 1 -- 172.21.15.70:0/1637696215 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a14ca0ff70 con 0x55a14cab8650 2023-10-18T12:30:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd92cff9700 1 -- 172.21.15.70:0/1637696215 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd940099290 con 0x55a14cab8650 2023-10-18T12:30:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd95eed9b80 10 client.26177 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:14.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd95eed9b80 10 client.26177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:14.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd95eed9b80 20 client.26177 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:14.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cad1b10 0x55a14cad3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd95eed9b80 4 client.26177 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:14.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a14ca87930 con 0x55a14cad1b10 2023-10-18T12:30:14.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.590+0000 7fd948b9a700 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cad1b10 0x55a14cad3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd948b9a700 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cad1b10 0x55a14cad3f00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4d64989d0ef7 server_cookie=7627acca74e5537a in_seq=0 out_seq=0 2023-10-18T12:30:14.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd92cff9700 10 client.26177 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd92cff9700 1 -- 172.21.15.70:0/1637696215 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a14ca87930 con 0x55a14cad1b10 2023-10-18T12:30:14.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd92cff9700 10 client.26177 handle_command_reply: tid=0 2023-10-18T12:30:14.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd95eed9b80 1 client.26177 shutdown 2023-10-18T12:30:14.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 msgr2=0x55a14cac4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.591+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 0x55a14cac4540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd940099f10 tx=0x7fd940010ab0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 shutdown_connections 2023-10-18T12:30:14.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cad1b10 0x55a14cad3f00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cabe580 0x55a14cac1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd92cff9700 0 client.26177 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cab8650 0x55a14cac4540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1637696215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cac1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 >> 172.21.15.70:0/1637696215 conn(0x55a14ca2c7a0 msgr2=0x55a14cab7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 shutdown_connections 2023-10-18T12:30:14.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1637696215 wait complete. 2023-10-18T12:30:14.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.592+0000 7fd95eed9b80 20 client.26177 trim_cache size 0 max 16384 2023-10-18T12:30:14.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.611+0000 7fd95eed9b80 1 Processor -- start 2023-10-18T12:30:14.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.611+0000 7fd95eed9b80 1 -- start start 2023-10-18T12:30:14.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.611+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cac8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.611+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cacd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.611+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cac0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.611+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c88b9a0 con 0x55a14cabdc60 2023-10-18T12:30:14.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cab90b0 con 0x55a14cabe580 2023-10-18T12:30:14.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cab4430 con 0x55a14cab8650 2023-10-18T12:30:14.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cac0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd94939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cac8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd948b9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cacd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cac0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57058/0 (socket says 172.21.15.70:57058) 2023-10-18T12:30:14.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd94939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cac8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58112/0 (socket says 172.21.15.70:58112) 2023-10-18T12:30:14.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 -- 172.21.15.70:0/1988237955 learned_addr learned my addr 172.21.15.70:0/1988237955 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 -- 172.21.15.70:0/1988237955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 msgr2=0x55a14cac8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cac8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 -- 172.21.15.70:0/1988237955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 msgr2=0x55a14cacd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cacd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 -- 172.21.15.70:0/1988237955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a14cab2df0 con 0x55a14cabe580 2023-10-18T12:30:14.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.612+0000 7fd949b9c700 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cac0120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd93c003b00 tx=0x7fd93c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66de491a7bab62ec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd92ffff700 1 -- 172.21.15.70:0/1988237955 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c004510 con 0x55a14cabe580 2023-10-18T12:30:14.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd92ffff700 1 -- 172.21.15.70:0/1988237955 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93c00f040 con 0x55a14cabe580 2023-10-18T12:30:14.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd92ffff700 1 -- 172.21.15.70:0/1988237955 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0086a0 con 0x55a14cabe580 2023-10-18T12:30:14.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1988237955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 msgr2=0x55a14cac0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cac0120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd93c003b00 tx=0x7fd93c009920 comp rx=0 tx=0).stop 2023-10-18T12:30:14.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1988237955 shutdown_connections 2023-10-18T12:30:14.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabe580 0x55a14cac0120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cac8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/1988237955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabdc60 0x55a14cacd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.613+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1988237955 >> 172.21.15.70:0/1988237955 conn(0x55a14ca2c7a0 msgr2=0x55a14cab7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.614+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1988237955 shutdown_connections 2023-10-18T12:30:14.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.614+0000 7fd95eed9b80 1 -- 172.21.15.70:0/1988237955 wait complete. 2023-10-18T12:30:14.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.614+0000 7fd95eed9b80 1 -- start start 2023-10-18T12:30:14.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.614+0000 7fd95eed9b80 1 Processor -- start 2023-10-18T12:30:14.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cb1b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabdc60 0x55a14cb1bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabe580 0x55a14cb1c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cb26970 con 0x55a14cabe580 2023-10-18T12:30:14.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cb26ae0 con 0x55a14cabdc60 2023-10-18T12:30:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14cb26ca0 con 0x55a14cab8650 2023-10-18T12:30:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd95eed9b80 10 client.0 _ll_register_callbacks cb 0x55a14c8f1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd94939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cb1b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd94939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cb1b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58116/0 (socket says 172.21.15.70:58116) 2023-10-18T12:30:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd94939b700 1 -- 172.21.15.70:0/68968059 learned_addr learned my addr 172.21.15.70:0/68968059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd949b9c700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabe580 0x55a14cb1c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.615+0000 7fd948b9a700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabdc60 0x55a14cb1bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd94939b700 1 -- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabdc60 msgr2=0x55a14cb1bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd94939b700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabdc60 0x55a14cb1bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd94939b700 1 -- 172.21.15.70:0/68968059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabe580 msgr2=0x55a14cb1c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd94939b700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabe580 0x55a14cb1c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd94939b700 1 -- 172.21.15.70:0/68968059 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c009300 con 0x55a14cab8650 2023-10-18T12:30:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd949b9c700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabe580 0x55a14cb1c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:30:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd94939b700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cb1b870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9400a2070 tx=0x7fd94009e4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60786c9b02646101 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd92cff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:30:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd92cff9700 1 -- 172.21.15.70:0/68968059 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94009f140 con 0x55a14cab8650 2023-10-18T12:30:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd92cff9700 1 -- 172.21.15.70:0/68968059 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd940018a60 con 0x55a14cab8650 2023-10-18T12:30:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.616+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9400a25a0 con 0x55a14cab8650 2023-10-18T12:30:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd92cff9700 1 -- 172.21.15.70:0/68968059 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd94009a000 con 0x55a14cab8650 2023-10-18T12:30:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 10 client.26182 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a14c81ece0 con 0x55a14cab8650 2023-10-18T12:30:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd92cff9700 1 -- 172.21.15.70:0/68968059 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9400a25a0 con 0x55a14cab8650 2023-10-18T12:30:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd92cff9700 1 -- 172.21.15.70:0/68968059 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd9400a2110 con 0x55a14cab8650 2023-10-18T12:30:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 20 client.26182 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cb28000 0x55a14cb2a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 4 client.26182 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:30:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a14ca03ed0 con 0x55a14cb28000 2023-10-18T12:30:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.617+0000 7fd948b9a700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cb28000 0x55a14cb2a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.618+0000 7fd948b9a700 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cb28000 0x55a14cb2a3f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476411f9d8212e72 server_cookie=f8d83a569460ff2 in_seq=0 out_seq=0 2023-10-18T12:30:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.618+0000 7fd92cff9700 10 client.26182 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd92cff9700 1 -- 172.21.15.70:0/68968059 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a14ca03ed0 con 0x55a14cb28000 2023-10-18T12:30:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd92cff9700 10 client.26182 handle_command_reply: tid=0 2023-10-18T12:30:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 client.26182 shutdown 2023-10-18T12:30:14.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 msgr2=0x55a14cb1b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cb1b870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9400a2070 tx=0x7fd94009e4d0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 shutdown_connections 2023-10-18T12:30:14.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a14cb28000 0x55a14cb2a3f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a14cabe580 0x55a14cb1c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a14cab8650 0x55a14cb1b870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 --2- 172.21.15.70:0/68968059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a14cabdc60 0x55a14cb1bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd92cff9700 0 client.26182 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 >> 172.21.15.70:0/68968059 conn(0x55a14ca2c7a0 msgr2=0x55a14cabb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 shutdown_connections 2023-10-18T12:30:14.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 1 -- 172.21.15.70:0/68968059 wait complete. 2023-10-18T12:30:14.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.619+0000 7fd95eed9b80 20 client.26182 trim_cache size 0 max 16384 2023-10-18T12:30:14.639 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:30:14.639 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:30:14.640 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "2ada7f7c-48d1-4ec7-a637-73dfa6442b3c", 2023-10-18T12:30:14.640 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:30:14.640 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:30:14.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9300a1670 msgr2=0x7fd9300a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9300a1670 0x7fd9300a3b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd94419a350 tx=0x7fd93400b260 comp rx=0 tx=0).stop 2023-10-18T12:30:14.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 msgr2=0x7fd944198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 0x7fd944198e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd94000e9a0 tx=0x7fd94000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 shutdown_connections 2023-10-18T12:30:14.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd944105b30 0x7fd94419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd9300a1670 0x7fd9300a3b30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944067d40 0x7fd944198e00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 --2- 172.21.15.70:0/426852665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd944105170 0x7fd944199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 >> 172.21.15.70:0/426852665 conn(0x7fd9440767f0 msgr2=0x7fd94410be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 shutdown_connections 2023-10-18T12:30:14.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.620+0000 7fd94b5ff700 1 -- 172.21.15.70:0/426852665 wait complete. 2023-10-18T12:30:14.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:30:14.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 415043 13.29 MB/sec execute 43 sec latency 11.787 ms 2023-10-18T12:30:14.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3402617878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8106680 msgr2=0x7f52e8106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3402617878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8106680 0x7f52e8106a60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009990 tx=0x7f52d8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3402617878 shutdown_connections 2023-10-18T12:30:14.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3402617878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e8100fa0 0x7f52e810d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3402617878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8106680 0x7f52e8106a60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3402617878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e81005e0 0x7f52e8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.866+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3402617878 >> 172.21.15.70:0/3402617878 conn(0x7f52e80fc0d0 msgr2=0x7f52e80fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.867+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3402617878 shutdown_connections 2023-10-18T12:30:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.867+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3402617878 wait complete. 2023-10-18T12:30:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 Processor -- start 2023-10-18T12:30:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 -- start start 2023-10-18T12:30:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e81005e0 0x7f52e819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 0x7f52e819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e8106680 0x7f52e8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8199140 con 0x7f52e81005e0 2023-10-18T12:30:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81992b0 con 0x7f52e8100fa0 2023-10-18T12:30:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.868+0000 7f52ecb9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8199470 con 0x7f52e8106680 2023-10-18T12:30:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 0x7f52e819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 0x7f52e819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57090/0 (socket says 172.21.15.70:57090) 2023-10-18T12:30:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e81005e0 0x7f52e819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 -- 172.21.15.70:0/3701258962 learned_addr learned my addr 172.21.15.70:0/3701258962 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e7fff700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e8106680 0x7f52e8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 -- 172.21.15.70:0/3701258962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e8106680 msgr2=0x7f52e8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e8106680 0x7f52e8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 -- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e81005e0 msgr2=0x7f52e819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e81005e0 0x7f52e819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 -- 172.21.15.70:0/3701258962 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80095f0 con 0x7f52e8100fa0 2023-10-18T12:30:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 0x7f52e819eef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00e9a0 tx=0x7f52dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.869+0000 7f52e4ff9700 1 -- 172.21.15.70:0/3701258962 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc009790 con 0x7f52e8100fa0 2023-10-18T12:30:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.870+0000 7f52e4ff9700 1 -- 172.21.15.70:0/3701258962 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52dc00cbd0 con 0x7f52e8100fa0 2023-10-18T12:30:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.870+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8199750 con 0x7f52e8100fa0 2023-10-18T12:30:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.870+0000 7f52e4ff9700 1 -- 172.21.15.70:0/3701258962 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc018680 con 0x7f52e8100fa0 2023-10-18T12:30:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.870+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81a59a0 con 0x7f52e8100fa0 2023-10-18T12:30:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.873+0000 7f52e4ff9700 1 -- 172.21.15.70:0/3701258962 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f52dc00cd40 con 0x7f52e8100fa0 2023-10-18T12:30:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.874+0000 7f52e4ff9700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d00a15f0 0x7f52d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 Processor -- start 2023-10-18T12:30:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f52e4ff9700 1 -- 172.21.15.70:0/3701258962 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52dc014070 con 0x7f52e8100fa0 2023-10-18T12:30:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f52e77fe700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d00a15f0 0x7f52d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 -- start start 2023-10-18T12:30:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd672c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd67262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6726c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd66996e0 con 0x555cd6726820 2023-10-18T12:30:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd667fc60 con 0x555cd672c680 2023-10-18T12:30:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f53017d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd667d580 con 0x555cd672bd60 2023-10-18T12:30:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.875+0000 7f52e77fe700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d00a15f0 0x7f52d00a3ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b2e0 tx=0x7f52d8005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd67262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd67262e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:57098/0 (socket says 172.21.15.70:57098) 2023-10-18T12:30:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1023170630 learned_addr learned my addr 172.21.15.70:0/1023170630 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e7fff700 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6726c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e77fe700 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd672c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1023170630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 msgr2=0x555cd672c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd672c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1023170630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 msgr2=0x555cd6726c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6726c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1023170630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cd66363f0 con 0x555cd672c680 2023-10-18T12:30:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e7fff700 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6726c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:30:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd67262e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52dc096e00 tx=0x7f52dc099cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8930c3aa9c3b270d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52cdffb700 1 -- 172.21.15.70:0/1023170630 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc09e4e0 con 0x555cd672c680 2023-10-18T12:30:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52cdffb700 1 -- 172.21.15.70:0/1023170630 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52dc0a2720 con 0x555cd672c680 2023-10-18T12:30:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.876+0000 7f52cdffb700 1 -- 172.21.15.70:0/1023170630 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc09a850 con 0x555cd672c680 2023-10-18T12:30:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 -- 172.21.15.70:0/1023170630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 msgr2=0x555cd67262e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd67262e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52dc096e00 tx=0x7f52dc099cf0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 -- 172.21.15.70:0/1023170630 shutdown_connections 2023-10-18T12:30:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6726c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd672c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1023170630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd67262e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 -- 172.21.15.70:0/1023170630 >> 172.21.15.70:0/1023170630 conn(0x555cd669a720 msgr2=0x555cd6720cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 -- 172.21.15.70:0/1023170630 shutdown_connections 2023-10-18T12:30:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.877+0000 7f53017d3b80 1 -- 172.21.15.70:0/1023170630 wait complete. 2023-10-18T12:30:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.878+0000 7f53017d3b80 1 -- start start 2023-10-18T12:30:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.878+0000 7f53017d3b80 1 Processor -- start 2023-10-18T12:30:14.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd672fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd67300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd6730610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd6678e70 con 0x555cd6726820 2023-10-18T12:30:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd6615710 con 0x555cd672c680 2023-10-18T12:30:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd66153b0 con 0x555cd672bd60 2023-10-18T12:30:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f53017d3b80 10 client.0 _ll_register_callbacks cb 0x555cd655fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd67300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd672fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f52e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd6730610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd67300d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58138/0 (socket says 172.21.15.70:58138) 2023-10-18T12:30:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.879+0000 7f52e6ffd700 1 -- 172.21.15.70:0/2164691867 learned_addr learned my addr 172.21.15.70:0/2164691867 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52e6ffd700 1 -- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 msgr2=0x555cd6730610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd6730610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52e6ffd700 1 -- 172.21.15.70:0/2164691867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 msgr2=0x555cd672fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd672fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52e6ffd700 1 -- 172.21.15.70:0/2164691867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81a59a0 con 0x555cd672bd60 2023-10-18T12:30:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd67300d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009230 tx=0x7f52dc0cd5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46cb45367b5f2f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:30:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52c6ffd700 1 -- 172.21.15.70:0/2164691867 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc099a90 con 0x555cd672bd60 2023-10-18T12:30:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52c6ffd700 1 -- 172.21.15.70:0/2164691867 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52dc09a5f0 con 0x555cd672bd60 2023-10-18T12:30:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cd667def0 con 0x555cd672bd60 2023-10-18T12:30:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f52c6ffd700 1 -- 172.21.15.70:0/2164691867 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc09a7d0 con 0x555cd672bd60 2023-10-18T12:30:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f53017d3b80 10 client.26194 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.880+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555cd64a3de0 con 0x555cd672bd60 2023-10-18T12:30:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f52c6ffd700 1 -- 172.21.15.70:0/2164691867 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555cd667def0 con 0x555cd672bd60 2023-10-18T12:30:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f52c6ffd700 1 -- 172.21.15.70:0/2164691867 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52dc07b070 con 0x555cd672bd60 2023-10-18T12:30:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f53017d3b80 10 client.26194 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:14.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f53017d3b80 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:14.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f53017d3b80 20 client.26194 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:14.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd673fcd0 0x555cd67420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f53017d3b80 4 client.26194 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:14.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.881+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cd66f58b0 con 0x555cd673fcd0 2023-10-18T12:30:14.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.882+0000 7f52e77fe700 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd673fcd0 0x555cd67420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.882+0000 7f52e77fe700 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd673fcd0 0x555cd67420f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89da239d794a3cf9 server_cookie=aad293c0227cb1e9 in_seq=0 out_seq=0 2023-10-18T12:30:14.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.882+0000 7f52c6ffd700 10 client.26194 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.883+0000 7f52c6ffd700 1 -- 172.21.15.70:0/2164691867 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555cd66f58b0 con 0x555cd673fcd0 2023-10-18T12:30:14.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.883+0000 7f52c6ffd700 10 client.26194 handle_command_reply: tid=0 2023-10-18T12:30:14.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.883+0000 7f53017d3b80 1 client.26194 shutdown 2023-10-18T12:30:14.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 msgr2=0x555cd67300d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd67300d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009230 tx=0x7f52dc0cd5e0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 shutdown_connections 2023-10-18T12:30:14.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd673fcd0 0x555cd67420f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd6730610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd672fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2164691867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672bd60 0x555cd67300d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f52c6ffd700 0 client.26194 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 >> 172.21.15.70:0/2164691867 conn(0x555cd669a720 msgr2=0x555cd67252d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 shutdown_connections 2023-10-18T12:30:14.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 1 -- 172.21.15.70:0/2164691867 wait complete. 2023-10-18T12:30:14.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.884+0000 7f53017d3b80 20 client.26194 trim_cache size 0 max 16384 2023-10-18T12:30:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 Processor -- start 2023-10-18T12:30:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 -- start start 2023-10-18T12:30:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd6726820 0x555cd67262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 0x555cd672f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd671a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd64f9920 con 0x555cd672bd60 2023-10-18T12:30:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd672e0c0 con 0x555cd672c680 2023-10-18T12:30:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.902+0000 7f53017d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd6721170 con 0x555cd6726820 2023-10-18T12:30:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 0x555cd672f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd671a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 0x555cd672f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47778/0 (socket says 172.21.15.70:47778) 2023-10-18T12:30:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e6ffd700 1 -- 172.21.15.70:0/2804279660 learned_addr learned my addr 172.21.15.70:0/2804279660 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e77fe700 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd6726820 0x555cd67262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 -- 172.21.15.70:0/2804279660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd6726820 msgr2=0x555cd67262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd6726820 0x555cd67262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 -- 172.21.15.70:0/2804279660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 msgr2=0x555cd672f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 0x555cd672f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 -- 172.21.15.70:0/2804279660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cd648cc60 con 0x555cd672c680 2023-10-18T12:30:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 0x555cd672f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.903+0000 7f52e7fff700 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd671a700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009460 tx=0x7f52e000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ffedef5abeaed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.904+0000 7f52ce7fc700 1 -- 172.21.15.70:0/2804279660 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000e040 con 0x555cd672c680 2023-10-18T12:30:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.904+0000 7f52ce7fc700 1 -- 172.21.15.70:0/2804279660 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e000f040 con 0x555cd672c680 2023-10-18T12:30:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.904+0000 7f52ce7fc700 1 -- 172.21.15.70:0/2804279660 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0005280 con 0x555cd672c680 2023-10-18T12:30:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 -- 172.21.15.70:0/2804279660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 msgr2=0x555cd671a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd671a700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009460 tx=0x7f52e000b040 comp rx=0 tx=0).stop 2023-10-18T12:30:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 -- 172.21.15.70:0/2804279660 shutdown_connections 2023-10-18T12:30:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672c680 0x555cd671a700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd6726820 0x555cd67262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 --2- 172.21.15.70:0/2804279660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd672bd60 0x555cd672f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 -- 172.21.15.70:0/2804279660 >> 172.21.15.70:0/2804279660 conn(0x555cd669a720 msgr2=0x555cd6720a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 -- 172.21.15.70:0/2804279660 shutdown_connections 2023-10-18T12:30:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 -- 172.21.15.70:0/2804279660 wait complete. 2023-10-18T12:30:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.905+0000 7f53017d3b80 1 -- start start 2023-10-18T12:30:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 Processor -- start 2023-10-18T12:30:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6735a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672bd60 0x555cd6735f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672c680 0x555cd67364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd6793c90 con 0x555cd6726820 2023-10-18T12:30:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd6793e00 con 0x555cd672bd60 2023-10-18T12:30:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cd6793fc0 con 0x555cd672c680 2023-10-18T12:30:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f53017d3b80 10 client.0 _ll_register_callbacks cb 0x555cd655fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6735a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6735a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47788/0 (socket says 172.21.15.70:47788) 2023-10-18T12:30:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f52e77fe700 1 -- 172.21.15.70:0/1998439390 learned_addr learned my addr 172.21.15.70:0/1998439390 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.906+0000 7f52e6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672bd60 0x555cd6735f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e7fff700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672c680 0x555cd67364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1998439390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672c680 msgr2=0x555cd67364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672c680 0x555cd67364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1998439390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 msgr2=0x555cd6735a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6735a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e6ffd700 1 -- 172.21.15.70:0/1998439390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cd64a3de0 con 0x555cd672bd60 2023-10-18T12:30:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e77fe700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6735a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52e6ffd700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672bd60 0x555cd6735f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52dc09a090 tx=0x7f52dc0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a285931454e273ad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52c6ffd700 1 -- 172.21.15.70:0/1998439390 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc09aa90 con 0x555cd672bd60 2023-10-18T12:30:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f52c6ffd700 1 -- 172.21.15.70:0/1998439390 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52dc0a2290 con 0x555cd672bd60 2023-10-18T12:30:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.907+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52dc0a2720 con 0x555cd672bd60 2023-10-18T12:30:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f52c6ffd700 1 -- 172.21.15.70:0/1998439390 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc0a2450 con 0x555cd672bd60 2023-10-18T12:30:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 10 client.26201 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f52e0009110 con 0x555cd672bd60 2023-10-18T12:30:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f52c6ffd700 1 -- 172.21.15.70:0/1998439390 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52dc0a2720 con 0x555cd672bd60 2023-10-18T12:30:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f52c6ffd700 1 -- 172.21.15.70:0/1998439390 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52dc07b070 con 0x555cd672bd60 2023-10-18T12:30:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 10 client.26201 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 20 client.26201 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd6795310 0x555cd6797700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 4 client.26201 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:30:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555cd6671e50 con 0x555cd6795310 2023-10-18T12:30:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.908+0000 7f52e77fe700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd6795310 0x555cd6797700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.909+0000 7f52c6ffd700 10 client.26201 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.909+0000 7f52e77fe700 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd6795310 0x555cd6797700 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261959187bebf4f6 server_cookie=a9f5ffae82f4ef78 in_seq=0 out_seq=0 2023-10-18T12:30:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.909+0000 7f52c6ffd700 1 -- 172.21.15.70:0/1998439390 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555cd6671e50 con 0x555cd6795310 2023-10-18T12:30:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.909+0000 7f52c6ffd700 10 client.26201 handle_command_reply: tid=0 2023-10-18T12:30:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.909+0000 7f53017d3b80 1 client.26201 shutdown 2023-10-18T12:30:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672bd60 msgr2=0x555cd6735f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672bd60 0x555cd6735f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52dc09a090 tx=0x7f52dc0092d0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 shutdown_connections 2023-10-18T12:30:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555cd6795310 0x555cd6797700 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cd672c680 0x555cd67364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555cd6726820 0x555cd6735a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f52c6ffd700 0 client.26201 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 --2- 172.21.15.70:0/1998439390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555cd672bd60 0x555cd6735f80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 >> 172.21.15.70:0/1998439390 conn(0x555cd669a720 msgr2=0x555cd671a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 shutdown_connections 2023-10-18T12:30:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 1 -- 172.21.15.70:0/1998439390 wait complete. 2023-10-18T12:30:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.910+0000 7f53017d3b80 20 client.26201 trim_cache size 0 max 16384 2023-10-18T12:30:14.926 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:30:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:30:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:30:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "2ada7f7c-48d1-4ec7-a637-73dfa6442b3c": { 2023-10-18T12:30:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:30:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "2ada7f7c-48d1-4ec7-a637-73dfa6442b3c", 2023-10-18T12:30:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:30:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:30:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:30:14.928 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:30:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d00a15f0 msgr2=0x7f52d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d00a15f0 0x7f52d00a3ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b2e0 tx=0x7f52d8005fd0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 msgr2=0x7f52e819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 0x7f52e819eef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00e9a0 tx=0x7f52dc00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 shutdown_connections 2023-10-18T12:30:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e8106680 0x7f52e8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e81005e0 0x7f52e819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f52d00a15f0 0x7f52d00a3ab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 --2- 172.21.15.70:0/3701258962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52e8100fa0 0x7f52e819eef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 >> 172.21.15.70:0/3701258962 conn(0x7f52e80fc0d0 msgr2=0x7f52e810c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 shutdown_connections 2023-10-18T12:30:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:14.911+0000 7f52ecb9d700 1 -- 172.21.15.70:0/3701258962 wait complete. 2023-10-18T12:30:14.934 INFO:tasks.cephfs.filesystem:scrub status for tag:2ada7f7c-48d1-4ec7-a637-73dfa6442b3c - {'path': '/', 'tag': '2ada7f7c-48d1-4ec7-a637-73dfa6442b3c', 'options': 'recursive,force'} 2023-10-18T12:30:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:15 smithi070 ceph-mon[98639]: pgmap v441: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2023-10-18T12:30:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:15 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:30:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:15 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:15 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:30:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:15 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:15 smithi152 ceph-mon[134767]: pgmap v441: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2023-10-18T12:30:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:15 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:30:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:15 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:15 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:30:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:15 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:15 smithi186 ceph-mon[135253]: pgmap v441: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2023-10-18T12:30:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:15 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:30:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:15 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:15 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:30:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:15 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:15.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 416492 13.15 MB/sec execute 44 sec latency 128.895 ms 2023-10-18T12:30:16.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 419209 13.17 MB/sec execute 45 sec latency 26.270 ms 2023-10-18T12:30:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:17 smithi070 ceph-mon[98639]: pgmap v442: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2023-10-18T12:30:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:17 smithi152 ceph-mon[134767]: pgmap v442: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2023-10-18T12:30:17.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:17 smithi186 ceph-mon[135253]: pgmap v442: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2023-10-18T12:30:17.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 422111 13.26 MB/sec execute 46 sec latency 60.638 ms 2023-10-18T12:30:18.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:18 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:30:18.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:18 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:30:18.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:18 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:30:18.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 424468 13.16 MB/sec execute 47 sec latency 8.390 ms 2023-10-18T12:30:19.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:19 smithi070 ceph-mon[98639]: pgmap v443: 129 pgs: 129 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 261 op/s 2023-10-18T12:30:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:19 smithi152 ceph-mon[134767]: pgmap v443: 129 pgs: 129 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 261 op/s 2023-10-18T12:30:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:19 smithi186 ceph-mon[135253]: pgmap v443: 129 pgs: 129 active+clean; 228 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 261 op/s 2023-10-18T12:30:19.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 426587 13.15 MB/sec execute 48 sec latency 12.344 ms 2023-10-18T12:30:20.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:20 smithi070 ceph-mon[98639]: pgmap v444: 129 pgs: 129 active+clean; 230 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 252 op/s 2023-10-18T12:30:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:20 smithi152 ceph-mon[134767]: pgmap v444: 129 pgs: 129 active+clean; 230 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 252 op/s 2023-10-18T12:30:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:20 smithi186 ceph-mon[135253]: pgmap v444: 129 pgs: 129 active+clean; 230 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 252 op/s 2023-10-18T12:30:20.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 429500 13.25 MB/sec execute 49 sec latency 39.914 ms 2023-10-18T12:30:21.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 431783 13.15 MB/sec execute 50 sec latency 62.035 ms 2023-10-18T12:30:22.846 INFO:tasks.workunit.client.0.smithi070.stdout: 1 433949 13.16 MB/sec execute 51 sec latency 16.161 ms 2023-10-18T12:30:23.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:22 smithi070 ceph-mon[98639]: pgmap v445: 129 pgs: 129 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 242 op/s 2023-10-18T12:30:23.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:22 smithi152 ceph-mon[134767]: pgmap v445: 129 pgs: 129 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 242 op/s 2023-10-18T12:30:23.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:22 smithi186 ceph-mon[135253]: pgmap v445: 129 pgs: 129 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 242 op/s 2023-10-18T12:30:23.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 436819 13.23 MB/sec execute 52 sec latency 9.326 ms 2023-10-18T12:30:24.845 INFO:tasks.workunit.client.0.smithi070.stdout: 1 439662 13.25 MB/sec execute 53 sec latency 30.814 ms 2023-10-18T12:30:25.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:25 smithi070 ceph-mon[98639]: pgmap v446: 129 pgs: 129 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2023-10-18T12:30:25.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:25 smithi152 ceph-mon[134767]: pgmap v446: 129 pgs: 129 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2023-10-18T12:30:25.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:25 smithi186 ceph-mon[135253]: pgmap v446: 129 pgs: 129 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2023-10-18T12:30:25.846 INFO:tasks.workunit.client.0.smithi070.stdout: 1 442151 13.22 MB/sec execute 54 sec latency 9.006 ms 2023-10-18T12:30:26.846 INFO:tasks.workunit.client.0.smithi070.stdout: 1 444126 13.21 MB/sec execute 55 sec latency 61.556 ms 2023-10-18T12:30:27.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:27 smithi070 ceph-mon[98639]: pgmap v447: 129 pgs: 129 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 255 op/s 2023-10-18T12:30:27.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:27 smithi152 ceph-mon[134767]: pgmap v447: 129 pgs: 129 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 255 op/s 2023-10-18T12:30:27.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:27 smithi186 ceph-mon[135253]: pgmap v447: 129 pgs: 129 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 255 op/s 2023-10-18T12:30:27.846 INFO:tasks.workunit.client.0.smithi070.stdout: 1 447158 13.30 MB/sec execute 56 sec latency 26.177 ms 2023-10-18T12:30:28.846 INFO:tasks.workunit.client.0.smithi070.stdout: 1 449528 13.22 MB/sec execute 57 sec latency 22.194 ms 2023-10-18T12:30:29.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:29 smithi070 ceph-mon[98639]: pgmap v448: 129 pgs: 129 active+clean; 244 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 266 op/s 2023-10-18T12:30:29.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:29 smithi152 ceph-mon[134767]: pgmap v448: 129 pgs: 129 active+clean; 244 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 266 op/s 2023-10-18T12:30:29.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:29 smithi186 ceph-mon[135253]: pgmap v448: 129 pgs: 129 active+clean; 244 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 266 op/s 2023-10-18T12:30:29.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 451674 13.22 MB/sec execute 58 sec latency 13.711 ms 2023-10-18T12:30:30.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 454563 13.28 MB/sec execute 59 sec latency 20.292 ms 2023-10-18T12:30:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:31 smithi070 ceph-mon[98639]: pgmap v449: 129 pgs: 129 active+clean; 244 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 267 op/s 2023-10-18T12:30:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:31 smithi152 ceph-mon[134767]: pgmap v449: 129 pgs: 129 active+clean; 244 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 267 op/s 2023-10-18T12:30:31.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:31 smithi186 ceph-mon[135253]: pgmap v449: 129 pgs: 129 active+clean; 244 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 267 op/s 2023-10-18T12:30:31.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 457232 13.28 MB/sec execute 60 sec latency 56.152 ms 2023-10-18T12:30:32.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 458615 13.33 MB/sec execute 61 sec latency 51.962 ms 2023-10-18T12:30:33.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:33 smithi070 ceph-mon[98639]: pgmap v450: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 278 op/s 2023-10-18T12:30:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:33 smithi152 ceph-mon[134767]: pgmap v450: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 278 op/s 2023-10-18T12:30:33.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:33 smithi186 ceph-mon[135253]: pgmap v450: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 278 op/s 2023-10-18T12:30:33.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 461533 13.50 MB/sec execute 62 sec latency 59.677 ms 2023-10-18T12:30:34.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 463562 13.47 MB/sec execute 63 sec latency 20.993 ms 2023-10-18T12:30:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:35 smithi070 ceph-mon[98639]: pgmap v451: 129 pgs: 129 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 239 op/s 2023-10-18T12:30:35.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:35 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:30:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:35 smithi152 ceph-mon[134767]: pgmap v451: 129 pgs: 129 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 239 op/s 2023-10-18T12:30:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:35 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:30:35.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:35 smithi186 ceph-mon[135253]: pgmap v451: 129 pgs: 129 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 239 op/s 2023-10-18T12:30:35.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:35 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:30:35.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 465916 13.47 MB/sec execute 64 sec latency 55.944 ms 2023-10-18T12:30:36.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:30:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:30:36.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:30:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:30:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:30:36.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:30:36.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:30:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:30:36.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:30:36.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 468455 13.46 MB/sec execute 65 sec latency 56.819 ms 2023-10-18T12:30:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:37 smithi070 ceph-mon[98639]: pgmap v452: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 280 op/s 2023-10-18T12:30:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:37 smithi152 ceph-mon[134767]: pgmap v452: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 280 op/s 2023-10-18T12:30:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:37 smithi186 ceph-mon[135253]: pgmap v452: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 280 op/s 2023-10-18T12:30:37.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 470446 13.43 MB/sec execute 66 sec latency 22.279 ms 2023-10-18T12:30:38.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 473206 13.44 MB/sec execute 67 sec latency 7.369 ms 2023-10-18T12:30:39.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:39 smithi070 ceph-mon[98639]: pgmap v453: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 246 op/s 2023-10-18T12:30:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:39 smithi152 ceph-mon[134767]: pgmap v453: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 246 op/s 2023-10-18T12:30:39.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:39 smithi186 ceph-mon[135253]: pgmap v453: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 246 op/s 2023-10-18T12:30:39.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 476312 13.52 MB/sec execute 68 sec latency 15.284 ms 2023-10-18T12:30:40.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 478686 13.44 MB/sec execute 69 sec latency 27.036 ms 2023-10-18T12:30:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:41 smithi070 ceph-mon[98639]: pgmap v454: 129 pgs: 129 active+clean; 260 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2023-10-18T12:30:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:41 smithi152 ceph-mon[134767]: pgmap v454: 129 pgs: 129 active+clean; 260 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2023-10-18T12:30:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:41 smithi186 ceph-mon[135253]: pgmap v454: 129 pgs: 129 active+clean; 260 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2023-10-18T12:30:41.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 480581 13.42 MB/sec execute 70 sec latency 51.374 ms 2023-10-18T12:30:42.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 483503 13.49 MB/sec execute 71 sec latency 31.857 ms 2023-10-18T12:30:43.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:43 smithi070 ceph-mon[98639]: pgmap v455: 129 pgs: 129 active+clean; 259 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-10-18T12:30:43.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:43 smithi152 ceph-mon[134767]: pgmap v455: 129 pgs: 129 active+clean; 259 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-10-18T12:30:43.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:43 smithi186 ceph-mon[135253]: pgmap v455: 129 pgs: 129 active+clean; 259 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 228 op/s 2023-10-18T12:30:43.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 486246 13.48 MB/sec execute 72 sec latency 6.284 ms 2023-10-18T12:30:44.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 488536 13.46 MB/sec execute 73 sec latency 8.405 ms 2023-10-18T12:30:44.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:30:45.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 -- 172.21.15.70:0/3940916993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 msgr2=0x7f606c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 --2- 172.21.15.70:0/3940916993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 0x7f606c106b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098f0 tx=0x7f605c009c00 comp rx=0 tx=0).stop 2023-10-18T12:30:45.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 -- 172.21.15.70:0/3940916993 shutdown_connections 2023-10-18T12:30:45.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 --2- 172.21.15.70:0/3940916993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f606c1010e0 0x7f606c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 --2- 172.21.15.70:0/3940916993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 0x7f606c106b10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 --2- 172.21.15.70:0/3940916993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 -- 172.21.15.70:0/3940916993 >> 172.21.15.70:0/3940916993 conn(0x7f606c0fc1b0 msgr2=0x7f606c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 -- 172.21.15.70:0/3940916993 shutdown_connections 2023-10-18T12:30:45.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.192+0000 7f60725ff700 1 -- 172.21.15.70:0/3940916993 wait complete. 2023-10-18T12:30:45.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.193+0000 7f60725ff700 1 Processor -- start 2023-10-18T12:30:45.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 -- start start 2023-10-18T12:30:45.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f606c1010e0 0x7f606c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 0x7f606c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1a4f90 con 0x7f606c1010e0 2023-10-18T12:30:45.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1a5100 con 0x7f606c106730 2023-10-18T12:30:45.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f60725ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1a52c0 con 0x7f606c100720 2023-10-18T12:30:45.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53860/0 (socket says 172.21.15.70:53860) 2023-10-18T12:30:45.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f606bfff700 1 -- 172.21.15.70:0/4188471981 learned_addr learned my addr 172.21.15.70:0/4188471981 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:45.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f606b7fe700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f606c1010e0 0x7f606c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.194+0000 7f6070b9c700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 0x7f606c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f606bfff700 1 -- 172.21.15.70:0/4188471981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 msgr2=0x7f606c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f606bfff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 0x7f606c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f606bfff700 1 -- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f606c1010e0 msgr2=0x7f606c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f606bfff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f606c1010e0 0x7f606c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f606bfff700 1 -- 172.21.15.70:0/4188471981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6064009d00 con 0x7f606c100720 2023-10-18T12:30:45.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f606bfff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c072af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f605c005950 tx=0x7f605c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f60697fa700 1 -- 172.21.15.70:0/4188471981 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c01c070 con 0x7f606c100720 2023-10-18T12:30:45.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f60697fa700 1 -- 172.21.15.70:0/4188471981 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c00baa0 con 0x7f606c100720 2023-10-18T12:30:45.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c009520 con 0x7f606c100720 2023-10-18T12:30:45.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.195+0000 7f60697fa700 1 -- 172.21.15.70:0/4188471981 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0038e0 con 0x7f606c100720 2023-10-18T12:30:45.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.196+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c1a58e0 con 0x7f606c100720 2023-10-18T12:30:45.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.198+0000 7f60697fa700 1 -- 172.21.15.70:0/4188471981 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f605c020030 con 0x7f606c100720 2023-10-18T12:30:45.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.200+0000 7f60697fa700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60540a1670 0x7f60540a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.200+0000 7f6085e72b80 1 Processor -- start 2023-10-18T12:30:45.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.200+0000 7f60697fa700 1 -- 172.21.15.70:0/4188471981 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f605c0ce0e0 con 0x7f606c100720 2023-10-18T12:30:45.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.200+0000 7f606b7fe700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60540a1670 0x7f60540a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.200+0000 7f6085e72b80 1 -- start start 2023-10-18T12:30:45.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 0x55db37b59fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b5a500 0x55db37b54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b54a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6085e72b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37ac76e0 con 0x55db37b5a500 2023-10-18T12:30:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6085e72b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37aadc60 con 0x55db37b545d0 2023-10-18T12:30:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6085e72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37aab580 con 0x55db37b59be0 2023-10-18T12:30:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 0x55db37b59fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 0x55db37b59fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53876/0 (socket says 172.21.15.70:53876) 2023-10-18T12:30:45.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 -- 172.21.15.70:0/94225366 learned_addr learned my addr 172.21.15.70:0/94225366 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:45.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b5a500 0x55db37b54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f6070b9c700 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b54a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606b7fe700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60540a1670 0x7f60540a3b30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f606c074220 tx=0x7f6060011720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 -- 172.21.15.70:0/94225366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 msgr2=0x55db37b54a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b54a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 -- 172.21.15.70:0/94225366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b5a500 msgr2=0x55db37b54090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b5a500 0x55db37b54090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.201+0000 7f606bfff700 1 -- 172.21.15.70:0/94225366 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db37a643f0 con 0x55db37b59be0 2023-10-18T12:30:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.202+0000 7f606bfff700 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 0x55db37b59fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f605c0972f0 tx=0x7f605c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bab13dd0a9e32ea server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.202+0000 7f60527fc700 1 -- 172.21.15.70:0/94225366 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c09adf0 con 0x55db37b59be0 2023-10-18T12:30:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.202+0000 7f60527fc700 1 -- 172.21.15.70:0/94225366 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c09ef40 con 0x55db37b59be0 2023-10-18T12:30:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.202+0000 7f60527fc700 1 -- 172.21.15.70:0/94225366 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0050d0 con 0x55db37b59be0 2023-10-18T12:30:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.202+0000 7f6085e72b80 1 -- 172.21.15.70:0/94225366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 msgr2=0x55db37b59fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.202+0000 7f6085e72b80 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 0x55db37b59fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f605c0972f0 tx=0x7f605c005040 comp rx=0 tx=0).stop 2023-10-18T12:30:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 -- 172.21.15.70:0/94225366 shutdown_connections 2023-10-18T12:30:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b54a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b59be0 0x55db37b59fc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 --2- 172.21.15.70:0/94225366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b5a500 0x55db37b54090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 -- 172.21.15.70:0/94225366 >> 172.21.15.70:0/94225366 conn(0x55db37ac8720 msgr2=0x55db37b4eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 -- 172.21.15.70:0/94225366 shutdown_connections 2023-10-18T12:30:45.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 -- 172.21.15.70:0/94225366 wait complete. 2023-10-18T12:30:45.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 -- start start 2023-10-18T12:30:45.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.203+0000 7f6085e72b80 1 Processor -- start 2023-10-18T12:30:45.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b5a500 0x55db37b5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37aa6e70 con 0x55db37b59be0 2023-10-18T12:30:45.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37a43710 con 0x55db37b545d0 2023-10-18T12:30:45.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37a433b0 con 0x55db37b5a500 2023-10-18T12:30:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6085e72b80 10 client.0 _ll_register_callbacks cb 0x55db3798da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f606bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36394/0 (socket says 172.21.15.70:36394) 2023-10-18T12:30:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f606b7fe700 1 -- 172.21.15.70:0/2700134966 learned_addr learned my addr 172.21.15.70:0/2700134966 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.204+0000 7f6070b9c700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b5a500 0x55db37b5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606bfff700 1 -- 172.21.15.70:0/2700134966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b5a500 msgr2=0x55db37b5dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606bfff700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b5a500 0x55db37b5dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606bfff700 1 -- 172.21.15.70:0/2700134966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 msgr2=0x55db37b5d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606bfff700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606bfff700 1 -- 172.21.15.70:0/2700134966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c1a58e0 con 0x55db37b545d0 2023-10-18T12:30:45.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606b7fe700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:45.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f606bfff700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b603e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f605c0972f0 tx=0x7f605c097730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d20bb396b481f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f604b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:45.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f604b7fe700 1 -- 172.21.15.70:0/2700134966 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0cb960 con 0x55db37b545d0 2023-10-18T12:30:45.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f604b7fe700 1 -- 172.21.15.70:0/2700134966 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f605c0a30f0 con 0x55db37b545d0 2023-10-18T12:30:45.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db37aabef0 con 0x55db37b545d0 2023-10-18T12:30:45.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f604b7fe700 1 -- 172.21.15.70:0/2700134966 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f605c09adf0 con 0x55db37b545d0 2023-10-18T12:30:45.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f6085e72b80 10 client.26213 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:45.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.205+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55db378d1de0 con 0x55db37b545d0 2023-10-18T12:30:45.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f604b7fe700 1 -- 172.21.15.70:0/2700134966 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db37aabef0 con 0x55db37b545d0 2023-10-18T12:30:45.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f604b7fe700 1 -- 172.21.15.70:0/2700134966 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f605c09b050 con 0x55db37b545d0 2023-10-18T12:30:45.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f6085e72b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:45.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f6085e72b80 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:45.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f6085e72b80 20 client.26213 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:45.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37b6d620 0x55db37b6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f6085e72b80 4 client.26213 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:45.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db37b238b0 con 0x55db37b6d620 2023-10-18T12:30:45.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.206+0000 7f606b7fe700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37b6d620 0x55db37b6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.207+0000 7f606b7fe700 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37b6d620 0x55db37b6fa10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7106cfd92855dbe8 server_cookie=de6ba5cbe0f8c59b in_seq=0 out_seq=0 2023-10-18T12:30:45.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.207+0000 7f604b7fe700 10 client.26213 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f604b7fe700 1 -- 172.21.15.70:0/2700134966 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55db37b238b0 con 0x55db37b6d620 2023-10-18T12:30:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f604b7fe700 10 client.26213 handle_command_reply: tid=0 2023-10-18T12:30:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 client.26213 shutdown 2023-10-18T12:30:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 msgr2=0x55db37b603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b603e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f605c0972f0 tx=0x7f605c097730 comp rx=0 tx=0).stop 2023-10-18T12:30:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 shutdown_connections 2023-10-18T12:30:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37b6d620 0x55db37b6fa10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b5a500 0x55db37b5dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b545d0 0x55db37b603e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f604b7fe700 0 client.26213 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2700134966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 >> 172.21.15.70:0/2700134966 conn(0x55db37ac8720 msgr2=0x55db37b53080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 shutdown_connections 2023-10-18T12:30:45.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 1 -- 172.21.15.70:0/2700134966 wait complete. 2023-10-18T12:30:45.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.208+0000 7f6085e72b80 20 client.26213 trim_cache size 0 max 16384 2023-10-18T12:30:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.228+0000 7f6085e72b80 1 Processor -- start 2023-10-18T12:30:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.228+0000 7f6085e72b80 1 -- start start 2023-10-18T12:30:45.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.228+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.228+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f6085e72b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37b4f9b0 con 0x55db37b59be0 2023-10-18T12:30:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f6085e72b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37927920 con 0x55db37b5a500 2023-10-18T12:30:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f6085e72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37b55030 con 0x55db37b545d0 2023-10-18T12:30:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f6070b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53904/0 (socket says 172.21.15.70:53904) 2023-10-18T12:30:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 -- 172.21.15.70:0/2955003179 learned_addr learned my addr 172.21.15.70:0/2955003179 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606b7fe700 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 -- 172.21.15.70:0/2955003179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 msgr2=0x55db37b5c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b5c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 -- 172.21.15.70:0/2955003179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 msgr2=0x55db37b5bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f606bfff700 1 -- 172.21.15.70:0/2955003179 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db378bac60 con 0x55db37b545d0 2023-10-18T12:30:45.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.229+0000 7f6070b9c700 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b5c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:45.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.230+0000 7f606bfff700 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b649f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f605c09ad80 tx=0x7f605c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6ce6834f73622e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.231+0000 7f6052ffd700 1 -- 172.21.15.70:0/2955003179 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0cb960 con 0x55db37b545d0 2023-10-18T12:30:45.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.231+0000 7f6052ffd700 1 -- 172.21.15.70:0/2955003179 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c0a1040 con 0x55db37b545d0 2023-10-18T12:30:45.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.231+0000 7f6052ffd700 1 -- 172.21.15.70:0/2955003179 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0050d0 con 0x55db37b545d0 2023-10-18T12:30:45.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 -- 172.21.15.70:0/2955003179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 msgr2=0x55db37b649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b649f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f605c09ad80 tx=0x7f605c009c00 comp rx=0 tx=0).stop 2023-10-18T12:30:45.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 -- 172.21.15.70:0/2955003179 shutdown_connections 2023-10-18T12:30:45.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b5c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b649f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 --2- 172.21.15.70:0/2955003179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b5bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 -- 172.21.15.70:0/2955003179 >> 172.21.15.70:0/2955003179 conn(0x55db37ac8720 msgr2=0x55db37b53890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 -- 172.21.15.70:0/2955003179 shutdown_connections 2023-10-18T12:30:45.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 -- 172.21.15.70:0/2955003179 wait complete. 2023-10-18T12:30:45.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.232+0000 7f6085e72b80 1 -- start start 2023-10-18T12:30:45.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 Processor -- start 2023-10-18T12:30:45.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b64be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b69340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37b69a80 con 0x55db37b59be0 2023-10-18T12:30:45.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37b69bf0 con 0x55db37b5a500 2023-10-18T12:30:45.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db37b69db0 con 0x55db37b545d0 2023-10-18T12:30:45.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6085e72b80 10 client.0 _ll_register_callbacks cb 0x55db3798da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:45.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b64be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f606bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b64be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36418/0 (socket says 172.21.15.70:36418) 2023-10-18T12:30:45.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f606b7fe700 1 -- 172.21.15.70:0/3684926444 learned_addr learned my addr 172.21.15.70:0/3684926444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:45.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.233+0000 7f6070b9c700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b69340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606b7fe700 1 -- 172.21.15.70:0/3684926444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 msgr2=0x55db37b646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606b7fe700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606b7fe700 1 -- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 msgr2=0x55db37b69340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:45.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606b7fe700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b69340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606b7fe700 1 -- 172.21.15.70:0/3684926444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6064003a00 con 0x55db37b59be0 2023-10-18T12:30:45.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606bfff700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:45.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f606b7fe700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b64be0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6060011b20 tx=0x7f60600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e006e7bc4ab7f66 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f604b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:30:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f604b7fe700 1 -- 172.21.15.70:0/3684926444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060023040 con 0x55db37b59be0 2023-10-18T12:30:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f604b7fe700 1 -- 172.21.15.70:0/3684926444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6060029470 con 0x55db37b59be0 2023-10-18T12:30:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.234+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f605c09ef40 con 0x55db37b59be0 2023-10-18T12:30:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f604b7fe700 1 -- 172.21.15.70:0/3684926444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606001e510 con 0x55db37b59be0 2023-10-18T12:30:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 10 client.16194 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f605c07e070 con 0x55db37b59be0 2023-10-18T12:30:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f604b7fe700 1 -- 172.21.15.70:0/3684926444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c09ef40 con 0x55db37b59be0 2023-10-18T12:30:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f604b7fe700 1 -- 172.21.15.70:0/3684926444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6060005040 con 0x55db37b59be0 2023-10-18T12:30:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 20 client.16194 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37bc2810 0x55db37bc4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 4 client.16194 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:30:45.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db37a9fe50 con 0x55db37bc2810 2023-10-18T12:30:45.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.235+0000 7f606bfff700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37bc2810 0x55db37bc4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.236+0000 7f604b7fe700 10 client.16194 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:45.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.236+0000 7f606bfff700 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37bc2810 0x55db37bc4c30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf83210915d28cf2 server_cookie=6d1ee0c457f33e7a in_seq=0 out_seq=0 2023-10-18T12:30:45.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.237+0000 7f604b7fe700 1 -- 172.21.15.70:0/3684926444 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55db37a9fe50 con 0x55db37bc2810 2023-10-18T12:30:45.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.237+0000 7f604b7fe700 10 client.16194 handle_command_reply: tid=0 2023-10-18T12:30:45.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.237+0000 7f6085e72b80 1 client.16194 shutdown 2023-10-18T12:30:45.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 msgr2=0x55db37b64be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b64be0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6060011b20 tx=0x7f60600047c0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 shutdown_connections 2023-10-18T12:30:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55db37bc2810 0x55db37bc4c30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db37b5a500 0x55db37b69340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db37b545d0 0x55db37b646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 --2- 172.21.15.70:0/3684926444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55db37b59be0 0x55db37b64be0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 >> 172.21.15.70:0/3684926444 conn(0x55db37ac8720 msgr2=0x55db37b5bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f604b7fe700 0 client.16194 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:45.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 shutdown_connections 2023-10-18T12:30:45.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 1 -- 172.21.15.70:0/3684926444 wait complete. 2023-10-18T12:30:45.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.238+0000 7f6085e72b80 20 client.16194 trim_cache size 0 max 16384 2023-10-18T12:30:45.254 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:30:45.254 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:30:45.254 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:30:45.254 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:30:45.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60540a1670 msgr2=0x7f60540a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60540a1670 0x7f60540a3b30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f606c074220 tx=0x7f6060011720 comp rx=0 tx=0).stop 2023-10-18T12:30:45.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 msgr2=0x7f606c072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c072af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f605c005950 tx=0x7f605c00fa00 comp rx=0 tx=0).stop 2023-10-18T12:30:45.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 shutdown_connections 2023-10-18T12:30:45.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60540a1670 0x7f60540a3b30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f606c106730 0x7f606c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f606c100720 0x7f606c072af0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 --2- 172.21.15.70:0/4188471981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f606c1010e0 0x7f606c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 >> 172.21.15.70:0/4188471981 conn(0x7f606c0fc1b0 msgr2=0x7f606c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 shutdown_connections 2023-10-18T12:30:45.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.239+0000 7f60725ff700 1 -- 172.21.15.70:0/4188471981 wait complete. 2023-10-18T12:30:45.260 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:30:45.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:30:45.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:45 smithi070 ceph-mon[98639]: pgmap v456: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 259 op/s 2023-10-18T12:30:45.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 -- 172.21.15.70:0/2976791250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60241010b0 msgr2=0x7f602410d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 --2- 172.21.15.70:0/2976791250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60241010b0 0x7f602410d5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f601c0089e0 tx=0x7f601c008cf0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 -- 172.21.15.70:0/2976791250 shutdown_connections 2023-10-18T12:30:45.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 --2- 172.21.15.70:0/2976791250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60241010b0 0x7f602410d5c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 --2- 172.21.15.70:0/2976791250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6024106700 0x7f6024106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 --2- 172.21.15.70:0/2976791250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60241006f0 0x7f6024100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 -- 172.21.15.70:0/2976791250 >> 172.21.15.70:0/2976791250 conn(0x7f60240fc1a0 msgr2=0x7f60240fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 -- 172.21.15.70:0/2976791250 shutdown_connections 2023-10-18T12:30:45.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.472+0000 7f602c1ff700 1 -- 172.21.15.70:0/2976791250 wait complete. 2023-10-18T12:30:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 Processor -- start 2023-10-18T12:30:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 -- start start 2023-10-18T12:30:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60241006f0 0x7f602419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60241010b0 0x7f602419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 0x7f60241946b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6024194e80 con 0x7f60241006f0 2023-10-18T12:30:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6024194ff0 con 0x7f6024106700 2023-10-18T12:30:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.473+0000 7f602c1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60241951b0 con 0x7f60241010b0 2023-10-18T12:30:45.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 0x7f60241946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f6029f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60241006f0 0x7f602419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 0x7f60241946b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39584/0 (socket says 172.21.15.70:39584) 2023-10-18T12:30:45.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 -- 172.21.15.70:0/1162184684 learned_addr learned my addr 172.21.15.70:0/1162184684 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:45.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602979a700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60241010b0 0x7f602419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 -- 172.21.15.70:0/1162184684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60241010b0 msgr2=0x7f602419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60241010b0 0x7f602419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 -- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60241006f0 msgr2=0x7f602419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60241006f0 0x7f602419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 -- 172.21.15.70:0/1162184684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c008610 con 0x7f6024106700 2023-10-18T12:30:45.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f602a79c700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 0x7f60241946b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f601c00d040 tx=0x7f601c00d8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.474+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00d2d0 con 0x7f6024106700 2023-10-18T12:30:45.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.475+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f601c00d430 con 0x7f6024106700 2023-10-18T12:30:45.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.475+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0097b0 con 0x7f6024106700 2023-10-18T12:30:45.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.475+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6024195430 con 0x7f6024106700 2023-10-18T12:30:45.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.475+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60241a1590 con 0x7f6024106700 2023-10-18T12:30:45.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.476+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6024108d10 con 0x7f6024106700 2023-10-18T12:30:45.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.478+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f601c013020 con 0x7f6024106700 2023-10-18T12:30:45.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.479+0000 7f601affd700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60100a15f0 0x7f60100a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:45.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.479+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f601c0cdad0 con 0x7f6024106700 2023-10-18T12:30:45.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.479+0000 7f6029f9b700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60100a15f0 0x7f60100a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:45.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.480+0000 7f6029f9b700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60100a15f0 0x7f60100a3ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f60200044f0 tx=0x7f6020013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:45.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.483+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f601c096c30 con 0x7f6024106700 2023-10-18T12:30:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:45 smithi152 ceph-mon[134767]: pgmap v456: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 259 op/s 2023-10-18T12:30:45.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:45 smithi186 ceph-mon[135253]: pgmap v456: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 259 op/s 2023-10-18T12:30:45.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.753+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6024101e30 con 0x7f6024106700 2023-10-18T12:30:45.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.754+0000 7f601affd700 1 -- 172.21.15.70:0/1162184684 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f601c09a250 con 0x7f6024106700 2023-10-18T12:30:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:30:45.757 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:30:45.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60100a15f0 msgr2=0x7f60100a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60100a15f0 0x7f60100a3ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f60200044f0 tx=0x7f6020013040 comp rx=0 tx=0).stop 2023-10-18T12:30:45.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 msgr2=0x7f60241946b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:45.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 0x7f60241946b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f601c00d040 tx=0x7f601c00d8e0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 shutdown_connections 2023-10-18T12:30:45.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60100a15f0 0x7f60100a3ab0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6024106700 0x7f60241946b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60241006f0 0x7f602419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 --2- 172.21.15.70:0/1162184684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60241010b0 0x7f602419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:45.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 >> 172.21.15.70:0/1162184684 conn(0x7f60240fc1a0 msgr2=0x7f60240fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:45.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 shutdown_connections 2023-10-18T12:30:45.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:45.756+0000 7f602c1ff700 1 -- 172.21.15.70:0/1162184684 wait complete. 2023-10-18T12:30:45.761 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:30:45.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:30:45.847 INFO:tasks.workunit.client.0.smithi070.stdout: 1 490902 13.46 MB/sec execute 74 sec latency 27.102 ms 2023-10-18T12:30:46.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 -- 172.21.15.70:0/466925932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 msgr2=0x7f15f810f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 --2- 172.21.15.70:0/466925932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 0x7f15f810f6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b0a0 tx=0x7f15f000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 -- 172.21.15.70:0/466925932 shutdown_connections 2023-10-18T12:30:46.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 --2- 172.21.15.70:0/466925932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 0x7f15f810f6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 --2- 172.21.15.70:0/466925932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 0x7f15f8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 --2- 172.21.15.70:0/466925932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 0x7f15f8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.017+0000 7f15fd59d700 1 -- 172.21.15.70:0/466925932 >> 172.21.15.70:0/466925932 conn(0x7f15f80fe290 msgr2=0x7f15f81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:46.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.018+0000 7f15fd59d700 1 -- 172.21.15.70:0/466925932 shutdown_connections 2023-10-18T12:30:46.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.018+0000 7f15fd59d700 1 -- 172.21.15.70:0/466925932 wait complete. 2023-10-18T12:30:46.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 Processor -- start 2023-10-18T12:30:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 -- start start 2023-10-18T12:30:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 0x7f15f819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 0x7f15f819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 0x7f15f81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f81991c0 con 0x7f15f81027e0 2023-10-18T12:30:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8199330 con 0x7f15f81087f0 2023-10-18T12:30:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.019+0000 7f15fd59d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f81994f0 con 0x7f15f81031a0 2023-10-18T12:30:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 0x7f15f819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 0x7f15f81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 0x7f15f819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36466/0 (socket says 172.21.15.70:36466) 2023-10-18T12:30:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15f7fff700 1 -- 172.21.15.70:0/1672600993 learned_addr learned my addr 172.21.15.70:0/1672600993 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15effff700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 0x7f15f819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 -- 172.21.15.70:0/1672600993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 msgr2=0x7f15f819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 0x7f15f819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 -- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 msgr2=0x7f15f819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 0x7f15f819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 -- 172.21.15.70:0/1672600993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009d00 con 0x7f15f81087f0 2023-10-18T12:30:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 0x7f15f81989f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b070 tx=0x7f15f00077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15f5ffb700 1 -- 172.21.15.70:0/1672600993 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f000e040 con 0x7f15f81087f0 2023-10-18T12:30:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f8199770 con 0x7f15f81087f0 2023-10-18T12:30:46.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15f5ffb700 1 -- 172.21.15.70:0/1672600993 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f0004e60 con 0x7f15f81087f0 2023-10-18T12:30:46.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.020+0000 7f15f5ffb700 1 -- 172.21.15.70:0/1672600993 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0005040 con 0x7f15f81087f0 2023-10-18T12:30:46.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.021+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f81a58d0 con 0x7f15f81087f0 2023-10-18T12:30:46.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.024+0000 7f15f5ffb700 1 -- 172.21.15.70:0/1672600993 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f15f001d470 con 0x7f15f81087f0 2023-10-18T12:30:46.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 Processor -- start 2023-10-18T12:30:46.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f15f5ffb700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15e40a1670 0x7f15e40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 -- start start 2023-10-18T12:30:46.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f15f5ffb700 1 -- 172.21.15.70:0/1672600993 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f15f00ce7e0 con 0x7f15f81087f0 2023-10-18T12:30:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f15effff700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15e40a1670 0x7f15e40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be50140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be4ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bdbd6e0 con 0x559e7be4fd60 2023-10-18T12:30:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bda3c60 con 0x559e7be4a820 2023-10-18T12:30:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.025+0000 7f16121e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bda1580 con 0x559e7be50680 2023-10-18T12:30:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be50140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be50140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36480/0 (socket says 172.21.15.70:36480) 2023-10-18T12:30:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 -- 172.21.15.70:0/1401812361 learned_addr learned my addr 172.21.15.70:0/1401812361 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15effff700 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be4ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 -- 172.21.15.70:0/1401812361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 msgr2=0x559e7be4a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be4a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 -- 172.21.15.70:0/1401812361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 msgr2=0x559e7be4ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be4ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 -- 172.21.15.70:0/1401812361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e7bd5a3f0 con 0x559e7be4fd60 2023-10-18T12:30:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15effff700 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:30:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15effff700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15e40a1670 0x7f15e40a3b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f15f819a160 tx=0x7f15e00118e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.026+0000 7f15f7fff700 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be50140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009dd0 tx=0x7f15e8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67527e8914c3aa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.027+0000 7f15ee7fc700 1 -- 172.21.15.70:0/1401812361 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8004a30 con 0x559e7be4fd60 2023-10-18T12:30:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.027+0000 7f15ee7fc700 1 -- 172.21.15.70:0/1401812361 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e80054e0 con 0x559e7be4fd60 2023-10-18T12:30:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.027+0000 7f15ee7fc700 1 -- 172.21.15.70:0/1401812361 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80197d0 con 0x559e7be4fd60 2023-10-18T12:30:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 -- 172.21.15.70:0/1401812361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 msgr2=0x559e7be50140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be50140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009dd0 tx=0x7f15e8011720 comp rx=0 tx=0).stop 2023-10-18T12:30:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 -- 172.21.15.70:0/1401812361 shutdown_connections 2023-10-18T12:30:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be4ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be50140 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 --2- 172.21.15.70:0/1401812361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 -- 172.21.15.70:0/1401812361 >> 172.21.15.70:0/1401812361 conn(0x559e7bdbe720 msgr2=0x559e7be44cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 -- 172.21.15.70:0/1401812361 shutdown_connections 2023-10-18T12:30:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 -- 172.21.15.70:0/1401812361 wait complete. 2023-10-18T12:30:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.028+0000 7f16121e6b80 1 -- start start 2023-10-18T12:30:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 Processor -- start 2023-10-18T12:30:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4a820 0x559e7be565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bd9ce70 con 0x559e7be4a820 2023-10-18T12:30:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bd39710 con 0x559e7be4fd60 2023-10-18T12:30:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bd393b0 con 0x559e7be50680 2023-10-18T12:30:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f16121e6b80 10 client.0 _ll_register_callbacks cb 0x559e7bc83a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4a820 0x559e7be565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f15effff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f15effff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39606/0 (socket says 172.21.15.70:39606) 2023-10-18T12:30:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f15effff700 1 -- 172.21.15.70:0/861458719 learned_addr learned my addr 172.21.15.70:0/861458719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.029+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15effff700 1 -- 172.21.15.70:0/861458719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 msgr2=0x559e7be53f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15effff700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be53f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15effff700 1 -- 172.21.15.70:0/861458719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4a820 msgr2=0x559e7be565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15effff700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4a820 0x559e7be565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15effff700 1 -- 172.21.15.70:0/861458719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8009a30 con 0x559e7be4fd60 2023-10-18T12:30:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15f7fff700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4a820 0x559e7be565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15effff700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be538f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15e0004640 tx=0x7f15e00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfe4ec419d26534 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15d77fe700 1 -- 172.21.15.70:0/861458719 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0029660 con 0x559e7be4fd60 2023-10-18T12:30:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e7bda1ef0 con 0x559e7be4fd60 2023-10-18T12:30:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.030+0000 7f15d77fe700 1 -- 172.21.15.70:0/861458719 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f15e00297c0 con 0x559e7be4fd60 2023-10-18T12:30:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f15d77fe700 1 -- 172.21.15.70:0/861458719 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e0029980 con 0x559e7be4fd60 2023-10-18T12:30:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 10 client.26237 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559e7bbc7de0 con 0x559e7be4fd60 2023-10-18T12:30:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f15d77fe700 1 -- 172.21.15.70:0/861458719 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e7bda1ef0 con 0x559e7be4fd60 2023-10-18T12:30:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f15d77fe700 1 -- 172.21.15.70:0/861458719 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f15e001d4e0 con 0x559e7be4fd60 2023-10-18T12:30:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 10 client.26237 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 20 client.26237 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7be63fc0 0x559e7be663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 4 client.26237 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e7be198b0 con 0x559e7be63fc0 2023-10-18T12:30:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.031+0000 7f15f7fff700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7be63fc0 0x559e7be663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.032+0000 7f15d77fe700 10 client.26237 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.032+0000 7f15f7fff700 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7be63fc0 0x559e7be663e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2133fe91b44cafea server_cookie=9ee6fabfa1172a0a in_seq=0 out_seq=0 2023-10-18T12:30:46.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.032+0000 7f15d77fe700 1 -- 172.21.15.70:0/861458719 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559e7be198b0 con 0x559e7be63fc0 2023-10-18T12:30:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.032+0000 7f15d77fe700 10 client.26237 handle_command_reply: tid=0 2023-10-18T12:30:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 client.26237 shutdown 2023-10-18T12:30:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 msgr2=0x559e7be538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be538f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15e0004640 tx=0x7f15e00047c0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 shutdown_connections 2023-10-18T12:30:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7be63fc0 0x559e7be663e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be53f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4a820 0x559e7be565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 --2- 172.21.15.70:0/861458719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be538f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 >> 172.21.15.70:0/861458719 conn(0x559e7bdbe720 msgr2=0x559e7be492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f15d77fe700 0 client.26237 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 shutdown_connections 2023-10-18T12:30:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 1 -- 172.21.15.70:0/861458719 wait complete. 2023-10-18T12:30:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.033+0000 7f16121e6b80 20 client.26237 trim_cache size 0 max 16384 2023-10-18T12:30:46.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.048+0000 7f16121e6b80 1 Processor -- start 2023-10-18T12:30:46.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.048+0000 7f16121e6b80 1 -- start start 2023-10-18T12:30:46.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.048+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 0x559e7be4dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be5b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be50680 0x559e7be5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f16121e6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7be3fd40 con 0x559e7be50680 2023-10-18T12:30:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f16121e6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7bc1d920 con 0x559e7be4fd60 2023-10-18T12:30:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f16121e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7be54810 con 0x559e7be4a820 2023-10-18T12:30:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 0x559e7be4dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15effff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be5b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 0x559e7be4dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53954/0 (socket says 172.21.15.70:53954) 2023-10-18T12:30:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15f7fff700 1 -- 172.21.15.70:0/894437578 learned_addr learned my addr 172.21.15.70:0/894437578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be50680 0x559e7be5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15effff700 1 -- 172.21.15.70:0/894437578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 msgr2=0x559e7be4dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15effff700 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 0x559e7be4dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15effff700 1 -- 172.21.15.70:0/894437578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be50680 msgr2=0x559e7be5f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15effff700 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be50680 0x559e7be5f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15effff700 1 -- 172.21.15.70:0/894437578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e7bbb0c60 con 0x559e7be4fd60 2023-10-18T12:30:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.049+0000 7f15f7fff700 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 0x559e7be4dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.050+0000 7f15effff700 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be5b060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009db0 tx=0x7f15e0009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cbf6448bfc9216 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.050+0000 7f15eeffd700 1 -- 172.21.15.70:0/894437578 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00092d0 con 0x559e7be4fd60 2023-10-18T12:30:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.050+0000 7f15eeffd700 1 -- 172.21.15.70:0/894437578 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e0016070 con 0x559e7be4fd60 2023-10-18T12:30:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.050+0000 7f15eeffd700 1 -- 172.21.15.70:0/894437578 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e001d620 con 0x559e7be4fd60 2023-10-18T12:30:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 -- 172.21.15.70:0/894437578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 msgr2=0x559e7be5b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be5b060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009db0 tx=0x7f15e0009d20 comp rx=0 tx=0).stop 2023-10-18T12:30:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 -- 172.21.15.70:0/894437578 shutdown_connections 2023-10-18T12:30:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be50680 0x559e7be5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be4a820 0x559e7be4dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 --2- 172.21.15.70:0/894437578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4fd60 0x559e7be5b060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 -- 172.21.15.70:0/894437578 >> 172.21.15.70:0/894437578 conn(0x559e7bdbe720 msgr2=0x559e7be4d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 -- 172.21.15.70:0/894437578 shutdown_connections 2023-10-18T12:30:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 -- 172.21.15.70:0/894437578 wait complete. 2023-10-18T12:30:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.051+0000 7f16121e6b80 1 -- start start 2023-10-18T12:30:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 Processor -- start 2023-10-18T12:30:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7be59b40 con 0x559e7be4fd60 2023-10-18T12:30:46.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7be59cb0 con 0x559e7be4a820 2023-10-18T12:30:46.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7be59e70 con 0x559e7be50680 2023-10-18T12:30:46.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.052+0000 7f16121e6b80 10 client.0 _ll_register_callbacks cb 0x559e7bc83a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:46.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be5ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39626/0 (socket says 172.21.15.70:39626) 2023-10-18T12:30:46.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 -- 172.21.15.70:0/3714203225 learned_addr learned my addr 172.21.15.70:0/3714203225 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:46.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15fcd9c700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15effff700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 -- 172.21.15.70:0/3714203225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 msgr2=0x559e7be59600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be59600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 -- 172.21.15.70:0/3714203225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 msgr2=0x559e7be590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 -- 172.21.15.70:0/3714203225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e7bd5a3f0 con 0x559e7be4a820 2023-10-18T12:30:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15f7fff700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be5ac00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009310 tx=0x7f15e80052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd8e6f07ee23010 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.053+0000 7f15d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f15d77fe700 1 -- 172.21.15.70:0/3714203225 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003a90 con 0x559e7be4a820 2023-10-18T12:30:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15e80054e0 con 0x559e7be4a820 2023-10-18T12:30:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f15d77fe700 1 -- 172.21.15.70:0/3714203225 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f15e8029070 con 0x559e7be4a820 2023-10-18T12:30:46.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f15d77fe700 1 -- 172.21.15.70:0/3714203225 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e8003bf0 con 0x559e7be4a820 2023-10-18T12:30:46.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 10 client.26243 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:46.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f15e002e040 con 0x559e7be4a820 2023-10-18T12:30:46.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f15d77fe700 1 -- 172.21.15.70:0/3714203225 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e80054e0 con 0x559e7be4a820 2023-10-18T12:30:46.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f15d77fe700 1 -- 172.21.15.70:0/3714203225 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f15e8011440 con 0x559e7be4a820 2023-10-18T12:30:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 10 client.26243 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 20 client.26243 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7beb9230 0x559e7bebb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 4 client.26243 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:30:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e7bd95e50 con 0x559e7beb9230 2023-10-18T12:30:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.054+0000 7f15effff700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7beb9230 0x559e7bebb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.055+0000 7f15d77fe700 10 client.26243 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.055+0000 7f15effff700 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7beb9230 0x559e7bebb650 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0d805e309835aa server_cookie=6c17aa95db23c87b in_seq=0 out_seq=0 2023-10-18T12:30:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.055+0000 7f15d77fe700 1 -- 172.21.15.70:0/3714203225 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e7bd95e50 con 0x559e7beb9230 2023-10-18T12:30:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.055+0000 7f15d77fe700 10 client.26243 handle_command_reply: tid=0 2023-10-18T12:30:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.055+0000 7f16121e6b80 1 client.26243 shutdown 2023-10-18T12:30:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 msgr2=0x559e7be5ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be5ac00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009310 tx=0x7f15e80052e0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 shutdown_connections 2023-10-18T12:30:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559e7beb9230 0x559e7bebb650 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e7be50680 0x559e7be59600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f15d77fe700 0 client.26243 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559e7be4a820 0x559e7be5ac00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 --2- 172.21.15.70:0/3714203225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e7be4fd60 0x559e7be590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 >> 172.21.15.70:0/3714203225 conn(0x559e7bdbe720 msgr2=0x559e7be3e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 shutdown_connections 2023-10-18T12:30:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 1 -- 172.21.15.70:0/3714203225 wait complete. 2023-10-18T12:30:46.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.056+0000 7f16121e6b80 20 client.26243 trim_cache size 0 max 16384 2023-10-18T12:30:46.075 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:30:46.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15e40a1670 msgr2=0x7f15e40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15e40a1670 0x7f15e40a3b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f15f819a160 tx=0x7f15e00118e0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 msgr2=0x7f15f81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:46.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 0x7f15f81989f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b070 tx=0x7f15f00077f0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 shutdown_connections 2023-10-18T12:30:46.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15e40a1670 0x7f15e40a3b30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f81087f0 0x7f15f81989f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f81027e0 0x7f15f819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 --2- 172.21.15.70:0/1672600993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15f81031a0 0x7f15f819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:46.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 >> 172.21.15.70:0/1672600993 conn(0x7f15f80fe290 msgr2=0x7f15f80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:46.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 shutdown_connections 2023-10-18T12:30:46.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:46.057+0000 7f15fd59d700 1 -- 172.21.15.70:0/1672600993 wait complete. 2023-10-18T12:30:46.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:46 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1162184684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:30:46.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:46 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1162184684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:30:46.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:46 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1162184684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:30:46.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 493550 13.46 MB/sec execute 75 sec latency 61.382 ms 2023-10-18T12:30:47.080 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:30:47.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:30:47.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 -- 172.21.15.70:0/1875871205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5270072c20 msgr2=0x7f5270073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 --2- 172.21.15.70:0/1875871205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5270072c20 0x7f5270073000 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f526c009990 tx=0x7f526c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 -- 172.21.15.70:0/1875871205 shutdown_connections 2023-10-18T12:30:47.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 --2- 172.21.15.70:0/1875871205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f527010cf60 0x7f527010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 --2- 172.21.15.70:0/1875871205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5270072c20 0x7f5270073000 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 --2- 172.21.15.70:0/1875871205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270073540 0x7f527010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.288+0000 7f52783ff700 1 -- 172.21.15.70:0/1875871205 >> 172.21.15.70:0/1875871205 conn(0x7f52700fc1b0 msgr2=0x7f52700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.289+0000 7f52783ff700 1 -- 172.21.15.70:0/1875871205 shutdown_connections 2023-10-18T12:30:47.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.289+0000 7f52783ff700 1 -- 172.21.15.70:0/1875871205 wait complete. 2023-10-18T12:30:47.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.290+0000 7f52783ff700 1 Processor -- start 2023-10-18T12:30:47.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.290+0000 7f52783ff700 1 -- start start 2023-10-18T12:30:47.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f52783ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270072c20 0x7f5270198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f52783ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 0x7f5270199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f52783ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527010cf60 0x7f527019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f52783ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527019dd40 con 0x7f527010cf60 2023-10-18T12:30:47.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f52783ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527019deb0 con 0x7f5270073540 2023-10-18T12:30:47.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f52783ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527019e070 con 0x7f5270072c20 2023-10-18T12:30:47.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527599a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 0x7f5270199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527699c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527010cf60 0x7f527019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527599a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 0x7f5270199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39638/0 (socket says 172.21.15.70:39638) 2023-10-18T12:30:47.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527619b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270072c20 0x7f5270198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527599a700 1 -- 172.21.15.70:0/3516243337 learned_addr learned my addr 172.21.15.70:0/3516243337 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527699c700 1 -- 172.21.15.70:0/3516243337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270072c20 msgr2=0x7f5270198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527699c700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270072c20 0x7f5270198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527699c700 1 -- 172.21.15.70:0/3516243337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 msgr2=0x7f5270199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527699c700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 0x7f5270199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527699c700 1 -- 172.21.15.70:0/3516243337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268009c40 con 0x7f527010cf60 2023-10-18T12:30:47.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527619b700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270072c20 0x7f5270198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:47.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.291+0000 7f527599a700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 0x7f5270199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:47.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.292+0000 7f527699c700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527010cf60 0x7f527019d700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f526800efb0 tx=0x7f526800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.292+0000 7f52677fe700 1 -- 172.21.15.70:0/3516243337 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268003a00 con 0x7f527010cf60 2023-10-18T12:30:47.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.292+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c0095f0 con 0x7f527010cf60 2023-10-18T12:30:47.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.292+0000 7f52677fe700 1 -- 172.21.15.70:0/3516243337 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526800f040 con 0x7f527010cf60 2023-10-18T12:30:47.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.292+0000 7f52677fe700 1 -- 172.21.15.70:0/3516243337 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268004540 con 0x7f527010cf60 2023-10-18T12:30:47.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.292+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52701a59b0 con 0x7f527010cf60 2023-10-18T12:30:47.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.295+0000 7f52677fe700 1 -- 172.21.15.70:0/3516243337 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5268003bf0 con 0x7f527010cf60 2023-10-18T12:30:47.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.296+0000 7f528bd2cb80 1 Processor -- start 2023-10-18T12:30:47.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.296+0000 7f52677fe700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f525c0a15f0 0x7f525c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f52677fe700 1 -- 172.21.15.70:0/3516243337 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52680ccff0 con 0x7f527010cf60 2023-10-18T12:30:47.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 -- start start 2023-10-18T12:30:47.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527619b700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f525c0a15f0 0x7f525c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228233c60 0x55e228234040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228234580 0x55e22822e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e22822ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2281a1760 con 0x55e22822e650 2023-10-18T12:30:47.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228187ce0 con 0x55e228233c60 2023-10-18T12:30:47.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228185600 con 0x55e228234580 2023-10-18T12:30:47.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527699c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e22822ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228234580 0x55e22822e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527699c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e22822ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36520/0 (socket says 172.21.15.70:36520) 2023-10-18T12:30:47.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228234580 0x55e22822e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:53986/0 (socket says 172.21.15.70:53986) 2023-10-18T12:30:47.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527699c700 1 -- 172.21.15.70:0/1714498674 learned_addr learned my addr 172.21.15.70:0/1714498674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.297+0000 7f527619b700 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228233c60 0x55e228234040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527619b700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f525c0a15f0 0x7f525c0a3ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b2e0 tx=0x7f526c004520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527699c700 1 -- 172.21.15.70:0/1714498674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228234580 msgr2=0x55e22822e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527699c700 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228234580 0x55e22822e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527699c700 1 -- 172.21.15.70:0/1714498674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228233c60 msgr2=0x55e228234040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527699c700 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228233c60 0x55e228234040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527699c700 1 -- 172.21.15.70:0/1714498674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e22813e470 con 0x55e22822e650 2023-10-18T12:30:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527619b700 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228233c60 0x55e228234040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:47.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f527699c700 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e22822ead0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f52680cd250 tx=0x7f5268096150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee027c5e128a237 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f5264bfe700 1 -- 172.21.15.70:0/1714498674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268099230 con 0x55e22822e650 2023-10-18T12:30:47.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f5264bfe700 1 -- 172.21.15.70:0/1714498674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526809a440 con 0x55e22822e650 2023-10-18T12:30:47.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.298+0000 7f5264bfe700 1 -- 172.21.15.70:0/1714498674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268096450 con 0x55e22822e650 2023-10-18T12:30:47.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1714498674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 msgr2=0x55e22822ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e22822ead0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f52680cd250 tx=0x7f5268096150 comp rx=0 tx=0).stop 2023-10-18T12:30:47.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1714498674 shutdown_connections 2023-10-18T12:30:47.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e22822ead0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228233c60 0x55e228234040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1714498674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228234580 0x55e22822e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1714498674 >> 172.21.15.70:0/1714498674 conn(0x55e2281a27a0 msgr2=0x55e228228be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1714498674 shutdown_connections 2023-10-18T12:30:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.299+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1714498674 wait complete. 2023-10-18T12:30:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 -- start start 2023-10-18T12:30:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 Processor -- start 2023-10-18T12:30:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 0x55e22823a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e228237820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228234580 0x55e228237f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228180ef0 con 0x55e228234580 2023-10-18T12:30:47.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22811d790 con 0x55e22822e650 2023-10-18T12:30:47.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22811d430 con 0x55e228233c60 2023-10-18T12:30:47.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.300+0000 7f528bd2cb80 10 client.0 _ll_register_callbacks cb 0x55e228067b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:47.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 0x55e22823a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 0x55e22823a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39668/0 (socket says 172.21.15.70:39668) 2023-10-18T12:30:47.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 -- 172.21.15.70:0/1363793847 learned_addr learned my addr 172.21.15.70:0/1363793847 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527599a700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e228237820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527699c700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228234580 0x55e228237f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 -- 172.21.15.70:0/1363793847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 msgr2=0x55e228237820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e228237820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 -- 172.21.15.70:0/1363793847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228234580 msgr2=0x55e228237f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228234580 0x55e228237f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 -- 172.21.15.70:0/1363793847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5270110090 con 0x55e22822e650 2023-10-18T12:30:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527599a700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e228237820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f527619b700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 0x55e22823a5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f526c009990 tx=0x7f526c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3718875f7a5dbbeb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.301+0000 7f5259ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f5259ffb700 1 -- 172.21.15.70:0/1363793847 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c005260 con 0x55e22822e650 2023-10-18T12:30:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e228185f70 con 0x55e22822e650 2023-10-18T12:30:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f5259ffb700 1 -- 172.21.15.70:0/1363793847 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f526c02c040 con 0x55e22822e650 2023-10-18T12:30:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f5259ffb700 1 -- 172.21.15.70:0/1363793847 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f526c0053c0 con 0x55e22822e650 2023-10-18T12:30:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 10 client.26255 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f52701a59b0 con 0x55e22822e650 2023-10-18T12:30:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f5259ffb700 1 -- 172.21.15.70:0/1363793847 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e228185f70 con 0x55e22822e650 2023-10-18T12:30:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f5259ffb700 1 -- 172.21.15.70:0/1363793847 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f526c0305b0 con 0x55e22822e650 2023-10-18T12:30:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 10 client.26255 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 20 client.26255 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.302+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e228247ef0 0x55e22824a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.303+0000 7f528bd2cb80 4 client.26255 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:47.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.303+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2281fd930 con 0x55e228247ef0 2023-10-18T12:30:47.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.303+0000 7f527599a700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e228247ef0 0x55e22824a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.303+0000 7f5259ffb700 10 client.26255 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.303+0000 7f527599a700 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e228247ef0 0x55e22824a310 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494205c3f60a4866 server_cookie=7b5e6d00fb14cc14 in_seq=0 out_seq=0 2023-10-18T12:30:47.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f5259ffb700 1 -- 172.21.15.70:0/1363793847 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2281fd930 con 0x55e228247ef0 2023-10-18T12:30:47.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f5259ffb700 10 client.26255 handle_command_reply: tid=0 2023-10-18T12:30:47.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 client.26255 shutdown 2023-10-18T12:30:47.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 msgr2=0x55e22823a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 0x55e22823a5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f526c009990 tx=0x7f526c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 shutdown_connections 2023-10-18T12:30:47.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e228247ef0 0x55e22824a310 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228234580 0x55e228237f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22822e650 0x55e22823a5a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1363793847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e228237820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 >> 172.21.15.70:0/1363793847 conn(0x55e2281a27a0 msgr2=0x55e22822d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f5259ffb700 0 client.26255 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 shutdown_connections 2023-10-18T12:30:47.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1363793847 wait complete. 2023-10-18T12:30:47.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.305+0000 7f528bd2cb80 20 client.26255 trim_cache size 0 max 16384 2023-10-18T12:30:47.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 Processor -- start 2023-10-18T12:30:47.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 -- start start 2023-10-18T12:30:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e22822e650 0x55e228231eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 0x55e22823ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e228237100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2280019a0 con 0x55e228233c60 2023-10-18T12:30:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228238560 con 0x55e228234580 2023-10-18T12:30:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228229920 con 0x55e22822e650 2023-10-18T12:30:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f527599a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 0x55e22823ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f527619b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e22822e650 0x55e228231eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f527599a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 0x55e22823ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36548/0 (socket says 172.21.15.70:36548) 2023-10-18T12:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.324+0000 7f527599a700 1 -- 172.21.15.70:0/841839881 learned_addr learned my addr 172.21.15.70:0/841839881 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527699c700 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e228237100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527599a700 1 -- 172.21.15.70:0/841839881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e22822e650 msgr2=0x55e228231eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527599a700 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e22822e650 0x55e228231eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527599a700 1 -- 172.21.15.70:0/841839881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 msgr2=0x55e228237100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527599a700 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e228237100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527599a700 1 -- 172.21.15.70:0/841839881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e227fabe60 con 0x55e228233c60 2023-10-18T12:30:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527619b700 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e22822e650 0x55e228231eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f527599a700 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 0x55e22823ef30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f52600096b0 tx=0x7f5260009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3149d8860925de97 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f52653ff700 1 -- 172.21.15.70:0/841839881 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260017040 con 0x55e228233c60 2023-10-18T12:30:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f52653ff700 1 -- 172.21.15.70:0/841839881 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5260025030 con 0x55e228233c60 2023-10-18T12:30:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.325+0000 7f52653ff700 1 -- 172.21.15.70:0/841839881 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260004a70 con 0x55e228233c60 2023-10-18T12:30:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 -- 172.21.15.70:0/841839881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 msgr2=0x55e22823ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 0x55e22823ef30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f52600096b0 tx=0x7f5260009260 comp rx=0 tx=0).stop 2023-10-18T12:30:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 -- 172.21.15.70:0/841839881 shutdown_connections 2023-10-18T12:30:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e228237100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e22822e650 0x55e228231eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/841839881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e228233c60 0x55e22823ef30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 -- 172.21.15.70:0/841839881 >> 172.21.15.70:0/841839881 conn(0x55e2281a27a0 msgr2=0x55e2282317a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 -- 172.21.15.70:0/841839881 shutdown_connections 2023-10-18T12:30:47.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 -- 172.21.15.70:0/841839881 wait complete. 2023-10-18T12:30:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.326+0000 7f528bd2cb80 1 -- start start 2023-10-18T12:30:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 Processor -- start 2023-10-18T12:30:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e228296370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e2282968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e22823e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22823e870 con 0x55e22822e650 2023-10-18T12:30:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22823e9e0 con 0x55e228234580 2023-10-18T12:30:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22823eba0 con 0x55e228233c60 2023-10-18T12:30:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f528bd2cb80 10 client.0 _ll_register_callbacks cb 0x55e228067b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f527619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e228296370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f527619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e228296370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36558/0 (socket says 172.21.15.70:36558) 2023-10-18T12:30:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f527619b700 1 -- 172.21.15.70:0/1385658484 learned_addr learned my addr 172.21.15.70:0/1385658484 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f527599a700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e2282968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.327+0000 7f527699c700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e22823e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f527619b700 1 -- 172.21.15.70:0/1385658484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 msgr2=0x55e2282968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f527619b700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e2282968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f527619b700 1 -- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 msgr2=0x55e22823e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f527619b700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e22823e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f527619b700 1 -- 172.21.15.70:0/1385658484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52701a59b0 con 0x55e22822e650 2023-10-18T12:30:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f527619b700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e228296370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f526c0098e0 tx=0x7f526c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4ccd98056c0be5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f5259ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:30:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f5259ffb700 1 -- 172.21.15.70:0/1385658484 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c030d30 con 0x55e22822e650 2023-10-18T12:30:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f526809a440 con 0x55e22822e650 2023-10-18T12:30:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f5259ffb700 1 -- 172.21.15.70:0/1385658484 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f526c005260 con 0x55e22822e650 2023-10-18T12:30:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f5259ffb700 1 -- 172.21.15.70:0/1385658484 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f526c005440 con 0x55e22822e650 2023-10-18T12:30:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f528bd2cb80 10 client.16236 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.328+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f5260009e80 con 0x55e22822e650 2023-10-18T12:30:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f5259ffb700 1 -- 172.21.15.70:0/1385658484 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526809a440 con 0x55e22822e650 2023-10-18T12:30:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f5259ffb700 1 -- 172.21.15.70:0/1385658484 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f526c00f7b0 con 0x55e22822e650 2023-10-18T12:30:47.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f528bd2cb80 10 client.16236 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:47.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f528bd2cb80 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:47.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f528bd2cb80 20 client.16236 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:47.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e22829de30 0x55e2282a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f528bd2cb80 4 client.16236 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:30:47.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e228179ed0 con 0x55e22829de30 2023-10-18T12:30:47.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.329+0000 7f527599a700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e22829de30 0x55e2282a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.330+0000 7f5259ffb700 10 client.16236 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.330+0000 7f527599a700 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e22829de30 0x55e2282a0250 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5427338662dfef3 server_cookie=df011b4b874d3d35 in_seq=0 out_seq=0 2023-10-18T12:30:47.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f5259ffb700 1 -- 172.21.15.70:0/1385658484 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e228179ed0 con 0x55e22829de30 2023-10-18T12:30:47.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f5259ffb700 10 client.16236 handle_command_reply: tid=0 2023-10-18T12:30:47.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 client.16236 shutdown 2023-10-18T12:30:47.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 msgr2=0x55e228296370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e228296370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f526c0098e0 tx=0x7f526c00bde0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 shutdown_connections 2023-10-18T12:30:47.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e22829de30 0x55e2282a0250 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e228234580 0x55e22823e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e22822e650 0x55e228296370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 --2- 172.21.15.70:0/1385658484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e228233c60 0x55e2282968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 >> 172.21.15.70:0/1385658484 conn(0x55e2281a27a0 msgr2=0x55e228222530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f5259ffb700 0 client.16236 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 shutdown_connections 2023-10-18T12:30:47.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.331+0000 7f528bd2cb80 1 -- 172.21.15.70:0/1385658484 wait complete. 2023-10-18T12:30:47.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f528bd2cb80 20 client.16236 trim_cache size 0 max 16384 2023-10-18T12:30:47.347 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:30:47.347 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:30:47.348 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "9bbc8234-4a72-44cb-931c-0cb72f78598c", 2023-10-18T12:30:47.348 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:30:47.348 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:30:47.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f525c0a15f0 msgr2=0x7f525c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f525c0a15f0 0x7f525c0a3ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f526c00b2e0 tx=0x7f526c004520 comp rx=0 tx=0).stop 2023-10-18T12:30:47.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527010cf60 msgr2=0x7f527019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527010cf60 0x7f527019d700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f526800efb0 tx=0x7f526800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 shutdown_connections 2023-10-18T12:30:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527010cf60 0x7f527019d700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5270072c20 0x7f5270198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f525c0a15f0 0x7f525c0a3ab0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 --2- 172.21.15.70:0/3516243337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5270073540 0x7f5270199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 >> 172.21.15.70:0/3516243337 conn(0x7f52700fc1b0 msgr2=0x7f52700fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 shutdown_connections 2023-10-18T12:30:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.332+0000 7f52783ff700 1 -- 172.21.15.70:0/3516243337 wait complete. 2023-10-18T12:30:47.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:30:47.415 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:47 smithi070 ceph-mon[98639]: pgmap v457: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-10-18T12:30:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.560+0000 7ff72bdff700 1 -- 172.21.15.70:0/2206637549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100740 msgr2=0x7ff724100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.560+0000 7ff72bdff700 1 --2- 172.21.15.70:0/2206637549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100740 0x7ff724100bc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7140098f0 tx=0x7ff714009c00 comp rx=0 tx=0).stop 2023-10-18T12:30:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 -- 172.21.15.70:0/2206637549 shutdown_connections 2023-10-18T12:30:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 --2- 172.21.15.70:0/2206637549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff724101100 0x7ff72410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 --2- 172.21.15.70:0/2206637549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724106750 0x7ff724106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 --2- 172.21.15.70:0/2206637549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100740 0x7ff724100bc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 -- 172.21.15.70:0/2206637549 >> 172.21.15.70:0/2206637549 conn(0x7ff7240fc230 msgr2=0x7ff7240fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 -- 172.21.15.70:0/2206637549 shutdown_connections 2023-10-18T12:30:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.561+0000 7ff72bdff700 1 -- 172.21.15.70:0/2206637549 wait complete. 2023-10-18T12:30:47.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 Processor -- start 2023-10-18T12:30:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 -- start start 2023-10-18T12:30:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 0x7ff72419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff724101100 0x7ff72419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724106750 0x7ff724196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724197260 con 0x7ff724106750 2023-10-18T12:30:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241973d0 con 0x7ff724101100 2023-10-18T12:30:47.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.562+0000 7ff72bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724197590 con 0x7ff724100740 2023-10-18T12:30:47.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 0x7ff72419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 0x7ff72419c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:54040/0 (socket says 172.21.15.70:54040) 2023-10-18T12:30:47.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 -- 172.21.15.70:0/1387642703 learned_addr learned my addr 172.21.15.70:0/1387642703 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff72939a700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff724101100 0x7ff72419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff72a39c700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724106750 0x7ff724196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 -- 172.21.15.70:0/1387642703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff724101100 msgr2=0x7ff72419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff724101100 0x7ff72419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 -- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724106750 msgr2=0x7ff724196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724106750 0x7ff724196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 -- 172.21.15.70:0/1387642703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200095f0 con 0x7ff724100740 2023-10-18T12:30:47.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.563+0000 7ff729b9b700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 0x7ff72419c9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff72000ec10 tx=0x7ff72000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.564+0000 7ff71affd700 1 -- 172.21.15.70:0/1387642703 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000cc90 con 0x7ff724100740 2023-10-18T12:30:47.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.564+0000 7ff71affd700 1 -- 172.21.15.70:0/1387642703 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7200044e0 con 0x7ff724100740 2023-10-18T12:30:47.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.564+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff714009520 con 0x7ff724100740 2023-10-18T12:30:47.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.564+0000 7ff71affd700 1 -- 172.21.15.70:0/1387642703 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720010430 con 0x7ff724100740 2023-10-18T12:30:47.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.564+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7241a3970 con 0x7ff724100740 2023-10-18T12:30:47.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.567+0000 7ff71affd700 1 -- 172.21.15.70:0/1387642703 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff720003fb0 con 0x7ff724100740 2023-10-18T12:30:47.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.568+0000 7ff71affd700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff7100a15f0 0x7ff7100a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.568+0000 7ff71affd700 1 -- 172.21.15.70:0/1387642703 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff720014070 con 0x7ff724100740 2023-10-18T12:30:47.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 Processor -- start 2023-10-18T12:30:47.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 -- start start 2023-10-18T12:30:47.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff72939a700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff7100a15f0 0x7ff7100a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aebf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aeb94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 0x558a7aeb9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ae286e0 con 0x558a7aebf910 2023-10-18T12:30:47.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ae0ec60 con 0x558a7aeb99e0 2023-10-18T12:30:47.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.569+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ae0c580 con 0x558a7aebeff0 2023-10-18T12:30:47.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aebf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 0x558a7aeb9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 0x558a7aeb9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39716/0 (socket says 172.21.15.70:39716) 2023-10-18T12:30:47.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72939a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aeb94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 -- 172.21.15.70:0/4079326460 learned_addr learned my addr 172.21.15.70:0/4079326460 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72939a700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff7100a15f0 0x7ff7100a3ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7140098c0 tx=0x7ff714019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 -- 172.21.15.70:0/4079326460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 msgr2=0x558a7aebf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aebf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 -- 172.21.15.70:0/4079326460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 msgr2=0x558a7aeb94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aeb94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 -- 172.21.15.70:0/4079326460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7adc53f0 con 0x558a7aeb99e0 2023-10-18T12:30:47.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.570+0000 7ff72a39c700 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 0x558a7aeb9e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff71c011600 tx=0x7ff71c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25554b7170438ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.571+0000 7ff70ffff700 1 -- 172.21.15.70:0/4079326460 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00e040 con 0x558a7aeb99e0 2023-10-18T12:30:47.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.571+0000 7ff70ffff700 1 -- 172.21.15.70:0/4079326460 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff71c00f040 con 0x558a7aeb99e0 2023-10-18T12:30:47.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.571+0000 7ff70ffff700 1 -- 172.21.15.70:0/4079326460 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c004800 con 0x558a7aeb99e0 2023-10-18T12:30:47.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.571+0000 7ff73f61fb80 1 -- 172.21.15.70:0/4079326460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 msgr2=0x558a7aeb9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.571+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 0x558a7aeb9e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff71c011600 tx=0x7ff71c009920 comp rx=0 tx=0).stop 2023-10-18T12:30:47.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 -- 172.21.15.70:0/4079326460 shutdown_connections 2023-10-18T12:30:47.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aeb99e0 0x558a7aeb9e60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aebf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/4079326460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aeb94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 -- 172.21.15.70:0/4079326460 >> 172.21.15.70:0/4079326460 conn(0x558a7ae29720 msgr2=0x558a7aeafb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 -- 172.21.15.70:0/4079326460 shutdown_connections 2023-10-18T12:30:47.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 -- 172.21.15.70:0/4079326460 wait complete. 2023-10-18T12:30:47.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.572+0000 7ff73f61fb80 1 -- start start 2023-10-18T12:30:47.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 Processor -- start 2023-10-18T12:30:47.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aec3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ae07e70 con 0x558a7aebf910 2023-10-18T12:30:47.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ada4710 con 0x558a7aebeff0 2023-10-18T12:30:47.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.573+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ada43b0 con 0x558a7aeb99e0 2023-10-18T12:30:47.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff73f61fb80 10 client.0 _ll_register_callbacks cb 0x558a7aceea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39718/0 (socket says 172.21.15.70:39718) 2023-10-18T12:30:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:54050/0 (socket says 172.21.15.70:54050) 2023-10-18T12:30:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 -- 172.21.15.70:0/2584189439 learned_addr learned my addr 172.21.15.70:0/2584189439 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72a39c700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aec3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 -- 172.21.15.70:0/2584189439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 msgr2=0x558a7aec58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 -- 172.21.15.70:0/2584189439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 msgr2=0x558a7aec3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aec3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff72939a700 1 -- 172.21.15.70:0/2584189439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c0112b0 con 0x558a7aebeff0 2023-10-18T12:30:47.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.574+0000 7ff729b9b700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:30:47.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff72939a700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec2c60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff714009230 tx=0x7ff714009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd80d73e358bf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff70d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:30:47.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff70d7fa700 1 -- 172.21.15.70:0/2584189439 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71402a050 con 0x558a7aebeff0 2023-10-18T12:30:47.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a7ae0cef0 con 0x558a7aebeff0 2023-10-18T12:30:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff70d7fa700 1 -- 172.21.15.70:0/2584189439 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff71400f910 con 0x558a7aebeff0 2023-10-18T12:30:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff70d7fa700 1 -- 172.21.15.70:0/2584189439 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff714005040 con 0x558a7aebeff0 2023-10-18T12:30:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff73f61fb80 10 client.26273 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558a7ac32de0 con 0x558a7aebeff0 2023-10-18T12:30:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff70d7fa700 1 -- 172.21.15.70:0/2584189439 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a7ae0cef0 con 0x558a7aebeff0 2023-10-18T12:30:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.575+0000 7ff70d7fa700 1 -- 172.21.15.70:0/2584189439 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff714037070 con 0x558a7aebeff0 2023-10-18T12:30:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff73f61fb80 10 client.26273 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff73f61fb80 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff73f61fb80 20 client.26273 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7aed3200 0x558a7aed5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff73f61fb80 4 client.26273 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:30:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a7ae848b0 con 0x558a7aed3200 2023-10-18T12:30:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff729b9b700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7aed3200 0x558a7aed5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff70d7fa700 10 client.26273 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.576+0000 7ff729b9b700 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7aed3200 0x558a7aed5620 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130c72526019988a server_cookie=bf28b682972c9c24 in_seq=0 out_seq=0 2023-10-18T12:30:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff70d7fa700 1 -- 172.21.15.70:0/2584189439 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a7ae848b0 con 0x558a7aed3200 2023-10-18T12:30:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff70d7fa700 10 client.26273 handle_command_reply: tid=0 2023-10-18T12:30:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 client.26273 shutdown 2023-10-18T12:30:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 msgr2=0x558a7aec2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec2c60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff714009230 tx=0x7ff714009c00 comp rx=0 tx=0).stop 2023-10-18T12:30:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 shutdown_connections 2023-10-18T12:30:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7aed3200 0x558a7aed5620 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7aec3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2584189439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec2c60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 >> 172.21.15.70:0/2584189439 conn(0x558a7ae29720 msgr2=0x558a7aeb8490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff70d7fa700 0 client.26273 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.578+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 shutdown_connections 2023-10-18T12:30:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.579+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2584189439 wait complete. 2023-10-18T12:30:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.579+0000 7ff73f61fb80 20 client.26273 trim_cache size 0 max 16384 2023-10-18T12:30:47.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 Processor -- start 2023-10-18T12:30:47.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 -- start start 2023-10-18T12:30:47.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aeb99e0 0x558a7aeca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aec21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebf910 0x558a7aeaf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ac88920 con 0x558a7aeb99e0 2023-10-18T12:30:47.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7aeb13b0 con 0x558a7aebf910 2023-10-18T12:30:47.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7aeb94a0 con 0x558a7aebeff0 2023-10-18T12:30:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.599+0000 7ff72939a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aec21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aec21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:54064/0 (socket says 172.21.15.70:54064) 2023-10-18T12:30:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 -- 172.21.15.70:0/2437319280 learned_addr learned my addr 172.21.15.70:0/2437319280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72a39c700 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebf910 0x558a7aeaf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff729b9b700 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aeb99e0 0x558a7aeca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 -- 172.21.15.70:0/2437319280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebf910 msgr2=0x558a7aeaf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebf910 0x558a7aeaf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 -- 172.21.15.70:0/2437319280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aeb99e0 msgr2=0x558a7aeca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aeb99e0 0x558a7aeca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 -- 172.21.15.70:0/2437319280 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7ac1bc60 con 0x558a7aebeff0 2023-10-18T12:30:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.600+0000 7ff72939a700 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aec21c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff714009fd0 tx=0x7ff7140059e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee58ff587067dfb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff718bff700 1 -- 172.21.15.70:0/2437319280 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71402a070 con 0x558a7aebeff0 2023-10-18T12:30:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff718bff700 1 -- 172.21.15.70:0/2437319280 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff71402b040 con 0x558a7aebeff0 2023-10-18T12:30:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff718bff700 1 -- 172.21.15.70:0/2437319280 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140053d0 con 0x558a7aebeff0 2023-10-18T12:30:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2437319280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 msgr2=0x558a7aec21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aec21c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff714009fd0 tx=0x7ff7140059e0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2437319280 shutdown_connections 2023-10-18T12:30:47.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebf910 0x558a7aeaf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aeb99e0 0x558a7aeca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/2437319280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aebeff0 0x558a7aec21c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2437319280 >> 172.21.15.70:0/2437319280 conn(0x558a7ae29720 msgr2=0x558a7aed5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.601+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2437319280 shutdown_connections 2023-10-18T12:30:47.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.602+0000 7ff73f61fb80 1 -- 172.21.15.70:0/2437319280 wait complete. 2023-10-18T12:30:47.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.602+0000 7ff73f61fb80 1 -- start start 2023-10-18T12:30:47.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 Processor -- start 2023-10-18T12:30:47.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7af26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7af26590 con 0x558a7aebf910 2023-10-18T12:30:47.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7af266d0 con 0x558a7aebeff0 2023-10-18T12:30:47.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7af26860 con 0x558a7aeb99e0 2023-10-18T12:30:47.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff73f61fb80 10 client.0 _ll_register_callbacks cb 0x558a7aceea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:30:47.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff729b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:54068/0 (socket says 172.21.15.70:54068) 2023-10-18T12:30:47.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff729b9b700 1 -- 172.21.15.70:0/1218330085 learned_addr learned my addr 172.21.15.70:0/1218330085 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:30:47.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff72939a700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.603+0000 7ff72a39c700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7af26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff729b9b700 1 -- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 msgr2=0x558a7aec8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff729b9b700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff729b9b700 1 -- 172.21.15.70:0/1218330085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 msgr2=0x558a7af26050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:30:47.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff729b9b700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7af26050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff729b9b700 1 -- 172.21.15.70:0/1218330085 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a7ac32de0 con 0x558a7aeb99e0 2023-10-18T12:30:47.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff729b9b700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec84c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff72009a560 tx=0x7ff720085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131bcfe4c16de4c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:30:47.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff70d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:30:47.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff70d7fa700 1 -- 172.21.15.70:0/1218330085 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720099f30 con 0x558a7aeb99e0 2023-10-18T12:30:47.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff70d7fa700 1 -- 172.21.15.70:0/1218330085 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff720016330 con 0x558a7aeb99e0 2023-10-18T12:30:47.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.604+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff71c00f040 con 0x558a7aeb99e0 2023-10-18T12:30:47.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff70d7fa700 1 -- 172.21.15.70:0/1218330085 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff72009acd0 con 0x558a7aeb99e0 2023-10-18T12:30:47.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 10 client.26278 fetch_fsmap learned FSMap version 20 2023-10-18T12:30:47.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558a7ac59250 con 0x558a7aeb99e0 2023-10-18T12:30:47.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff70d7fa700 1 -- 172.21.15.70:0/1218330085 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff71c00f040 con 0x558a7aeb99e0 2023-10-18T12:30:47.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff70d7fa700 1 -- 172.21.15.70:0/1218330085 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff72009a830 con 0x558a7aeb99e0 2023-10-18T12:30:47.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 10 client.26278 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:30:47.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:30:47.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 20 client.26278 populate_metadata read hostname 'smithi070' 2023-10-18T12:30:47.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7af280c0 0x558a7af2a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:30:47.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 4 client.26278 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:30:47.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558a7ae00e50 con 0x558a7af280c0 2023-10-18T12:30:47.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.605+0000 7ff72939a700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7af280c0 0x558a7af2a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:30:47.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff70d7fa700 10 client.26278 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff72939a700 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7af280c0 0x558a7af2a4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc26eb6d8dd6f24 server_cookie=f1743ac445b267f in_seq=0 out_seq=0 2023-10-18T12:30:47.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff70d7fa700 1 -- 172.21.15.70:0/1218330085 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558a7ae00e50 con 0x558a7af280c0 2023-10-18T12:30:47.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff70d7fa700 10 client.26278 handle_command_reply: tid=0 2023-10-18T12:30:47.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 client.26278 shutdown 2023-10-18T12:30:47.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 msgr2=0x558a7aec84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec84c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff72009a560 tx=0x7ff720085040 comp rx=0 tx=0).stop 2023-10-18T12:30:47.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 shutdown_connections 2023-10-18T12:30:47.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558a7af280c0 0x558a7af2a4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a7aebf910 0x558a7af26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a7aeb99e0 0x558a7aec84c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 --2- 172.21.15.70:0/1218330085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a7aebeff0 0x558a7aec8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff70d7fa700 0 client.26278 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:30:47.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.606+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 >> 172.21.15.70:0/1218330085 conn(0x558a7ae29720 msgr2=0x558a7aeaf810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 shutdown_connections 2023-10-18T12:30:47.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff73f61fb80 1 -- 172.21.15.70:0/1218330085 wait complete. 2023-10-18T12:30:47.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff73f61fb80 20 client.26278 trim_cache size 0 max 16384 2023-10-18T12:30:47.623 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:30:47.623 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:30:47.624 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:30:47.624 INFO:teuthology.orchestra.run.smithi070.stdout: "9bbc8234-4a72-44cb-931c-0cb72f78598c": { 2023-10-18T12:30:47.624 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:30:47.624 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "9bbc8234-4a72-44cb-931c-0cb72f78598c", 2023-10-18T12:30:47.624 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:30:47.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:30:47.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:30:47.625 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:30:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff7100a15f0 msgr2=0x7ff7100a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff7100a15f0 0x7ff7100a3ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7140098c0 tx=0x7ff714019040 comp rx=0 tx=0).stop 2023-10-18T12:30:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 msgr2=0x7ff72419c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:30:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 0x7ff72419c9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff72000ec10 tx=0x7ff72000ef20 comp rx=0 tx=0).stop 2023-10-18T12:30:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 shutdown_connections 2023-10-18T12:30:47.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724106750 0x7ff724196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff724100740 0x7ff72419c9f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff7100a15f0 0x7ff7100a3ab0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 --2- 172.21.15.70:0/1387642703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff724101100 0x7ff72419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:30:47.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 >> 172.21.15.70:0/1387642703 conn(0x7ff7240fc230 msgr2=0x7ff7240fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:30:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 shutdown_connections 2023-10-18T12:30:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:30:47.607+0000 7ff72bdff700 1 -- 172.21.15.70:0/1387642703 wait complete. 2023-10-18T12:30:47.629 INFO:tasks.cephfs.filesystem:scrub status for tag:9bbc8234-4a72-44cb-931c-0cb72f78598c - {'path': '/', 'tag': '9bbc8234-4a72-44cb-931c-0cb72f78598c', 'options': 'recursive,force'} 2023-10-18T12:30:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:47 smithi152 ceph-mon[134767]: pgmap v457: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-10-18T12:30:47.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:47 smithi186 ceph-mon[135253]: pgmap v457: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 248 op/s 2023-10-18T12:30:47.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 495410 13.43 MB/sec execute 76 sec latency 18.232 ms 2023-10-18T12:30:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:48 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:30:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:48 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:48 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:30:48.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:48 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:48.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:48 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:30:48.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:48 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:30:48.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:48 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:48.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:48 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:30:48.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:48 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:48.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:48 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:30:48.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:48 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:30:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:48 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:48 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:30:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:48 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:30:48.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:48 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:30:48.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 498088 13.44 MB/sec execute 77 sec latency 9.872 ms 2023-10-18T12:30:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:49 smithi070 ceph-mon[98639]: pgmap v458: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2023-10-18T12:30:49.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:49 smithi152 ceph-mon[134767]: pgmap v458: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2023-10-18T12:30:49.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:49 smithi186 ceph-mon[135253]: pgmap v458: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2023-10-18T12:30:49.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 501191 13.50 MB/sec execute 78 sec latency 8.129 ms 2023-10-18T12:30:50.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 503699 13.47 MB/sec execute 79 sec latency 28.134 ms 2023-10-18T12:30:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:51 smithi070 ceph-mon[98639]: pgmap v459: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2023-10-18T12:30:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:51 smithi152 ceph-mon[134767]: pgmap v459: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2023-10-18T12:30:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:51 smithi186 ceph-mon[135253]: pgmap v459: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2023-10-18T12:30:51.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 505527 13.43 MB/sec execute 80 sec latency 50.323 ms 2023-10-18T12:30:52.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 508299 13.47 MB/sec execute 81 sec latency 19.854 ms 2023-10-18T12:30:53.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:53 smithi070 ceph-mon[98639]: pgmap v460: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 269 op/s 2023-10-18T12:30:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:53 smithi152 ceph-mon[134767]: pgmap v460: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 269 op/s 2023-10-18T12:30:53.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:53 smithi186 ceph-mon[135253]: pgmap v460: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 269 op/s 2023-10-18T12:30:53.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 511013 13.47 MB/sec execute 82 sec latency 8.813 ms 2023-10-18T12:30:54.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 513470 13.45 MB/sec execute 83 sec latency 6.910 ms 2023-10-18T12:30:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:55 smithi070 ceph-mon[98639]: pgmap v461: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 304 op/s 2023-10-18T12:30:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:55 smithi152 ceph-mon[134767]: pgmap v461: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 304 op/s 2023-10-18T12:30:55.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:55 smithi186 ceph-mon[135253]: pgmap v461: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 304 op/s 2023-10-18T12:30:55.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 515937 13.46 MB/sec execute 84 sec latency 8.735 ms 2023-10-18T12:30:56.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 518499 13.46 MB/sec execute 85 sec latency 68.344 ms 2023-10-18T12:30:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:57 smithi070 ceph-mon[98639]: pgmap v462: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2023-10-18T12:30:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:57 smithi152 ceph-mon[134767]: pgmap v462: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2023-10-18T12:30:57.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:57 smithi186 ceph-mon[135253]: pgmap v462: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2023-10-18T12:30:57.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 520449 13.42 MB/sec execute 86 sec latency 34.978 ms 2023-10-18T12:30:58.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 522897 13.43 MB/sec execute 87 sec latency 8.979 ms 2023-10-18T12:30:59.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:30:59 smithi070 ceph-mon[98639]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 273 op/s 2023-10-18T12:30:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:30:59 smithi152 ceph-mon[134767]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 273 op/s 2023-10-18T12:30:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:30:59 smithi186 ceph-mon[135253]: pgmap v463: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 273 op/s 2023-10-18T12:30:59.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 525993 13.49 MB/sec execute 88 sec latency 8.160 ms 2023-10-18T12:31:00.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 526796 13.36 MB/sec execute 89 sec latency 55.784 ms 2023-10-18T12:31:01.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:01 smithi070 ceph-mon[98639]: pgmap v464: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2023-10-18T12:31:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:01 smithi152 ceph-mon[134767]: pgmap v464: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2023-10-18T12:31:01.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:01 smithi186 ceph-mon[135253]: pgmap v464: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2023-10-18T12:31:01.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 528322 13.28 MB/sec execute 90 sec latency 182.378 ms 2023-10-18T12:31:02.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 530233 13.27 MB/sec execute 91 sec latency 33.346 ms 2023-10-18T12:31:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:03 smithi070 ceph-mon[98639]: pgmap v465: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2023-10-18T12:31:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:03 smithi152 ceph-mon[134767]: pgmap v465: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2023-10-18T12:31:03.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:03 smithi186 ceph-mon[135253]: pgmap v465: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2023-10-18T12:31:03.848 INFO:tasks.workunit.client.0.smithi070.stdout: 1 533310 13.32 MB/sec execute 92 sec latency 9.881 ms 2023-10-18T12:31:04.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 536244 13.33 MB/sec execute 93 sec latency 8.425 ms 2023-10-18T12:31:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:05 smithi070 ceph-mon[98639]: pgmap v466: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2023-10-18T12:31:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:05 smithi152 ceph-mon[134767]: pgmap v466: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2023-10-18T12:31:05.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:05 smithi186 ceph-mon[135253]: pgmap v466: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2023-10-18T12:31:05.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 538766 13.31 MB/sec execute 94 sec latency 8.657 ms 2023-10-18T12:31:06.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 540903 13.31 MB/sec execute 95 sec latency 19.288 ms 2023-10-18T12:31:07.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:07 smithi070 ceph-mon[98639]: pgmap v467: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2023-10-18T12:31:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:07 smithi152 ceph-mon[134767]: pgmap v467: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2023-10-18T12:31:07.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:07 smithi186 ceph-mon[135253]: pgmap v467: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2023-10-18T12:31:07.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 543335 13.32 MB/sec execute 96 sec latency 168.434 ms 2023-10-18T12:31:08.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 544973 13.28 MB/sec execute 97 sec latency 36.854 ms 2023-10-18T12:31:09.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:09 smithi070 ceph-mon[98639]: pgmap v468: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 290 op/s 2023-10-18T12:31:09.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:09 smithi152 ceph-mon[134767]: pgmap v468: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 290 op/s 2023-10-18T12:31:09.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:09 smithi186 ceph-mon[135253]: pgmap v468: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 290 op/s 2023-10-18T12:31:09.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 547643 13.29 MB/sec execute 98 sec latency 13.903 ms 2023-10-18T12:31:10.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 550701 13.33 MB/sec execute 99 sec latency 28.582 ms 2023-10-18T12:31:11.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:11 smithi070 ceph-mon[98639]: pgmap v469: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 292 op/s 2023-10-18T12:31:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:11 smithi152 ceph-mon[134767]: pgmap v469: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 292 op/s 2023-10-18T12:31:11.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:11 smithi186 ceph-mon[135253]: pgmap v469: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 292 op/s 2023-10-18T12:31:11.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 553056 13.28 MB/sec execute 100 sec latency 7.411 ms 2023-10-18T12:31:12.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 554754 13.27 MB/sec execute 101 sec latency 95.507 ms 2023-10-18T12:31:13.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:13 smithi070 ceph-mon[98639]: pgmap v470: 129 pgs: 129 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 261 op/s 2023-10-18T12:31:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:13 smithi152 ceph-mon[134767]: pgmap v470: 129 pgs: 129 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 261 op/s 2023-10-18T12:31:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:13 smithi186 ceph-mon[135253]: pgmap v470: 129 pgs: 129 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 261 op/s 2023-10-18T12:31:13.849 INFO:tasks.workunit.client.0.smithi070.stdout: 1 554754 13.14 MB/sec execute 102 sec latency 1095.597 ms 2023-10-18T12:31:14.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 554754 13.01 MB/sec execute 103 sec latency 2095.727 ms 2023-10-18T12:31:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:15 smithi070 ceph-mon[98639]: pgmap v471: 129 pgs: 129 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 287 op/s 2023-10-18T12:31:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:15 smithi152 ceph-mon[134767]: pgmap v471: 129 pgs: 129 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 287 op/s 2023-10-18T12:31:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:15 smithi186 ceph-mon[135253]: pgmap v471: 129 pgs: 129 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 287 op/s 2023-10-18T12:31:15.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 554754 12.89 MB/sec execute 104 sec latency 3095.818 ms 2023-10-18T12:31:16.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 554754 12.76 MB/sec execute 105 sec latency 4095.941 ms 2023-10-18T12:31:17.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:17 smithi070 ceph-mon[98639]: pgmap v472: 129 pgs: 129 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 214 op/s 2023-10-18T12:31:17.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:31:17.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:17 smithi152 ceph-mon[134767]: pgmap v472: 129 pgs: 129 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 214 op/s 2023-10-18T12:31:17.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:17 smithi186 ceph-mon[135253]: pgmap v472: 129 pgs: 129 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 214 op/s 2023-10-18T12:31:17.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 -- 172.21.15.70:0/1202710287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230106730 msgr2=0x7f8230106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 --2- 172.21.15.70:0/1202710287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230106730 0x7f8230106b10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8224009990 tx=0x7f8224009ca0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 -- 172.21.15.70:0/1202710287 shutdown_connections 2023-10-18T12:31:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 --2- 172.21.15.70:0/1202710287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82301010e0 0x7f823010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 --2- 172.21.15.70:0/1202710287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230106730 0x7f8230106b10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 --2- 172.21.15.70:0/1202710287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230100720 0x7f8230100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 -- 172.21.15.70:0/1202710287 >> 172.21.15.70:0/1202710287 conn(0x7f82300fc1b0 msgr2=0x7f82300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 -- 172.21.15.70:0/1202710287 shutdown_connections 2023-10-18T12:31:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.836+0000 7f8236dff700 1 -- 172.21.15.70:0/1202710287 wait complete. 2023-10-18T12:31:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.837+0000 7f8236dff700 1 Processor -- start 2023-10-18T12:31:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.837+0000 7f8236dff700 1 -- start start 2023-10-18T12:31:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8236dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8230100720 0x7f823019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8236dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 0x7f823019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8236dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230106730 0x7f8230198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8236dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230199200 con 0x7f82301010e0 2023-10-18T12:31:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8236dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230199370 con 0x7f8230106730 2023-10-18T12:31:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8236dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230199530 con 0x7f8230100720 2023-10-18T12:31:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 0x7f823019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 0x7f823019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35392/0 (socket says 172.21.15.70:35392) 2023-10-18T12:31:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 -- 172.21.15.70:0/2443598618 learned_addr learned my addr 172.21.15.70:0/2443598618 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8234b9b700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8230100720 0x7f823019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f823539c700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230106730 0x7f8230198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 -- 172.21.15.70:0/2443598618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8230100720 msgr2=0x7f823019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8230100720 0x7f823019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 -- 172.21.15.70:0/2443598618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230106730 msgr2=0x7f8230198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230106730 0x7f8230198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f822ffff700 1 -- 172.21.15.70:0/2443598618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240095f0 con 0x7f82301010e0 2023-10-18T12:31:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f8234b9b700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8230100720 0x7f823019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.838+0000 7f823539c700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230106730 0x7f8230198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:31:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.839+0000 7f822ffff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 0x7f823019efb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9a0 tx=0x7f821c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.839+0000 7f822dffb700 1 -- 172.21.15.70:0/2443598618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c009790 con 0x7f82301010e0 2023-10-18T12:31:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.839+0000 7f822dffb700 1 -- 172.21.15.70:0/2443598618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821c00cbd0 con 0x7f82301010e0 2023-10-18T12:31:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.839+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8230199810 con 0x7f82301010e0 2023-10-18T12:31:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.839+0000 7f822dffb700 1 -- 172.21.15.70:0/2443598618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c018680 con 0x7f82301010e0 2023-10-18T12:31:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.839+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82301a5a60 con 0x7f82301010e0 2023-10-18T12:31:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.842+0000 7f822dffb700 1 -- 172.21.15.70:0/2443598618 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f821c00cd40 con 0x7f82301010e0 2023-10-18T12:31:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 Processor -- start 2023-10-18T12:31:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 -- start start 2023-10-18T12:31:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f822dffb700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82200a15f0 0x7f82200a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 0x55ec6d224fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d225500 0x55ec6d21f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d1926e0 con 0x55ec6d224be0 2023-10-18T12:31:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d178c60 con 0x55ec6d21f5d0 2023-10-18T12:31:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.844+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d176580 con 0x55ec6d225500 2023-10-18T12:31:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822dffb700 1 -- 172.21.15.70:0/2443598618 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f821c014070 con 0x7f82301010e0 2023-10-18T12:31:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f8234b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 0x55ec6d224fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f823539c700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82200a15f0 0x7f82200a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d225500 0x55ec6d21f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f8234b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 0x55ec6d224fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35402/0 (socket says 172.21.15.70:35402) 2023-10-18T12:31:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f8234b9b700 1 -- 172.21.15.70:0/2081562190 learned_addr learned my addr 172.21.15.70:0/2081562190 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f823539c700 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 -- 172.21.15.70:0/2081562190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 msgr2=0x55ec6d21fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 -- 172.21.15.70:0/2081562190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 msgr2=0x55ec6d224fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 0x55ec6d224fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 -- 172.21.15.70:0/2081562190 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6d12f3f0 con 0x55ec6d225500 2023-10-18T12:31:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f8234b9b700 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 0x55ec6d224fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f823539c700 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f822ffff700 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d225500 0x55ec6d21f090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f821c0cab00 tx=0x7f821c099f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93db7192088eed69 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.845+0000 7f823539c700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82200a15f0 0x7f82200a3ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f82280060b0 tx=0x7f8228009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f821affd700 1 -- 172.21.15.70:0/2081562190 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c063940 con 0x55ec6d225500 2023-10-18T12:31:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f821affd700 1 -- 172.21.15.70:0/2081562190 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821c062dd0 con 0x55ec6d225500 2023-10-18T12:31:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f821affd700 1 -- 172.21.15.70:0/2081562190 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c0a2720 con 0x55ec6d225500 2023-10-18T12:31:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 -- 172.21.15.70:0/2081562190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d225500 msgr2=0x55ec6d21f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d225500 0x55ec6d21f090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f821c0cab00 tx=0x7f821c099f50 comp rx=0 tx=0).stop 2023-10-18T12:31:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 -- 172.21.15.70:0/2081562190 shutdown_connections 2023-10-18T12:31:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d224be0 0x55ec6d224fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/2081562190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d225500 0x55ec6d21f090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 -- 172.21.15.70:0/2081562190 >> 172.21.15.70:0/2081562190 conn(0x55ec6d193720 msgr2=0x55ec6d219b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.846+0000 7f824a6ebb80 1 -- 172.21.15.70:0/2081562190 shutdown_connections 2023-10-18T12:31:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.847+0000 7f824a6ebb80 1 -- 172.21.15.70:0/2081562190 wait complete. 2023-10-18T12:31:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.847+0000 7f824a6ebb80 1 -- start start 2023-10-18T12:31:17.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 554754 12.64 MB/sec execute 106 sec latency 5096.016 ms 2023-10-18T12:31:17.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.847+0000 7f824a6ebb80 1 Processor -- start 2023-10-18T12:31:17.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d22b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d228720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d228d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d171e70 con 0x55ec6d225500 2023-10-18T12:31:17.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d10e710 con 0x55ec6d224be0 2023-10-18T12:31:17.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d10e3b0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f824a6ebb80 10 client.0 _ll_register_callbacks cb 0x55ec6d058a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:17.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f823539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d228d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d22b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f822ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d228720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f822ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d228720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49808/0 (socket says 172.21.15.70:49808) 2023-10-18T12:31:17.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d22b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50054/0 (socket says 172.21.15.70:50054) 2023-10-18T12:31:17.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f822ffff700 1 -- 172.21.15.70:0/3452048362 learned_addr learned my addr 172.21.15.70:0/3452048362 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 -- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 msgr2=0x55ec6d228720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d228720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 -- 172.21.15.70:0/3452048362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 msgr2=0x55ec6d228d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d228d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.848+0000 7f8234b9b700 1 -- 172.21.15.70:0/3452048362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823010def0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f8234b9b700 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d22b3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8224015000 tx=0x7f8224005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb449f501f79a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f820ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:31:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f820ffff700 1 -- 172.21.15.70:0/3452048362 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822401d070 con 0x55ec6d21f5d0 2023-10-18T12:31:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec6d176ef0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f820ffff700 1 -- 172.21.15.70:0/3452048362 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8224004910 con 0x55ec6d21f5d0 2023-10-18T12:31:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f820ffff700 1 -- 172.21.15.70:0/3452048362 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82240056d0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f824a6ebb80 10 client.26290 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.849+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f82301a5a60 con 0x55ec6d21f5d0 2023-10-18T12:31:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f820ffff700 1 -- 172.21.15.70:0/3452048362 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec6d176ef0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f820ffff700 1 -- 172.21.15.70:0/3452048362 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f822400fbe0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f824a6ebb80 10 client.26290 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:17.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f824a6ebb80 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:17.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f824a6ebb80 20 client.26290 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:17.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d238ef0 0x55ec6d23b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f824a6ebb80 4 client.26290 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:17.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.850+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec6d1ee8b0 con 0x55ec6d238ef0 2023-10-18T12:31:17.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.851+0000 7f822ffff700 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d238ef0 0x55ec6d23b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.851+0000 7f820ffff700 10 client.26290 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:17.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.851+0000 7f822ffff700 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d238ef0 0x55ec6d23b310 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f340f9ab908eb9ba server_cookie=b842b404e56962a4 in_seq=0 out_seq=0 2023-10-18T12:31:17.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f820ffff700 1 -- 172.21.15.70:0/3452048362 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec6d1ee8b0 con 0x55ec6d238ef0 2023-10-18T12:31:17.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f820ffff700 10 client.26290 handle_command_reply: tid=0 2023-10-18T12:31:17.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 client.26290 shutdown 2023-10-18T12:31:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 msgr2=0x55ec6d22b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d22b3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8224015000 tx=0x7f8224005250 comp rx=0 tx=0).stop 2023-10-18T12:31:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 shutdown_connections 2023-10-18T12:31:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d238ef0 0x55ec6d23b310 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d228d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f820ffff700 0 client.26290 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d22b3e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3452048362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d228720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 >> 172.21.15.70:0/3452048362 conn(0x55ec6d193720 msgr2=0x55ec6d219ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 shutdown_connections 2023-10-18T12:31:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3452048362 wait complete. 2023-10-18T12:31:17.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.852+0000 7f824a6ebb80 20 client.26290 trim_cache size 0 max 16384 2023-10-18T12:31:17.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.869+0000 7f824a6ebb80 1 Processor -- start 2023-10-18T12:31:17.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.869+0000 7f824a6ebb80 1 -- start start 2023-10-18T12:31:17.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d234600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d234b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d21a130 con 0x55ec6d225500 2023-10-18T12:31:17.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6cff2920 con 0x55ec6d224be0 2023-10-18T12:31:17.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d220030 con 0x55ec6d21f5d0 2023-10-18T12:31:17.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f823539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d234b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f823539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d234b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35412/0 (socket says 172.21.15.70:35412) 2023-10-18T12:31:17.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f823539c700 1 -- 172.21.15.70:0/624136182 learned_addr learned my addr 172.21.15.70:0/624136182 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:17.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f8234b9b700 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f822ffff700 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d234600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f8234b9b700 1 -- 172.21.15.70:0/624136182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 msgr2=0x55ec6d234600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f8234b9b700 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d234600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f8234b9b700 1 -- 172.21.15.70:0/624136182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 msgr2=0x55ec6d234b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f8234b9b700 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d234b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f8234b9b700 1 -- 172.21.15.70:0/624136182 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6cf9cde0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.870+0000 7f823539c700 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d234b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.871+0000 7f8234b9b700 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21f090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f822400b6f0 tx=0x7f8224009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0e2a06f99d92c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.871+0000 7f821b7fe700 1 -- 172.21.15.70:0/624136182 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822401d070 con 0x55ec6d21f5d0 2023-10-18T12:31:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.871+0000 7f821b7fe700 1 -- 172.21.15.70:0/624136182 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822401e040 con 0x55ec6d21f5d0 2023-10-18T12:31:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.871+0000 7f821b7fe700 1 -- 172.21.15.70:0/624136182 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224017820 con 0x55ec6d21f5d0 2023-10-18T12:31:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 -- 172.21.15.70:0/624136182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 msgr2=0x55ec6d21f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21f090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f822400b6f0 tx=0x7f8224009290 comp rx=0 tx=0).stop 2023-10-18T12:31:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 -- 172.21.15.70:0/624136182 shutdown_connections 2023-10-18T12:31:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d234b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d21f5d0 0x55ec6d21f090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/624136182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d224be0 0x55ec6d234600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 -- 172.21.15.70:0/624136182 >> 172.21.15.70:0/624136182 conn(0x55ec6d193720 msgr2=0x55ec6d226ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:17.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 -- 172.21.15.70:0/624136182 shutdown_connections 2023-10-18T12:31:17.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 -- 172.21.15.70:0/624136182 wait complete. 2023-10-18T12:31:17.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 -- start start 2023-10-18T12:31:17.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.872+0000 7f824a6ebb80 1 Processor -- start 2023-10-18T12:31:17.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d221bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d224be0 0x55ec6d220630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d220b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d222250 con 0x55ec6d225500 2023-10-18T12:31:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d2223c0 con 0x55ec6d21f5d0 2023-10-18T12:31:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6d2210b0 con 0x55ec6d224be0 2023-10-18T12:31:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f824a6ebb80 10 client.0 _ll_register_callbacks cb 0x55ec6d058a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d220b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d220b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35414/0 (socket says 172.21.15.70:35414) 2023-10-18T12:31:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 -- 172.21.15.70:0/3279518431 learned_addr learned my addr 172.21.15.70:0/3279518431 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f8234b9b700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d221bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f822ffff700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d224be0 0x55ec6d220630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 -- 172.21.15.70:0/3279518431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d224be0 msgr2=0x55ec6d220630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d224be0 0x55ec6d220630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 -- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 msgr2=0x55ec6d221bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d221bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.873+0000 7f823539c700 1 -- 172.21.15.70:0/3279518431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8228013070 con 0x55ec6d225500 2023-10-18T12:31:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f822ffff700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d224be0 0x55ec6d220630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f823539c700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d220b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f822801cec0 tx=0x7f8228004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3522dea63441102 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f820ffff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:31:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f820ffff700 1 -- 172.21.15.70:0/3279518431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228021040 con 0x55ec6d225500 2023-10-18T12:31:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f820ffff700 1 -- 172.21.15.70:0/3279518431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8228026480 con 0x55ec6d225500 2023-10-18T12:31:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f821c062dd0 con 0x55ec6d225500 2023-10-18T12:31:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f820ffff700 1 -- 172.21.15.70:0/3279518431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8228026640 con 0x55ec6d225500 2023-10-18T12:31:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f824a6ebb80 10 client.16272 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f82301a5a60 con 0x55ec6d225500 2023-10-18T12:31:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.874+0000 7f820ffff700 1 -- 172.21.15.70:0/3279518431 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821c062dd0 con 0x55ec6d225500 2023-10-18T12:31:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f820ffff700 1 -- 172.21.15.70:0/3279518431 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f822802e070 con 0x55ec6d225500 2023-10-18T12:31:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f824a6ebb80 10 client.16272 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f824a6ebb80 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:17.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f824a6ebb80 20 client.16272 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:17.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d28e000 0x55ec6d290420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:17.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f824a6ebb80 4 client.16272 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:31:17.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec6d16ae50 con 0x55ec6d28e000 2023-10-18T12:31:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.875+0000 7f8234b9b700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d28e000 0x55ec6d290420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f820ffff700 10 client.16272 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f8234b9b700 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d28e000 0x55ec6d290420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25034092dcfc4b1c server_cookie=3c75dce94a1e1950 in_seq=0 out_seq=0 2023-10-18T12:31:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f820ffff700 1 -- 172.21.15.70:0/3279518431 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ec6d16ae50 con 0x55ec6d28e000 2023-10-18T12:31:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f820ffff700 10 client.16272 handle_command_reply: tid=0 2023-10-18T12:31:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 client.16272 shutdown 2023-10-18T12:31:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 msgr2=0x55ec6d220b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d220b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f822801cec0 tx=0x7f8228004ad0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 shutdown_connections 2023-10-18T12:31:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ec6d28e000 0x55ec6d290420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec6d225500 0x55ec6d220b70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec6d21f5d0 0x55ec6d221bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 --2- 172.21.15.70:0/3279518431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec6d224be0 0x55ec6d220630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f820ffff700 0 client.16272 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 >> 172.21.15.70:0/3279518431 conn(0x55ec6d193720 msgr2=0x55ec6d2134b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 shutdown_connections 2023-10-18T12:31:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 1 -- 172.21.15.70:0/3279518431 wait complete. 2023-10-18T12:31:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.876+0000 7f824a6ebb80 20 client.16272 trim_cache size 0 max 16384 2023-10-18T12:31:17.893 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:31:17.893 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:31:17.893 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:31:17.893 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:31:17.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82200a15f0 msgr2=0x7f82200a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82200a15f0 0x7f82200a3ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f82280060b0 tx=0x7f8228009920 comp rx=0 tx=0).stop 2023-10-18T12:31:17.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 msgr2=0x7f823019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 0x7f823019efb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9a0 tx=0x7f821c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 shutdown_connections 2023-10-18T12:31:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f82200a15f0 0x7f82200a3ab0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8230106730 0x7f8230198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8230100720 0x7f823019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 --2- 172.21.15.70:0/2443598618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82301010e0 0x7f823019efb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 >> 172.21.15.70:0/2443598618 conn(0x7f82300fc1b0 msgr2=0x7f823010be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 shutdown_connections 2023-10-18T12:31:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:17.877+0000 7f8236dff700 1 -- 172.21.15.70:0/2443598618 wait complete. 2023-10-18T12:31:17.898 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:31:17.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:31:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 -- 172.21.15.70:0/2071701636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c1006a0 msgr2=0x7f607c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 --2- 172.21.15.70:0/2071701636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c1006a0 0x7f607c100b20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f606c009990 tx=0x7f606c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 -- 172.21.15.70:0/2071701636 shutdown_connections 2023-10-18T12:31:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 --2- 172.21.15.70:0/2071701636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c101060 0x7f607c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 --2- 172.21.15.70:0/2071701636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f607c1066b0 0x7f607c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 --2- 172.21.15.70:0/2071701636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c1006a0 0x7f607c100b20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 -- 172.21.15.70:0/2071701636 >> 172.21.15.70:0/2071701636 conn(0x7f607c0fc1d0 msgr2=0x7f607c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 -- 172.21.15.70:0/2071701636 shutdown_connections 2023-10-18T12:31:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.093+0000 7f6083bff700 1 -- 172.21.15.70:0/2071701636 wait complete. 2023-10-18T12:31:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 Processor -- start 2023-10-18T12:31:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 -- start start 2023-10-18T12:31:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f607c1006a0 0x7f607c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 0x7f607c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c1066b0 0x7f607c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19cb60 con 0x7f607c101060 2023-10-18T12:31:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19ccd0 con 0x7f607c1066b0 2023-10-18T12:31:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.095+0000 7f6083bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19ce90 con 0x7f607c1006a0 2023-10-18T12:31:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 0x7f607c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f607c1006a0 0x7f607c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 0x7f607c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35436/0 (socket says 172.21.15.70:35436) 2023-10-18T12:31:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 -- 172.21.15.70:0/3973057152 learned_addr learned my addr 172.21.15.70:0/3973057152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608219c700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c1066b0 0x7f607c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 -- 172.21.15.70:0/3973057152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f607c1006a0 msgr2=0x7f607c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f607c1006a0 0x7f607c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 -- 172.21.15.70:0/3973057152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c1066b0 msgr2=0x7f607c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c1066b0 0x7f607c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 -- 172.21.15.70:0/3973057152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0095f0 con 0x7f607c101060 2023-10-18T12:31:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f608119a700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 0x7f607c199270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.096+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c01d070 con 0x7f607c101060 2023-10-18T12:31:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.097+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c022470 con 0x7f607c101060 2023-10-18T12:31:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.097+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c19d110 con 0x7f607c101060 2023-10-18T12:31:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.097+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c017650 con 0x7f607c101060 2023-10-18T12:31:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.097+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a5ad0 con 0x7f607c101060 2023-10-18T12:31:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.098+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c101de0 con 0x7f607c101060 2023-10-18T12:31:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.100+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f606c0225e0 con 0x7f607c101060 2023-10-18T12:31:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.101+0000 7f6072ffd700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60680a15f0 0x7f60680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.101+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f606c0ce740 con 0x7f607c101060 2023-10-18T12:31:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.101+0000 7f608199b700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60680a15f0 0x7f60680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.102+0000 7f608199b700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60680a15f0 0x7f60680a3ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6078009be0 tx=0x7f6078009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.105+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f606c097790 con 0x7f607c101060 2023-10-18T12:31:18.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.386+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f607c067530 con 0x7f607c101060 2023-10-18T12:31:18.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.387+0000 7f6072ffd700 1 -- 172.21.15.70:0/3973057152 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f606c09adb0 con 0x7f607c101060 2023-10-18T12:31:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:31:18.390 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:31:18.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.390+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60680a15f0 msgr2=0x7f60680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.390+0000 7f6083bff700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60680a15f0 0x7f60680a3ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6078009be0 tx=0x7f6078009260 comp rx=0 tx=0).stop 2023-10-18T12:31:18.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 msgr2=0x7f607c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 0x7f607c199270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c005740 comp rx=0 tx=0).stop 2023-10-18T12:31:18.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 shutdown_connections 2023-10-18T12:31:18.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f60680a15f0 0x7f60680a3ab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c1066b0 0x7f607c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f607c1006a0 0x7f607c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 --2- 172.21.15.70:0/3973057152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c101060 0x7f607c199270 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 >> 172.21.15.70:0/3973057152 conn(0x7f607c0fc1d0 msgr2=0x7f607c10be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 shutdown_connections 2023-10-18T12:31:18.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.391+0000 7f6083bff700 1 -- 172.21.15.70:0/3973057152 wait complete. 2023-10-18T12:31:18.396 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:31:18.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:31:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/357048525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e80ff640 msgr2=0x7ff0e80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/357048525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e80ff640 0x7ff0e80ffac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009990 tx=0x7ff0d8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/357048525 shutdown_connections 2023-10-18T12:31:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/357048525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e8100000 0x7ff0e810c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/357048525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e81056e0 0x7ff0e8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/357048525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e80ff640 0x7ff0e80ffac0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/357048525 >> 172.21.15.70:0/357048525 conn(0x7ff0e80767f0 msgr2=0x7ff0e8076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/357048525 shutdown_connections 2023-10-18T12:31:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.630+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/357048525 wait complete. 2023-10-18T12:31:18.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.631+0000 7ff0ef9ff700 1 Processor -- start 2023-10-18T12:31:18.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.631+0000 7ff0ef9ff700 1 -- start start 2023-10-18T12:31:18.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ef9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 0x7ff0e8101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ef9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e8100000 0x7ff0e81017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ef9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e81056e0 0x7ff0e8101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ef9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e81a4f80 con 0x7ff0e80ff640 2023-10-18T12:31:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ef9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e81a50f0 con 0x7ff0e8100000 2023-10-18T12:31:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ef9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e81a52b0 con 0x7ff0e81056e0 2023-10-18T12:31:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 0x7ff0e8101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 0x7ff0e8101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35444/0 (socket says 172.21.15.70:35444) 2023-10-18T12:31:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2610981830 learned_addr learned my addr 172.21.15.70:0/2610981830 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ecf9a700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e8100000 0x7ff0e81017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e81056e0 0x7ff0e8101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2610981830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e81056e0 msgr2=0x7ff0e8101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e81056e0 0x7ff0e8101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2610981830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e8100000 msgr2=0x7ff0e81017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e8100000 0x7ff0e81017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.632+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2610981830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d80095f0 con 0x7ff0e80ff640 2023-10-18T12:31:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.633+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 0x7ff0e8101290 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400e9a0 tx=0x7ff0e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.633+0000 7ff0de7fc700 1 -- 172.21.15.70:0/2610981830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4009790 con 0x7ff0e80ff640 2023-10-18T12:31:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.633+0000 7ff0de7fc700 1 -- 172.21.15.70:0/2610981830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e400cbd0 con 0x7ff0e80ff640 2023-10-18T12:31:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.633+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e81a5590 con 0x7ff0e80ff640 2023-10-18T12:31:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.633+0000 7ff0de7fc700 1 -- 172.21.15.70:0/2610981830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4018680 con 0x7ff0e80ff640 2023-10-18T12:31:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.633+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e81a5aa0 con 0x7ff0e80ff640 2023-10-18T12:31:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.636+0000 7ff0de7fc700 1 -- 172.21.15.70:0/2610981830 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff0e400cd40 con 0x7ff0e80ff640 2023-10-18T12:31:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.637+0000 7ff103375b80 1 Processor -- start 2023-10-18T12:31:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 -- start start 2023-10-18T12:31:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0de7fc700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff0d40a15f0 0x7ff0d40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0de7fc700 1 -- 172.21.15.70:0/2610981830 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff0e4014070 con 0x7ff0e80ff640 2023-10-18T12:31:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d5156e0 con 0x563c5d5a25d0 2023-10-18T12:31:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d4fbc60 con 0x563c5d5a8500 2023-10-18T12:31:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff103375b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d4f9580 con 0x563c5d5a7be0 2023-10-18T12:31:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff0d40a15f0 0x7ff0d40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0ed79b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0ed79b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50112/0 (socket says 172.21.15.70:50112) 2023-10-18T12:31:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2466079479 learned_addr learned my addr 172.21.15.70:0/2466079479 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0ecf9a700 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.638+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2466079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 msgr2=0x563c5d5a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2466079479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 msgr2=0x563c5d5a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ed79b700 1 -- 172.21.15.70:0/2466079479 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5d4b23f0 con 0x563c5d5a7be0 2023-10-18T12:31:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ecf9a700 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7fc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40a2160 tx=0x7ff0e4099c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be44e79fc274a49c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff0d40a15f0 0x7ff0d40a3ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0015d10 tx=0x7ff0e0015440 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0d37fe700 1 -- 172.21.15.70:0/2466079479 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e409e450 con 0x563c5d5a7be0 2023-10-18T12:31:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0d37fe700 1 -- 172.21.15.70:0/2466079479 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e40a2690 con 0x563c5d5a7be0 2023-10-18T12:31:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.639+0000 7ff0d37fe700 1 -- 172.21.15.70:0/2466079479 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e409a560 con 0x563c5d5a7be0 2023-10-18T12:31:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 -- 172.21.15.70:0/2466079479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 msgr2=0x563c5d5a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7fc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40a2160 tx=0x7ff0e4099c60 comp rx=0 tx=0).stop 2023-10-18T12:31:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 -- 172.21.15.70:0/2466079479 shutdown_connections 2023-10-18T12:31:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7fc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 --2- 172.21.15.70:0/2466079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 -- 172.21.15.70:0/2466079479 >> 172.21.15.70:0/2466079479 conn(0x563c5d516720 msgr2=0x563c5d59cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 -- 172.21.15.70:0/2466079479 shutdown_connections 2023-10-18T12:31:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.640+0000 7ff103375b80 1 -- 172.21.15.70:0/2466079479 wait complete. 2023-10-18T12:31:18.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 -- start start 2023-10-18T12:31:18.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 Processor -- start 2023-10-18T12:31:18.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a25d0 0x563c5d5ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 0x563c5d5ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d5abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d4f4e70 con 0x563c5d5a8500 2023-10-18T12:31:18.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d491710 con 0x563c5d5a7be0 2023-10-18T12:31:18.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d4913b0 con 0x563c5d5a25d0 2023-10-18T12:31:18.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.641+0000 7ff103375b80 10 client.0 _ll_register_callbacks cb 0x563c5d3dba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:18.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d5abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0ecf9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 0x563c5d5ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0ecf9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 0x563c5d5ab820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49884/0 (socket says 172.21.15.70:49884) 2023-10-18T12:31:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d5abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35468/0 (socket says 172.21.15.70:35468) 2023-10-18T12:31:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0ecf9a700 1 -- 172.21.15.70:0/3671939699 learned_addr learned my addr 172.21.15.70:0/3671939699 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a25d0 0x563c5d5ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 -- 172.21.15.70:0/3671939699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a25d0 msgr2=0x563c5d5ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a25d0 0x563c5d5ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 -- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 msgr2=0x563c5d5ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 0x563c5d5ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 -- 172.21.15.70:0/3671939699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e810cd90 con 0x563c5d5a8500 2023-10-18T12:31:18.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0ecf9a700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 0x563c5d5ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:18.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d5abf60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0015380 tx=0x7ff0e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a921e1173a20fa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.642+0000 7ff0d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:31:18.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/3671939699 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000e040 con 0x563c5d5a8500 2023-10-18T12:31:18.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/3671939699 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff0e0020040 con 0x563c5d5a8500 2023-10-18T12:31:18.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c5d4f9ef0 con 0x563c5d5a8500 2023-10-18T12:31:18.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/3671939699 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0e00093a0 con 0x563c5d5a8500 2023-10-18T12:31:18.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 10 client.16290 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:18.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7ff0e81a5aa0 con 0x563c5d5a8500 2023-10-18T12:31:18.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/3671939699 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c5d4f9ef0 con 0x563c5d5a8500 2023-10-18T12:31:18.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/3671939699 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff0e0024070 con 0x563c5d5a8500 2023-10-18T12:31:18.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 10 client.16290 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:18.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:18.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 20 client.16290 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:18.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d5bbfb0 0x563c5d5be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 4 client.16290 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:18.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.643+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c5d5718b0 con 0x563c5d5bbfb0 2023-10-18T12:31:18.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.644+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d5bbfb0 0x563c5d5be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.644+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d5bbfb0 0x563c5d5be3d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1401a566a05580b server_cookie=a7de69b95666f3db in_seq=0 out_seq=0 2023-10-18T12:31:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.644+0000 7ff0d0ff9700 10 client.16290 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.645+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/3671939699 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c5d5718b0 con 0x563c5d5bbfb0 2023-10-18T12:31:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.645+0000 7ff0d0ff9700 10 client.16290 handle_command_reply: tid=0 2023-10-18T12:31:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.645+0000 7ff103375b80 1 client.16290 shutdown 2023-10-18T12:31:18.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 msgr2=0x563c5d5abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d5abf60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0015380 tx=0x7ff0e000b040 comp rx=0 tx=0).stop 2023-10-18T12:31:18.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 shutdown_connections 2023-10-18T12:31:18.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d5bbfb0 0x563c5d5be3d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d5abf60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a25d0 0x563c5d5ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 --2- 172.21.15.70:0/3671939699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a7be0 0x563c5d5ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 >> 172.21.15.70:0/3671939699 conn(0x563c5d516720 msgr2=0x563c5d5a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff0d0ff9700 0 client.16290 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:18.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 shutdown_connections 2023-10-18T12:31:18.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 1 -- 172.21.15.70:0/3671939699 wait complete. 2023-10-18T12:31:18.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.646+0000 7ff103375b80 20 client.16290 trim_cache size 0 max 16384 2023-10-18T12:31:18.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 Processor -- start 2023-10-18T12:31:18.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 -- start start 2023-10-18T12:31:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d59d130 con 0x563c5d5a25d0 2023-10-18T12:31:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d375920 con 0x563c5d5a8500 2023-10-18T12:31:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.660+0000 7ff103375b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d5b8a10 con 0x563c5d5a7be0 2023-10-18T12:31:18.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35484/0 (socket says 172.21.15.70:35484) 2023-10-18T12:31:18.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 -- 172.21.15.70:0/164953299 learned_addr learned my addr 172.21.15.70:0/164953299 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:18.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ecf9a700 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 -- 172.21.15.70:0/164953299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 msgr2=0x563c5d5bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 -- 172.21.15.70:0/164953299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 msgr2=0x563c5d5aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 -- 172.21.15.70:0/164953299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5d31fde0 con 0x563c5d5a25d0 2023-10-18T12:31:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5b2eb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400e970 tx=0x7ff0e401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fd984198382d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.661+0000 7ff0d3fff700 1 -- 172.21.15.70:0/164953299 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40986a0 con 0x563c5d5a25d0 2023-10-18T12:31:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff0d3fff700 1 -- 172.21.15.70:0/164953299 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e409e120 con 0x563c5d5a25d0 2023-10-18T12:31:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff0d3fff700 1 -- 172.21.15.70:0/164953299 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4099a50 con 0x563c5d5a25d0 2023-10-18T12:31:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 -- 172.21.15.70:0/164953299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 msgr2=0x563c5d5b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5b2eb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400e970 tx=0x7ff0e401f000 comp rx=0 tx=0).stop 2023-10-18T12:31:18.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 -- 172.21.15.70:0/164953299 shutdown_connections 2023-10-18T12:31:18.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a8500 0x563c5d5aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a25d0 0x563c5d5b2eb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 --2- 172.21.15.70:0/164953299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 -- 172.21.15.70:0/164953299 >> 172.21.15.70:0/164953299 conn(0x563c5d516720 msgr2=0x563c5d5a1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 -- 172.21.15.70:0/164953299 shutdown_connections 2023-10-18T12:31:18.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.662+0000 7ff103375b80 1 -- 172.21.15.70:0/164953299 wait complete. 2023-10-18T12:31:18.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 -- start start 2023-10-18T12:31:18.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 Processor -- start 2023-10-18T12:31:18.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a25d0 0x563c5d5a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d60ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d60f320 con 0x563c5d5a8500 2023-10-18T12:31:18.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d60f490 con 0x563c5d5a25d0 2023-10-18T12:31:18.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.663+0000 7ff103375b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5d60f650 con 0x563c5d5a7be0 2023-10-18T12:31:18.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff103375b80 10 client.0 _ll_register_callbacks cb 0x563c5d3dba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:18.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d60ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0ecf9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0ecf9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50136/0 (socket says 172.21.15.70:50136) 2023-10-18T12:31:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d60ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35492/0 (socket says 172.21.15.70:35492) 2023-10-18T12:31:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0ecf9a700 1 -- 172.21.15.70:0/2323524555 learned_addr learned my addr 172.21.15.70:0/2323524555 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a25d0 0x563c5d5a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 -- 172.21.15.70:0/2323524555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 msgr2=0x563c5d5a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 -- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a25d0 msgr2=0x563c5d5a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:18.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a25d0 0x563c5d5a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 -- 172.21.15.70:0/2323524555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e81a5aa0 con 0x563c5d5a8500 2023-10-18T12:31:18.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0edf9c700 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d60ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00156d0 tx=0x7ff0e001a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb291290be84cb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:18.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.664+0000 7ff0d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:31:18.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/2323524555 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000e040 con 0x563c5d5a8500 2023-10-18T12:31:18.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/2323524555 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff0e0020040 con 0x563c5d5a8500 2023-10-18T12:31:18.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0e40a2690 con 0x563c5d5a8500 2023-10-18T12:31:18.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/2323524555 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0e0015e50 con 0x563c5d5a8500 2023-10-18T12:31:18.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 10 client.16296 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:18.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563c5d4b23f0 con 0x563c5d5a8500 2023-10-18T12:31:18.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/2323524555 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e40a2690 con 0x563c5d5a8500 2023-10-18T12:31:18.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/2323524555 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff0e00075f0 con 0x563c5d5a8500 2023-10-18T12:31:18.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:18.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:18.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 20 client.16296 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:18.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d610d40 0x563c5d613160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:18.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 4 client.16296 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:31:18.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.665+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c5d4ede50 con 0x563c5d610d40 2023-10-18T12:31:18.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.666+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d610d40 0x563c5d613160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:18.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.666+0000 7ff0d0ff9700 10 client.16296 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:18.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.666+0000 7ff0ed79b700 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d610d40 0x563c5d613160 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c7e037c186ef3f server_cookie=472428f39169c075 in_seq=0 out_seq=0 2023-10-18T12:31:18.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff0d0ff9700 1 -- 172.21.15.70:0/2323524555 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c5d4ede50 con 0x563c5d610d40 2023-10-18T12:31:18.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff0d0ff9700 10 client.16296 handle_command_reply: tid=0 2023-10-18T12:31:18.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 client.16296 shutdown 2023-10-18T12:31:18.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 msgr2=0x563c5d60ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d60ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00156d0 tx=0x7ff0e001a9a0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 shutdown_connections 2023-10-18T12:31:18.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563c5d610d40 0x563c5d613160 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563c5d5a8500 0x563c5d60ede0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c5d5a25d0 0x563c5d5a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff0d0ff9700 0 client.16296 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:18.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 --2- 172.21.15.70:0/2323524555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c5d5a7be0 0x563c5d5a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.667+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 >> 172.21.15.70:0/2323524555 conn(0x563c5d516720 msgr2=0x563c5d5a56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 shutdown_connections 2023-10-18T12:31:18.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff103375b80 1 -- 172.21.15.70:0/2323524555 wait complete. 2023-10-18T12:31:18.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff103375b80 20 client.16296 trim_cache size 0 max 16384 2023-10-18T12:31:18.688 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:31:18.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff0d40a15f0 msgr2=0x7ff0d40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff0d40a15f0 0x7ff0d40a3ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0015d10 tx=0x7ff0e0015440 comp rx=0 tx=0).stop 2023-10-18T12:31:18.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 msgr2=0x7ff0e8101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:18.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 0x7ff0e8101290 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400e9a0 tx=0x7ff0e400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 shutdown_connections 2023-10-18T12:31:18.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0e81056e0 0x7ff0e8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0e80ff640 0x7ff0e8101290 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff0d40a15f0 0x7ff0d40a3ab0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 --2- 172.21.15.70:0/2610981830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0e8100000 0x7ff0e81017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:18.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.668+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 >> 172.21.15.70:0/2610981830 conn(0x7ff0e80767f0 msgr2=0x7ff0e8067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:18.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.669+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 shutdown_connections 2023-10-18T12:31:18.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:18.669+0000 7ff0ef9ff700 1 -- 172.21.15.70:0/2610981830 wait complete. 2023-10-18T12:31:18.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 555864 12.58 MB/sec execute 107 sec latency 5517.118 ms 2023-10-18T12:31:19.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:19 smithi070 ceph-mon[98639]: pgmap v473: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 212 op/s 2023-10-18T12:31:19.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:19 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3973057152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:31:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:19 smithi152 ceph-mon[134767]: pgmap v473: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 212 op/s 2023-10-18T12:31:19.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:19 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3973057152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:31:19.693 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:31:19.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:31:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:19 smithi186 ceph-mon[135253]: pgmap v473: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 212 op/s 2023-10-18T12:31:19.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:19 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3973057152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:31:19.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 558163 12.59 MB/sec execute 108 sec latency 53.278 ms 2023-10-18T12:31:19.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 -- 172.21.15.70:0/3464035706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886010cfd0 msgr2=0x7f886010f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 --2- 172.21.15.70:0/3464035706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886010cfd0 0x7f886010f400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8858009f90 tx=0x7f885800b330 comp rx=0 tx=0).stop 2023-10-18T12:31:19.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 -- 172.21.15.70:0/3464035706 shutdown_connections 2023-10-18T12:31:19.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 --2- 172.21.15.70:0/3464035706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f886010cfd0 0x7f886010f400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 --2- 172.21.15.70:0/3464035706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8860072bc0 0x7f8860072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 --2- 172.21.15.70:0/3464035706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88600734e0 0x7f886010ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 -- 172.21.15.70:0/3464035706 >> 172.21.15.70:0/3464035706 conn(0x7f88600fc200 msgr2=0x7f88600fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:19.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.901+0000 7f8867dff700 1 -- 172.21.15.70:0/3464035706 shutdown_connections 2023-10-18T12:31:19.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.902+0000 7f8867dff700 1 -- 172.21.15.70:0/3464035706 wait complete. 2023-10-18T12:31:19.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 Processor -- start 2023-10-18T12:31:19.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 -- start start 2023-10-18T12:31:19.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 0x7f8860198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88600734e0 0x7f88601994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f886010cfd0 0x7f886019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886019de40 con 0x7f886010cfd0 2023-10-18T12:31:19.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886019dfb0 con 0x7f8860072bc0 2023-10-18T12:31:19.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8867dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886019e170 con 0x7f88600734e0 2023-10-18T12:31:19.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 0x7f8860198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f886639c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f886010cfd0 0x7f886019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 0x7f8860198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49930/0 (socket says 172.21.15.70:49930) 2023-10-18T12:31:19.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.903+0000 7f8865b9b700 1 -- 172.21.15.70:0/3269100632 learned_addr learned my addr 172.21.15.70:0/3269100632 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:19.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f886539a700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88600734e0 0x7f88601994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8865b9b700 1 -- 172.21.15.70:0/3269100632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88600734e0 msgr2=0x7f88601994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8865b9b700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88600734e0 0x7f88601994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8865b9b700 1 -- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f886010cfd0 msgr2=0x7f886019d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8865b9b700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f886010cfd0 0x7f886019d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8865b9b700 1 -- 172.21.15.70:0/3269100632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0095f0 con 0x7f8860072bc0 2023-10-18T12:31:19.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f886539a700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88600734e0 0x7f88601994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:19.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f886639c700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f886010cfd0 0x7f886019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:19.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8865b9b700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 0x7f8860198f70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ec10 tx=0x7f885c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:19.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8856ffd700 1 -- 172.21.15.70:0/3269100632 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00cd40 con 0x7f8860072bc0 2023-10-18T12:31:19.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8850009520 con 0x7f8860072bc0 2023-10-18T12:31:19.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8856ffd700 1 -- 172.21.15.70:0/3269100632 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f885c0044e0 con 0x7f8860072bc0 2023-10-18T12:31:19.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8856ffd700 1 -- 172.21.15.70:0/3269100632 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c010690 con 0x7f8860072bc0 2023-10-18T12:31:19.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.904+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8858009c40 con 0x7f8860072bc0 2023-10-18T12:31:19.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.907+0000 7f8856ffd700 1 -- 172.21.15.70:0/3269100632 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f885c003710 con 0x7f8860072bc0 2023-10-18T12:31:19.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 Processor -- start 2023-10-18T12:31:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 -- start start 2023-10-18T12:31:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f8856ffd700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f884c0a1670 0x7f884c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f8856ffd700 1 -- 172.21.15.70:0/3269100632 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f885c014070 con 0x7f8860072bc0 2023-10-18T12:31:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 0x55cbaf432040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf42c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf42cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf39f760 con 0x55cbaf42c650 2023-10-18T12:31:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf385ce0 con 0x55cbaf431c60 2023-10-18T12:31:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f886639c700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f884c0a1670 0x7f884c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f887b723b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf383600 con 0x55cbaf432580 2023-10-18T12:31:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.909+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 0x55cbaf432040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f886539a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf42c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 0x55cbaf432040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49936/0 (socket says 172.21.15.70:49936) 2023-10-18T12:31:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 -- 172.21.15.70:0/808688306 learned_addr learned my addr 172.21.15.70:0/808688306 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f886639c700 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf42cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 -- 172.21.15.70:0/808688306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 msgr2=0x55cbaf42c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf42c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 -- 172.21.15.70:0/808688306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 msgr2=0x55cbaf42cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf42cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 -- 172.21.15.70:0/808688306 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbaf33c470 con 0x55cbaf431c60 2023-10-18T12:31:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f886539a700 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf42c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f886639c700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f884c0a1670 0x7f884c0a3b30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8858009790 tx=0x7f8858003a80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f8865b9b700 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 0x55cbaf432040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f885c0a2460 tx=0x7f885c09a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e66304c347fdd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f884bfff700 1 -- 172.21.15.70:0/808688306 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0850c0 con 0x55cbaf431c60 2023-10-18T12:31:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f884bfff700 1 -- 172.21.15.70:0/808688306 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f885c09a600 con 0x55cbaf431c60 2023-10-18T12:31:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.910+0000 7f884bfff700 1 -- 172.21.15.70:0/808688306 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c083be0 con 0x55cbaf431c60 2023-10-18T12:31:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 -- 172.21.15.70:0/808688306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 msgr2=0x55cbaf432040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 0x55cbaf432040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f885c0a2460 tx=0x7f885c09a1c0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 -- 172.21.15.70:0/808688306 shutdown_connections 2023-10-18T12:31:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf42cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf431c60 0x55cbaf432040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 --2- 172.21.15.70:0/808688306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf42c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 -- 172.21.15.70:0/808688306 >> 172.21.15.70:0/808688306 conn(0x55cbaf3a07a0 msgr2=0x55cbaf426be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 -- 172.21.15.70:0/808688306 shutdown_connections 2023-10-18T12:31:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.911+0000 7f887b723b80 1 -- 172.21.15.70:0/808688306 wait complete. 2023-10-18T12:31:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 -- start start 2023-10-18T12:31:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 Processor -- start 2023-10-18T12:31:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf435850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf435f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf37eef0 con 0x55cbaf431c60 2023-10-18T12:31:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf31b790 con 0x55cbaf42c650 2023-10-18T12:31:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.912+0000 7f887b723b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf31b430 con 0x55cbaf432580 2023-10-18T12:31:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f887b723b80 10 client.0 _ll_register_callbacks cb 0x55cbaf265b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f886539a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf435850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4385d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49942/0 (socket says 172.21.15.70:49942) 2023-10-18T12:31:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 -- 172.21.15.70:0/2116938467 learned_addr learned my addr 172.21.15.70:0/2116938467 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f886639c700 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf435f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 -- 172.21.15.70:0/2116938467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 msgr2=0x55cbaf435f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf435f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 -- 172.21.15.70:0/2116938467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 msgr2=0x55cbaf435850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf435850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 -- 172.21.15.70:0/2116938467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8850009a60 con 0x55cbaf42c650 2023-10-18T12:31:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f8865b9b700 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4385d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f885c09a590 tx=0x7f885c09e840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5912b8e7d4098d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f88497fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.913+0000 7f88497fa700 1 -- 172.21.15.70:0/2116938467 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c083de0 con 0x55cbaf42c650 2023-10-18T12:31:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f88497fa700 1 -- 172.21.15.70:0/2116938467 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f885c085050 con 0x55cbaf42c650 2023-10-18T12:31:19.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbaf383f70 con 0x55cbaf42c650 2023-10-18T12:31:19.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f88497fa700 1 -- 172.21.15.70:0/2116938467 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f885c09a860 con 0x55cbaf42c650 2023-10-18T12:31:19.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 10 client.26330 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:19.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8858009c40 con 0x55cbaf42c650 2023-10-18T12:31:19.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f88497fa700 1 -- 172.21.15.70:0/2116938467 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbaf383f70 con 0x55cbaf42c650 2023-10-18T12:31:19.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f88497fa700 1 -- 172.21.15.70:0/2116938467 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f885c099af0 con 0x55cbaf42c650 2023-10-18T12:31:19.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 10 client.26330 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:19.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:19.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 20 client.26330 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:19.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf445b40 0x55cbaf447f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 4 client.26330 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:19.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.914+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbaf3fb930 con 0x55cbaf445b40 2023-10-18T12:31:19.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.915+0000 7f886539a700 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf445b40 0x55cbaf447f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.915+0000 7f886539a700 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf445b40 0x55cbaf447f60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae187194d569b4a server_cookie=266b5e948c6fff22 in_seq=0 out_seq=0 2023-10-18T12:31:19.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.915+0000 7f88497fa700 10 client.26330 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:19.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.918+0000 7f88497fa700 1 -- 172.21.15.70:0/2116938467 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cbaf3fb930 con 0x55cbaf445b40 2023-10-18T12:31:19.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.918+0000 7f88497fa700 10 client.26330 handle_command_reply: tid=0 2023-10-18T12:31:19.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.918+0000 7f887b723b80 1 client.26330 shutdown 2023-10-18T12:31:19.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 msgr2=0x55cbaf4385d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4385d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f885c09a590 tx=0x7f885c09e840 comp rx=0 tx=0).stop 2023-10-18T12:31:19.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 shutdown_connections 2023-10-18T12:31:19.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf445b40 0x55cbaf447f60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf435f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4385d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 --2- 172.21.15.70:0/2116938467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf435850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f88497fa700 0 client.26330 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:19.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 >> 172.21.15.70:0/2116938467 conn(0x55cbaf3a07a0 msgr2=0x55cbaf42b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:19.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 shutdown_connections 2023-10-18T12:31:19.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 1 -- 172.21.15.70:0/2116938467 wait complete. 2023-10-18T12:31:19.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.919+0000 7f887b723b80 20 client.26330 trim_cache size 0 max 16384 2023-10-18T12:31:19.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.936+0000 7f887b723b80 1 Processor -- start 2023-10-18T12:31:19.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.936+0000 7f887b723b80 1 -- start start 2023-10-18T12:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f887b723b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf435810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f887b723b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf431c60 0x55cbaf43cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f887b723b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf432580 0x55cbaf430a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f887b723b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf1ff9a0 con 0x55cbaf42c650 2023-10-18T12:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f887b723b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf436590 con 0x55cbaf432580 2023-10-18T12:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f887b723b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf41f050 con 0x55cbaf431c60 2023-10-18T12:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf435810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886539a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf431c60 0x55cbaf43cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886639c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf432580 0x55cbaf430a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf435810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35542/0 (socket says 172.21.15.70:35542) 2023-10-18T12:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f8865b9b700 1 -- 172.21.15.70:0/2374627068 learned_addr learned my addr 172.21.15.70:0/2374627068 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886539a700 1 -- 172.21.15.70:0/2374627068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf432580 msgr2=0x55cbaf430a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886539a700 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf432580 0x55cbaf430a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886539a700 1 -- 172.21.15.70:0/2374627068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 msgr2=0x55cbaf435810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886539a700 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf435810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.937+0000 7f886539a700 1 -- 172.21.15.70:0/2374627068 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbaf426df0 con 0x55cbaf431c60 2023-10-18T12:31:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f886539a700 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf431c60 0x55cbaf43cf30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55cbaf436e90 tx=0x7f8850009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1667266542a9a5a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f8854bff700 1 -- 172.21.15.70:0/2374627068 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850016040 con 0x55cbaf431c60 2023-10-18T12:31:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f8854bff700 1 -- 172.21.15.70:0/2374627068 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8850017040 con 0x55cbaf431c60 2023-10-18T12:31:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f8854bff700 1 -- 172.21.15.70:0/2374627068 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500038e0 con 0x55cbaf431c60 2023-10-18T12:31:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 -- 172.21.15.70:0/2374627068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf431c60 msgr2=0x55cbaf43cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf431c60 0x55cbaf43cf30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55cbaf436e90 tx=0x7f8850009260 comp rx=0 tx=0).stop 2023-10-18T12:31:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 -- 172.21.15.70:0/2374627068 shutdown_connections 2023-10-18T12:31:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf432580 0x55cbaf430a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf42c650 0x55cbaf435810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 --2- 172.21.15.70:0/2374627068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf431c60 0x55cbaf43cf30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.938+0000 7f887b723b80 1 -- 172.21.15.70:0/2374627068 >> 172.21.15.70:0/2374627068 conn(0x55cbaf3a07a0 msgr2=0x55cbaf435100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.939+0000 7f887b723b80 1 -- 172.21.15.70:0/2374627068 shutdown_connections 2023-10-18T12:31:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.939+0000 7f887b723b80 1 -- 172.21.15.70:0/2374627068 wait complete. 2023-10-18T12:31:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.939+0000 7f887b723b80 1 -- start start 2023-10-18T12:31:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.939+0000 7f887b723b80 1 Processor -- start 2023-10-18T12:31:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.939+0000 7f887b723b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f887b723b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf4936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f887b723b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf43c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f887b723b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf43ca10 con 0x55cbaf431c60 2023-10-18T12:31:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f887b723b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf43cb80 con 0x55cbaf42c650 2023-10-18T12:31:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f887b723b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaf43cd40 con 0x55cbaf432580 2023-10-18T12:31:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f887b723b80 10 client.0 _ll_register_callbacks cb 0x55cbaf265b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4931a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49962/0 (socket says 172.21.15.70:49962) 2023-10-18T12:31:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 -- 172.21.15.70:0/2739233821 learned_addr learned my addr 172.21.15.70:0/2739233821 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f886539a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf4936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f886639c700 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf43c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 -- 172.21.15.70:0/2739233821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 msgr2=0x55cbaf43c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf43c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 -- 172.21.15.70:0/2739233821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 msgr2=0x55cbaf4936e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf4936e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 -- 172.21.15.70:0/2739233821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885001fa60 con 0x55cbaf42c650 2023-10-18T12:31:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f8865b9b700 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4931a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f885c097070 tx=0x7f885c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7929e94f78fae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.940+0000 7f88497fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f88497fa700 1 -- 172.21.15.70:0/2739233821 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c083de0 con 0x55cbaf42c650 2023-10-18T12:31:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f88497fa700 1 -- 172.21.15.70:0/2739233821 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f885c0a2990 con 0x55cbaf42c650 2023-10-18T12:31:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f885c09a600 con 0x55cbaf42c650 2023-10-18T12:31:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f88497fa700 1 -- 172.21.15.70:0/2739233821 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f885c0a27b0 con 0x55cbaf42c650 2023-10-18T12:31:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 10 client.26336 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cbaf1a9e60 con 0x55cbaf42c650 2023-10-18T12:31:19.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f88497fa700 1 -- 172.21.15.70:0/2739233821 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f885c09a600 con 0x55cbaf42c650 2023-10-18T12:31:19.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f88497fa700 1 -- 172.21.15.70:0/2739233821 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f885c0a2d00 con 0x55cbaf42c650 2023-10-18T12:31:19.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:19.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 10 client.26336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:19.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 20 client.26336 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf49bca0 0x55cbaf49e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 4 client.26336 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:31:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.941+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cbaf377ed0 con 0x55cbaf49bca0 2023-10-18T12:31:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.942+0000 7f886539a700 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf49bca0 0x55cbaf49e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.942+0000 7f88497fa700 10 client.26336 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:19.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.942+0000 7f886539a700 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf49bca0 0x55cbaf49e090 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad2e6c48d0fddf server_cookie=fe75c704d2e6ed98 in_seq=0 out_seq=0 2023-10-18T12:31:19.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.958+0000 7f88497fa700 1 -- 172.21.15.70:0/2739233821 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cbaf377ed0 con 0x55cbaf49bca0 2023-10-18T12:31:19.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.958+0000 7f88497fa700 10 client.26336 handle_command_reply: tid=0 2023-10-18T12:31:19.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.958+0000 7f887b723b80 1 client.26336 shutdown 2023-10-18T12:31:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 msgr2=0x55cbaf4931a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4931a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f885c097070 tx=0x7f885c000bc0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 shutdown_connections 2023-10-18T12:31:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cbaf49bca0 0x55cbaf49e090 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbaf432580 0x55cbaf43c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cbaf42c650 0x55cbaf4931a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 --2- 172.21.15.70:0/2739233821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cbaf431c60 0x55cbaf4936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f88497fa700 0 client.26336 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:19.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 >> 172.21.15.70:0/2739233821 conn(0x55cbaf3a07a0 msgr2=0x55cbaf420530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:19.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 shutdown_connections 2023-10-18T12:31:19.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 1 -- 172.21.15.70:0/2739233821 wait complete. 2023-10-18T12:31:19.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.959+0000 7f887b723b80 20 client.26336 trim_cache size 0 max 16384 2023-10-18T12:31:19.965 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:31:19.965 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:31:19.965 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "877dd2c8-8109-4fb7-93df-1bc36459c0d5", 2023-10-18T12:31:19.966 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:31:19.966 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:31:19.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f884c0a1670 msgr2=0x7f884c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f884c0a1670 0x7f884c0a3b30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8858009790 tx=0x7f8858003a80 comp rx=0 tx=0).stop 2023-10-18T12:31:19.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 msgr2=0x7f8860198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:19.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 0x7f8860198f70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ec10 tx=0x7f885c00ef20 comp rx=0 tx=0).stop 2023-10-18T12:31:19.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 shutdown_connections 2023-10-18T12:31:19.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f886010cfd0 0x7f886019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f884c0a1670 0x7f884c0a3b30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8860072bc0 0x7f8860198f70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 --2- 172.21.15.70:0/3269100632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88600734e0 0x7f88601994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:19.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 >> 172.21.15.70:0/3269100632 conn(0x7f88600fc200 msgr2=0x7f88600fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 shutdown_connections 2023-10-18T12:31:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:19.960+0000 7f8867dff700 1 -- 172.21.15.70:0/3269100632 wait complete. 2023-10-18T12:31:19.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:31:20.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 -- 172.21.15.70:0/3507167728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c1115d0 msgr2=0x7fa45c1139c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 --2- 172.21.15.70:0/3507167728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c1115d0 0x7fa45c1139c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b0a0 tx=0x7fa45400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 -- 172.21.15.70:0/3507167728 shutdown_connections 2023-10-18T12:31:20.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 --2- 172.21.15.70:0/3507167728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c1115d0 0x7fa45c1139c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 --2- 172.21.15.70:0/3507167728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 0x7fa45c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 --2- 172.21.15.70:0/3507167728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c0734b0 0x7fa45c111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 -- 172.21.15.70:0/3507167728 >> 172.21.15.70:0/3507167728 conn(0x7fa45c0fc0f0 msgr2=0x7fa45c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.209+0000 7fa45bfff700 1 -- 172.21.15.70:0/3507167728 shutdown_connections 2023-10-18T12:31:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.210+0000 7fa45bfff700 1 -- 172.21.15.70:0/3507167728 wait complete. 2023-10-18T12:31:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.210+0000 7fa45bfff700 1 Processor -- start 2023-10-18T12:31:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 -- start start 2023-10-18T12:31:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 0x7fa45c19d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 0x7fa45c19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c1115d0 0x7fa45c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1a2120 con 0x7fa45c0734b0 2023-10-18T12:31:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1a2290 con 0x7fa45c072b90 2023-10-18T12:31:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1a2450 con 0x7fa45c1115d0 2023-10-18T12:31:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 0x7fa45c19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 0x7fa45c19d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35556/0 (socket says 172.21.15.70:35556) 2023-10-18T12:31:20.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 0x7fa45c19d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2281959253 learned_addr learned my addr 172.21.15.70:0/2281959253 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:20.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.211+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c1115d0 0x7fa45c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2281959253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c1115d0 msgr2=0x7fa45c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c1115d0 0x7fa45c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2281959253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 msgr2=0x7fa45c19d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 0x7fa45c19d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2281959253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c009520 con 0x7fa45c0734b0 2023-10-18T12:31:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45affd700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 0x7fa45c19d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c1115d0 0x7fa45c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 0x7fa45c19d790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e9a0 tx=0x7fa45000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa4609fd700 1 -- 172.21.15.70:0/2281959253 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450009790 con 0x7fa45c0734b0 2023-10-18T12:31:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa454009d00 con 0x7fa45c0734b0 2023-10-18T12:31:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa4609fd700 1 -- 172.21.15.70:0/2281959253 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa450010430 con 0x7fa45c0734b0 2023-10-18T12:31:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa4609fd700 1 -- 172.21.15.70:0/2281959253 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450018680 con 0x7fa45c0734b0 2023-10-18T12:31:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.212+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c1a9d90 con 0x7fa45c0734b0 2023-10-18T12:31:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.215+0000 7fa4609fd700 1 -- 172.21.15.70:0/2281959253 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa4500105a0 con 0x7fa45c0734b0 2023-10-18T12:31:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 Processor -- start 2023-10-18T12:31:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 -- start start 2023-10-18T12:31:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa4609fd700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4440a15f0 0x7fa4440a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa4609fd700 1 -- 172.21.15.70:0/2281959253 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa450014070 con 0x7fa45c0734b0 2023-10-18T12:31:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e7680 0x5646715e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e1820 0x5646715e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646715546e0 con 0x5646715e1820 2023-10-18T12:31:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467153ac60 con 0x5646715e7680 2023-10-18T12:31:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa475308b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564671538580 con 0x5646715e6d60 2023-10-18T12:31:20.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4440a15f0 0x7fa4440a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.217+0000 7fa45affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715e7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:50214/0 (socket says 172.21.15.70:50214) 2023-10-18T12:31:20.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45affd700 1 -- 172.21.15.70:0/2687396334 learned_addr learned my addr 172.21.15.70:0/2687396334 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e1820 0x5646715e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e7680 0x5646715e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2687396334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 msgr2=0x5646715e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2687396334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e1820 msgr2=0x5646715e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e1820 0x5646715e1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 -- 172.21.15.70:0/2687396334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646714f13f0 con 0x5646715e7680 2023-10-18T12:31:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45affd700 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e7680 0x5646715e12e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4500ca9b0 tx=0x7fa450099c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f158376be840c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4440a15f0 0x7fa4440a3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b780 tx=0x7fa454009380 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa441ffb700 1 -- 172.21.15.70:0/2687396334 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45009e390 con 0x5646715e7680 2023-10-18T12:31:20.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa441ffb700 1 -- 172.21.15.70:0/2687396334 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4500a25d0 con 0x5646715e7680 2023-10-18T12:31:20.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.218+0000 7fa441ffb700 1 -- 172.21.15.70:0/2687396334 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45009a6a0 con 0x5646715e7680 2023-10-18T12:31:20.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 -- 172.21.15.70:0/2687396334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e7680 msgr2=0x5646715e12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e7680 0x5646715e12e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4500ca9b0 tx=0x7fa450099c20 comp rx=0 tx=0).stop 2023-10-18T12:31:20.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 -- 172.21.15.70:0/2687396334 shutdown_connections 2023-10-18T12:31:20.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e1820 0x5646715e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 --2- 172.21.15.70:0/2687396334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e7680 0x5646715e12e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 -- 172.21.15.70:0/2687396334 >> 172.21.15.70:0/2687396334 conn(0x564671555720 msgr2=0x5646715dbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:20.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 -- 172.21.15.70:0/2687396334 shutdown_connections 2023-10-18T12:31:20.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.219+0000 7fa475308b80 1 -- 172.21.15.70:0/2687396334 wait complete. 2023-10-18T12:31:20.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.220+0000 7fa475308b80 1 -- start start 2023-10-18T12:31:20.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.220+0000 7fa475308b80 1 Processor -- start 2023-10-18T12:31:20.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e1820 0x5646715eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 0x5646715eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564671533e70 con 0x5646715e7680 2023-10-18T12:31:20.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646714d0710 con 0x5646715e6d60 2023-10-18T12:31:20.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646714d03b0 con 0x5646715e1820 2023-10-18T12:31:20.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa475308b80 10 client.0 _ll_register_callbacks cb 0x56467141aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:20.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 0x5646715eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 0x5646715eafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50004/0 (socket says 172.21.15.70:50004) 2023-10-18T12:31:20.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 -- 172.21.15.70:0/562142303 learned_addr learned my addr 172.21.15.70:0/562142303 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:20.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45affd700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e1820 0x5646715eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 -- 172.21.15.70:0/562142303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e1820 msgr2=0x5646715eaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e1820 0x5646715eaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 -- 172.21.15.70:0/562142303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 msgr2=0x5646715eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45a7fc700 1 -- 172.21.15.70:0/562142303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c011440 con 0x5646715e6d60 2023-10-18T12:31:20.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.221+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:20.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 0x5646715eafa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e970 tx=0x7fa45000c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e87e77c1ee0bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:20.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa43affd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:20.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa43affd700 1 -- 172.21.15.70:0/562142303 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500b84c0 con 0x5646715e6d60 2023-10-18T12:31:20.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564671538ef0 con 0x5646715e6d60 2023-10-18T12:31:20.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa43affd700 1 -- 172.21.15.70:0/562142303 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa450099730 con 0x5646715e6d60 2023-10-18T12:31:20.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa43affd700 1 -- 172.21.15.70:0/562142303 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4500998f0 con 0x5646715e6d60 2023-10-18T12:31:20.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa475308b80 10 client.26348 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:20.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa45c1a9d90 con 0x5646715e6d60 2023-10-18T12:31:20.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa43affd700 1 -- 172.21.15.70:0/562142303 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564671538ef0 con 0x5646715e6d60 2023-10-18T12:31:20.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa43affd700 1 -- 172.21.15.70:0/562142303 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa4500991a0 con 0x5646715e6d60 2023-10-18T12:31:20.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa475308b80 10 client.26348 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:20.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.222+0000 7fa475308b80 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:20.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa475308b80 20 client.26348 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:20.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa475308b80 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646715fac10 0x5646715fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa475308b80 4 client.26348 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646715b08b0 con 0x5646715fac10 2023-10-18T12:31:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa45affd700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646715fac10 0x5646715fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa45affd700 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646715fac10 0x5646715fd000 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9730e305b850185d server_cookie=cc57bf297832e097 in_seq=0 out_seq=0 2023-10-18T12:31:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.223+0000 7fa43affd700 10 client.26348 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa43affd700 1 -- 172.21.15.70:0/562142303 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5646715b08b0 con 0x5646715fac10 2023-10-18T12:31:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa43affd700 10 client.26348 handle_command_reply: tid=0 2023-10-18T12:31:20.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 client.26348 shutdown 2023-10-18T12:31:20.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 msgr2=0x5646715eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 0x5646715eafa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e970 tx=0x7fa45000c480 comp rx=0 tx=0).stop 2023-10-18T12:31:20.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 shutdown_connections 2023-10-18T12:31:20.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646715fac10 0x5646715fd000 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e1820 0x5646715eaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa43affd700 0 client.26348 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:20.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 --2- 172.21.15.70:0/562142303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e6d60 0x5646715eafa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.224+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 >> 172.21.15.70:0/562142303 conn(0x564671555720 msgr2=0x5646715e02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:20.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.225+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 shutdown_connections 2023-10-18T12:31:20.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.225+0000 7fa475308b80 1 -- 172.21.15.70:0/562142303 wait complete. 2023-10-18T12:31:20.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.225+0000 7fa475308b80 20 client.26348 trim_cache size 0 max 16384 2023-10-18T12:31:20.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 Processor -- start 2023-10-18T12:31:20.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 -- start start 2023-10-18T12:31:20.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646713b4920 con 0x5646715e7680 2023-10-18T12:31:20.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646715ec060 con 0x5646715e1820 2023-10-18T12:31:20.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa475308b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646715eda30 con 0x5646715e6d60 2023-10-18T12:31:20.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa45affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa45a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa45affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50014/0 (socket says 172.21.15.70:50014) 2023-10-18T12:31:20.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa45affd700 1 -- 172.21.15.70:0/3101269317 learned_addr learned my addr 172.21.15.70:0/3101269317 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:20.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.245+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45affd700 1 -- 172.21.15.70:0/3101269317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 msgr2=0x5646715f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45affd700 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45affd700 1 -- 172.21.15.70:0/3101269317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 msgr2=0x5646715e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45affd700 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45affd700 1 -- 172.21.15.70:0/3101269317 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56467135ede0 con 0x5646715e1820 2023-10-18T12:31:20.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:20.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa45affd700 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f2030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa44c005920 tx=0x7fa44c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdb40cac9895d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:20.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa4427fc700 1 -- 172.21.15.70:0/3101269317 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c016040 con 0x5646715e1820 2023-10-18T12:31:20.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa4427fc700 1 -- 172.21.15.70:0/3101269317 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c017040 con 0x5646715e1820 2023-10-18T12:31:20.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.246+0000 7fa4427fc700 1 -- 172.21.15.70:0/3101269317 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004cf0 con 0x5646715e1820 2023-10-18T12:31:20.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 -- 172.21.15.70:0/3101269317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 msgr2=0x5646715f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f2030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa44c005920 tx=0x7fa44c009260 comp rx=0 tx=0).stop 2023-10-18T12:31:20.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 -- 172.21.15.70:0/3101269317 shutdown_connections 2023-10-18T12:31:20.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e7680 0x5646715e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f2030 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 --2- 172.21.15.70:0/3101269317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e6d60 0x5646715f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 -- 172.21.15.70:0/3101269317 >> 172.21.15.70:0/3101269317 conn(0x564671555720 msgr2=0x5646715e02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:20.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 -- 172.21.15.70:0/3101269317 shutdown_connections 2023-10-18T12:31:20.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.247+0000 7fa475308b80 1 -- 172.21.15.70:0/3101269317 wait complete. 2023-10-18T12:31:20.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 -- start start 2023-10-18T12:31:20.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 Processor -- start 2023-10-18T12:31:20.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e6d60 0x5646715f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e7680 0x5646715f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646715f1310 con 0x5646715e6d60 2023-10-18T12:31:20.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646715f1480 con 0x5646715e1820 2023-10-18T12:31:20.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467164e9e0 con 0x5646715e7680 2023-10-18T12:31:20.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa475308b80 10 client.0 _ll_register_callbacks cb 0x56467141aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:20.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.248+0000 7fa45affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50020/0 (socket says 172.21.15.70:50020) 2023-10-18T12:31:20.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 -- 172.21.15.70:0/2775861990 learned_addr learned my addr 172.21.15.70:0/2775861990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:20.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e7680 0x5646715f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e6d60 0x5646715f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 -- 172.21.15.70:0/2775861990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e7680 msgr2=0x5646715f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e7680 0x5646715f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 -- 172.21.15.70:0/2775861990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e6d60 msgr2=0x5646715f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:20.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e6d60 0x5646715f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 -- 172.21.15.70:0/2775861990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c01fa60 con 0x5646715e1820 2023-10-18T12:31:20.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45b7fe700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e7680 0x5646715f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:20.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa45affd700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f20d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa44c006010 tx=0x7fa44c01f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2938fd301433ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:20.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.249+0000 7fa43affd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:20.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.250+0000 7fa43affd700 1 -- 172.21.15.70:0/2775861990 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c016050 con 0x5646715e1820 2023-10-18T12:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.250+0000 7fa43affd700 1 -- 172.21.15.70:0/2775861990 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa44c0039c0 con 0x5646715e1820 2023-10-18T12:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.250+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4500a25d0 con 0x5646715e1820 2023-10-18T12:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa43affd700 1 -- 172.21.15.70:0/2775861990 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c003ba0 con 0x5646715e1820 2023-10-18T12:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa475308b80 10 client.26354 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564671347c60 con 0x5646715e1820 2023-10-18T12:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa43affd700 1 -- 172.21.15.70:0/2775861990 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4500a25d0 con 0x5646715e1820 2023-10-18T12:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa43affd700 1 -- 172.21.15.70:0/2775861990 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa44c02e070 con 0x5646715e1820 2023-10-18T12:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa475308b80 10 client.26354 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa475308b80 10 client.26354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.254+0000 7fa475308b80 20 client.26354 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.255+0000 7fa475308b80 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646716501a0 0x5646716525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.255+0000 7fa475308b80 4 client.26354 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:31:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.255+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56467152ce50 con 0x5646716501a0 2023-10-18T12:31:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.255+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646716501a0 0x5646716525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.255+0000 7fa43affd700 10 client.26354 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.255+0000 7fa45a7fc700 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646716501a0 0x5646716525c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2519b772e41f2e server_cookie=5f78d52f6e0656c2 in_seq=0 out_seq=0 2023-10-18T12:31:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa43affd700 1 -- 172.21.15.70:0/2775861990 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56467152ce50 con 0x5646716501a0 2023-10-18T12:31:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa43affd700 10 client.26354 handle_command_reply: tid=0 2023-10-18T12:31:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 client.26354 shutdown 2023-10-18T12:31:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 msgr2=0x5646715f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f20d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa44c006010 tx=0x7fa44c01f250 comp rx=0 tx=0).stop 2023-10-18T12:31:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 shutdown_connections 2023-10-18T12:31:20.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5646716501a0 0x5646716525c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646715e7680 0x5646715f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646715e1820 0x5646715f20d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa43affd700 0 client.26354 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 --2- 172.21.15.70:0/2775861990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5646715e6d60 0x5646715f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.257+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 >> 172.21.15.70:0/2775861990 conn(0x564671555720 msgr2=0x5646715d5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 shutdown_connections 2023-10-18T12:31:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa475308b80 1 -- 172.21.15.70:0/2775861990 wait complete. 2023-10-18T12:31:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa475308b80 20 client.26354 trim_cache size 0 max 16384 2023-10-18T12:31:20.271 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:31:20.272 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:31:20.272 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:31:20.272 INFO:teuthology.orchestra.run.smithi070.stdout: "877dd2c8-8109-4fb7-93df-1bc36459c0d5": { 2023-10-18T12:31:20.272 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:31:20.273 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "877dd2c8-8109-4fb7-93df-1bc36459c0d5", 2023-10-18T12:31:20.273 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:31:20.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:31:20.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:31:20.273 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:31:20.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4440a15f0 msgr2=0x7fa4440a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa45bfff700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4440a15f0 0x7fa4440a3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b780 tx=0x7fa454009380 comp rx=0 tx=0).stop 2023-10-18T12:31:20.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 msgr2=0x7fa45c19d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:20.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa45bfff700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 0x7fa45c19d790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e9a0 tx=0x7fa45000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 shutdown_connections 2023-10-18T12:31:20.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.258+0000 7fa45bfff700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa45c1115d0 0x7fa45c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.259+0000 7fa45bfff700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa4440a15f0 0x7fa4440a3ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.259+0000 7fa45bfff700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa45c072b90 0x7fa45c19d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.259+0000 7fa45bfff700 1 --2- 172.21.15.70:0/2281959253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0734b0 0x7fa45c19d790 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:20.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.259+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 >> 172.21.15.70:0/2281959253 conn(0x7fa45c0fc0f0 msgr2=0x7fa45c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:20.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.259+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 shutdown_connections 2023-10-18T12:31:20.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:20.259+0000 7fa45bfff700 1 -- 172.21.15.70:0/2281959253 wait complete. 2023-10-18T12:31:20.280 INFO:tasks.cephfs.filesystem:scrub status for tag:877dd2c8-8109-4fb7-93df-1bc36459c0d5 - {'path': '/', 'tag': '877dd2c8-8109-4fb7-93df-1bc36459c0d5', 'options': 'recursive,force'} 2023-10-18T12:31:20.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:20 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:31:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:20 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:20 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:31:20.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:20 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:20 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:31:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:20 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:20 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:31:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:20 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:20 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:31:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:20 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:20 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:31:20.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:20 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:20.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 560602 12.58 MB/sec execute 109 sec latency 15.668 ms 2023-10-18T12:31:21.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:21 smithi070 ceph-mon[98639]: pgmap v474: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 183 op/s 2023-10-18T12:31:21.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:21 smithi152 ceph-mon[134767]: pgmap v474: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 183 op/s 2023-10-18T12:31:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:21 smithi186 ceph-mon[135253]: pgmap v474: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 183 op/s 2023-10-18T12:31:21.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 562679 12.58 MB/sec execute 110 sec latency 12.116 ms 2023-10-18T12:31:22.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:22 smithi070 ceph-mon[98639]: pgmap v475: 129 pgs: 129 active+clean; 337 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 3.6 MiB/s wr, 148 op/s 2023-10-18T12:31:22.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:22 smithi152 ceph-mon[134767]: pgmap v475: 129 pgs: 129 active+clean; 337 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 3.6 MiB/s wr, 148 op/s 2023-10-18T12:31:22.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:22 smithi186 ceph-mon[135253]: pgmap v475: 129 pgs: 129 active+clean; 337 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 3.6 MiB/s wr, 148 op/s 2023-10-18T12:31:22.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 564880 12.58 MB/sec execute 111 sec latency 116.078 ms 2023-10-18T12:31:23.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:23 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:31:23.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:23 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:31:23.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:23 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:31:23.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 566348 12.53 MB/sec execute 112 sec latency 25.534 ms 2023-10-18T12:31:24.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 568776 12.54 MB/sec execute 113 sec latency 16.765 ms 2023-10-18T12:31:25.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:25 smithi070 ceph-mon[98639]: pgmap v476: 129 pgs: 129 active+clean; 333 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.2 MiB/s wr, 161 op/s 2023-10-18T12:31:25.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:25 smithi152 ceph-mon[134767]: pgmap v476: 129 pgs: 129 active+clean; 333 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.2 MiB/s wr, 161 op/s 2023-10-18T12:31:25.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:25 smithi186 ceph-mon[135253]: pgmap v476: 129 pgs: 129 active+clean; 333 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.2 MiB/s wr, 161 op/s 2023-10-18T12:31:25.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 570798 12.49 MB/sec execute 114 sec latency 21.588 ms 2023-10-18T12:31:26.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 572832 12.49 MB/sec execute 115 sec latency 20.210 ms 2023-10-18T12:31:27.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:27 smithi070 ceph-mon[98639]: pgmap v477: 129 pgs: 129 active+clean; 339 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 171 op/s 2023-10-18T12:31:27.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:27 smithi152 ceph-mon[134767]: pgmap v477: 129 pgs: 129 active+clean; 339 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 171 op/s 2023-10-18T12:31:27.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:27 smithi186 ceph-mon[135253]: pgmap v477: 129 pgs: 129 active+clean; 339 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 171 op/s 2023-10-18T12:31:27.850 INFO:tasks.workunit.client.0.smithi070.stdout: 1 575189 12.50 MB/sec execute 116 sec latency 95.806 ms 2023-10-18T12:31:28.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 576927 12.48 MB/sec execute 117 sec latency 27.796 ms 2023-10-18T12:31:29.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:29 smithi070 ceph-mon[98639]: pgmap v478: 129 pgs: 129 active+clean; 339 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2023-10-18T12:31:29.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:29 smithi152 ceph-mon[134767]: pgmap v478: 129 pgs: 129 active+clean; 339 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2023-10-18T12:31:29.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:29 smithi186 ceph-mon[135253]: pgmap v478: 129 pgs: 129 active+clean; 339 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2023-10-18T12:31:29.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 579449 12.49 MB/sec execute 118 sec latency 33.991 ms 2023-10-18T12:31:30.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 581892 12.49 MB/sec execute 119 sec latency 13.412 ms 2023-10-18T12:31:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:31 smithi070 ceph-mon[98639]: pgmap v479: 129 pgs: 129 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 230 op/s 2023-10-18T12:31:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:31 smithi152 ceph-mon[134767]: pgmap v479: 129 pgs: 129 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 230 op/s 2023-10-18T12:31:31.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:31 smithi186 ceph-mon[135253]: pgmap v479: 129 pgs: 129 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 230 op/s 2023-10-18T12:31:31.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 583911 12.49 MB/sec execute 120 sec latency 21.491 ms 2023-10-18T12:31:32.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 586295 12.50 MB/sec execute 121 sec latency 63.999 ms 2023-10-18T12:31:33.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:33 smithi070 ceph-mon[98639]: pgmap v480: 129 pgs: 129 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 235 op/s 2023-10-18T12:31:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:33 smithi152 ceph-mon[134767]: pgmap v480: 129 pgs: 129 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 235 op/s 2023-10-18T12:31:33.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:33 smithi186 ceph-mon[135253]: pgmap v480: 129 pgs: 129 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 235 op/s 2023-10-18T12:31:33.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 587777 12.44 MB/sec execute 122 sec latency 28.154 ms 2023-10-18T12:31:34.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 590214 12.45 MB/sec execute 123 sec latency 14.455 ms 2023-10-18T12:31:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:35 smithi070 ceph-mon[98639]: pgmap v481: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2023-10-18T12:31:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:35 smithi152 ceph-mon[134767]: pgmap v481: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2023-10-18T12:31:35.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:35 smithi186 ceph-mon[135253]: pgmap v481: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2023-10-18T12:31:35.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 592398 12.44 MB/sec execute 124 sec latency 21.288 ms 2023-10-18T12:31:36.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:36 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:31:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:36 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:31:36.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:36 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:31:36.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 594415 12.42 MB/sec execute 125 sec latency 16.588 ms 2023-10-18T12:31:37.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:37 smithi070 ceph-mon[98639]: pgmap v482: 129 pgs: 129 active+clean; 354 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 270 op/s 2023-10-18T12:31:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:31:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:31:37.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:31:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:37 smithi152 ceph-mon[134767]: pgmap v482: 129 pgs: 129 active+clean; 354 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 270 op/s 2023-10-18T12:31:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:31:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:31:37.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:31:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:37 smithi186 ceph-mon[135253]: pgmap v482: 129 pgs: 129 active+clean; 354 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 270 op/s 2023-10-18T12:31:37.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:31:37.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:31:37.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:31:37.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 596848 12.45 MB/sec execute 126 sec latency 85.551 ms 2023-10-18T12:31:38.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 598403 12.40 MB/sec execute 127 sec latency 27.428 ms 2023-10-18T12:31:39.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:39 smithi070 ceph-mon[98639]: pgmap v483: 129 pgs: 129 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 226 op/s 2023-10-18T12:31:39.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:39 smithi152 ceph-mon[134767]: pgmap v483: 129 pgs: 129 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 226 op/s 2023-10-18T12:31:39.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:39 smithi186 ceph-mon[135253]: pgmap v483: 129 pgs: 129 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 226 op/s 2023-10-18T12:31:39.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 600798 12.41 MB/sec execute 128 sec latency 17.036 ms 2023-10-18T12:31:40.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 602850 12.38 MB/sec execute 129 sec latency 33.037 ms 2023-10-18T12:31:41.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:41 smithi070 ceph-mon[98639]: pgmap v484: 129 pgs: 129 active+clean; 357 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-10-18T12:31:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:41 smithi152 ceph-mon[134767]: pgmap v484: 129 pgs: 129 active+clean; 357 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-10-18T12:31:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:41 smithi186 ceph-mon[135253]: pgmap v484: 129 pgs: 129 active+clean; 357 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-10-18T12:31:41.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 605061 12.39 MB/sec execute 130 sec latency 14.288 ms 2023-10-18T12:31:42.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:42 smithi070 ceph-mon[98639]: pgmap v485: 129 pgs: 129 active+clean; 359 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-10-18T12:31:42.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:42 smithi152 ceph-mon[134767]: pgmap v485: 129 pgs: 129 active+clean; 359 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-10-18T12:31:42.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:42 smithi186 ceph-mon[135253]: pgmap v485: 129 pgs: 129 active+clean; 359 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-10-18T12:31:42.851 INFO:tasks.workunit.client.0.smithi070.stdout: 1 607441 12.41 MB/sec execute 131 sec latency 41.484 ms 2023-10-18T12:31:43.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 608876 12.37 MB/sec execute 132 sec latency 27.803 ms 2023-10-18T12:31:44.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 611437 12.38 MB/sec execute 133 sec latency 12.330 ms 2023-10-18T12:31:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:45 smithi070 ceph-mon[98639]: pgmap v486: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-18T12:31:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:45 smithi152 ceph-mon[134767]: pgmap v486: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-18T12:31:45.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:45 smithi186 ceph-mon[135253]: pgmap v486: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 239 op/s 2023-10-18T12:31:45.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 613771 12.37 MB/sec execute 134 sec latency 61.974 ms 2023-10-18T12:31:46.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 615863 12.37 MB/sec execute 135 sec latency 13.702 ms 2023-10-18T12:31:47.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:47 smithi070 ceph-mon[98639]: pgmap v487: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 203 op/s 2023-10-18T12:31:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:47 smithi152 ceph-mon[134767]: pgmap v487: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 203 op/s 2023-10-18T12:31:47.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:47 smithi186 ceph-mon[135253]: pgmap v487: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 203 op/s 2023-10-18T12:31:47.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 618345 12.39 MB/sec execute 136 sec latency 14.557 ms 2023-10-18T12:31:48.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 620384 12.35 MB/sec execute 137 sec latency 29.025 ms 2023-10-18T12:31:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:49 smithi070 ceph-mon[98639]: pgmap v488: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2023-10-18T12:31:49.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:49 smithi152 ceph-mon[134767]: pgmap v488: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2023-10-18T12:31:49.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:49 smithi186 ceph-mon[135253]: pgmap v488: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2023-10-18T12:31:49.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 622501 12.35 MB/sec execute 138 sec latency 19.566 ms 2023-10-18T12:31:50.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:31:50.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 -- 172.21.15.70:0/1557388580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb490100720 msgr2=0x7fb490100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 --2- 172.21.15.70:0/1557388580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb490100720 0x7fb490100ba0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009990 tx=0x7fb47c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 -- 172.21.15.70:0/1557388580 shutdown_connections 2023-10-18T12:31:50.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 --2- 172.21.15.70:0/1557388580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4901010e0 0x7fb49010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 --2- 172.21.15.70:0/1557388580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb490106730 0x7fb490106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 --2- 172.21.15.70:0/1557388580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb490100720 0x7fb490100ba0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 -- 172.21.15.70:0/1557388580 >> 172.21.15.70:0/1557388580 conn(0x7fb4900fc1b0 msgr2=0x7fb4900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 -- 172.21.15.70:0/1557388580 shutdown_connections 2023-10-18T12:31:50.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.494+0000 7fb496bff700 1 -- 172.21.15.70:0/1557388580 wait complete. 2023-10-18T12:31:50.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 Processor -- start 2023-10-18T12:31:50.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 -- start start 2023-10-18T12:31:50.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb490100720 0x7fb4901096d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 0x7fb490109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb490106730 0x7fb49010a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901a9450 con 0x7fb490106730 2023-10-18T12:31:50.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901a9590 con 0x7fb4901010e0 2023-10-18T12:31:50.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.495+0000 7fb496bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901a96d0 con 0x7fb490100720 2023-10-18T12:31:50.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 0x7fb490109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 0x7fb490109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48876/0 (socket says 172.21.15.70:48876) 2023-10-18T12:31:50.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 -- 172.21.15.70:0/3236149021 learned_addr learned my addr 172.21.15.70:0/3236149021 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:50.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb49519c700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb490106730 0x7fb49010a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb49499b700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb490100720 0x7fb4901096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 -- 172.21.15.70:0/3236149021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb490100720 msgr2=0x7fb4901096d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb490100720 0x7fb4901096d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 -- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb490106730 msgr2=0x7fb49010a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb490106730 0x7fb49010a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 -- 172.21.15.70:0/3236149021 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009520 con 0x7fb4901010e0 2023-10-18T12:31:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.496+0000 7fb48ffff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 0x7fb490109c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005910 tx=0x7fb47c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.497+0000 7fb48dffb700 1 -- 172.21.15.70:0/3236149021 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c01d070 con 0x7fb4901010e0 2023-10-18T12:31:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.497+0000 7fb48dffb700 1 -- 172.21.15.70:0/3236149021 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb47c022470 con 0x7fb4901010e0 2023-10-18T12:31:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.497+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47c0095f0 con 0x7fb4901010e0 2023-10-18T12:31:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.497+0000 7fb48dffb700 1 -- 172.21.15.70:0/3236149021 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c017650 con 0x7fb4901010e0 2023-10-18T12:31:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.497+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4901a9c70 con 0x7fb4901010e0 2023-10-18T12:31:50.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.500+0000 7fb48dffb700 1 -- 172.21.15.70:0/3236149021 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb47c0225e0 con 0x7fb4901010e0 2023-10-18T12:31:50.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.501+0000 7fb48dffb700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4800a1670 0x7fb4800a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.501+0000 7fb4aa56db80 1 Processor -- start 2023-10-18T12:31:50.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.501+0000 7fb48dffb700 1 -- 172.21.15.70:0/3236149021 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb47c0ce630 con 0x7fb4901010e0 2023-10-18T12:31:50.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.501+0000 7fb49499b700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4800a1670 0x7fb4800a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.501+0000 7fb4aa56db80 1 -- start start 2023-10-18T12:31:50.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ee7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee8500 0x558706ee2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706ee2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706e556e0 con 0x558706ee8500 2023-10-18T12:31:50.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706e3bc60 con 0x558706ee7be0 2023-10-18T12:31:50.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706e39580 con 0x558706ee25d0 2023-10-18T12:31:50.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb49519c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706ee2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb49499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ee7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb49519c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706ee2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58806/0 (socket says 172.21.15.70:58806) 2023-10-18T12:31:50.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb49519c700 1 -- 172.21.15.70:0/20930563 learned_addr learned my addr 172.21.15.70:0/20930563 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:50.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.502+0000 7fb48ffff700 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee8500 0x558706ee2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 -- 172.21.15.70:0/20930563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 msgr2=0x558706ee2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706ee2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 -- 172.21.15.70:0/20930563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee8500 msgr2=0x558706ee2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee8500 0x558706ee2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 -- 172.21.15.70:0/20930563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558706df23f0 con 0x558706ee7be0 2023-10-18T12:31:50.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49519c700 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706ee2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:50.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4800a1670 0x7fb4800a3b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb484011660 tx=0x7fb484011a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb49499b700 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ee7fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb484011e20 tx=0x7fb48401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add43835d81ea11b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb47affd700 1 -- 172.21.15.70:0/20930563 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484003b90 con 0x558706ee7be0 2023-10-18T12:31:50.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb47affd700 1 -- 172.21.15.70:0/20930563 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb484015070 con 0x558706ee7be0 2023-10-18T12:31:50.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.503+0000 7fb47affd700 1 -- 172.21.15.70:0/20930563 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484020450 con 0x558706ee7be0 2023-10-18T12:31:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 -- 172.21.15.70:0/20930563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 msgr2=0x558706ee7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ee7fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb484011e20 tx=0x7fb48401d000 comp rx=0 tx=0).stop 2023-10-18T12:31:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 -- 172.21.15.70:0/20930563 shutdown_connections 2023-10-18T12:31:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706ee2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ee7fc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/20930563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee8500 0x558706ee2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 -- 172.21.15.70:0/20930563 >> 172.21.15.70:0/20930563 conn(0x558706e56720 msgr2=0x558706edcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.504+0000 7fb4aa56db80 1 -- 172.21.15.70:0/20930563 shutdown_connections 2023-10-18T12:31:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.505+0000 7fb4aa56db80 1 -- 172.21.15.70:0/20930563 wait complete. 2023-10-18T12:31:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.505+0000 7fb4aa56db80 1 -- start start 2023-10-18T12:31:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.505+0000 7fb4aa56db80 1 Processor -- start 2023-10-18T12:31:50.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706eeb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706eebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 0x558706eec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706e34e70 con 0x558706ee7be0 2023-10-18T12:31:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706dd1710 con 0x558706ee8500 2023-10-18T12:31:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706dd13b0 con 0x558706ee25d0 2023-10-18T12:31:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb4aa56db80 10 client.0 _ll_register_callbacks cb 0x558706d1ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706eebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb49519c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 0x558706eec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb49499b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706eeb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb49519c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 0x558706eec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48898/0 (socket says 172.21.15.70:48898) 2023-10-18T12:31:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.506+0000 7fb49519c700 1 -- 172.21.15.70:0/2669934693 learned_addr learned my addr 172.21.15.70:0/2669934693 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb49519c700 1 -- 172.21.15.70:0/2669934693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 msgr2=0x558706eeb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb49519c700 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706eeb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb49519c700 1 -- 172.21.15.70:0/2669934693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 msgr2=0x558706eebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb49519c700 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706eebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb49519c700 1 -- 172.21.15.70:0/2669934693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb488011330 con 0x558706ee8500 2023-10-18T12:31:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb49519c700 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 0x558706eec350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb488014600 tx=0x7fb488014910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cce837560fe9d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb46ffff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb46ffff700 1 -- 172.21.15.70:0/2669934693 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800e040 con 0x558706ee8500 2023-10-18T12:31:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558706e39ef0 con 0x558706ee8500 2023-10-18T12:31:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.507+0000 7fb46ffff700 1 -- 172.21.15.70:0/2669934693 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb48801a070 con 0x558706ee8500 2023-10-18T12:31:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb46ffff700 1 -- 172.21.15.70:0/2669934693 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb488014eb0 con 0x558706ee8500 2023-10-18T12:31:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 10 client.26366 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fb484004040 con 0x558706ee8500 2023-10-18T12:31:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb46ffff700 1 -- 172.21.15.70:0/2669934693 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558706e39ef0 con 0x558706ee8500 2023-10-18T12:31:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb46ffff700 1 -- 172.21.15.70:0/2669934693 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb48801f510 con 0x558706ee8500 2023-10-18T12:31:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 10 client.26366 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 20 client.26366 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:50.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706efc010 0x558706efe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 4 client.26366 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:50.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558706eb18b0 con 0x558706efc010 2023-10-18T12:31:50.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.508+0000 7fb49499b700 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706efc010 0x558706efe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.509+0000 7fb49499b700 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706efc010 0x558706efe430 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15190cabeaba0503 server_cookie=9b3018837874cb3d in_seq=0 out_seq=0 2023-10-18T12:31:50.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.509+0000 7fb46ffff700 10 client.26366 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:50.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.510+0000 7fb46ffff700 1 -- 172.21.15.70:0/2669934693 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558706eb18b0 con 0x558706efc010 2023-10-18T12:31:50.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.510+0000 7fb46ffff700 10 client.26366 handle_command_reply: tid=0 2023-10-18T12:31:50.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.510+0000 7fb4aa56db80 1 client.26366 shutdown 2023-10-18T12:31:50.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.510+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 msgr2=0x558706eec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.510+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 0x558706eec350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb488014600 tx=0x7fb488014910 comp rx=0 tx=0).stop 2023-10-18T12:31:50.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 shutdown_connections 2023-10-18T12:31:50.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706efc010 0x558706efe430 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee8500 0x558706eec350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee25d0 0x558706eeb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/2669934693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706eebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb46ffff700 0 client.26366 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 >> 172.21.15.70:0/2669934693 conn(0x558706e56720 msgr2=0x558706ee1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 shutdown_connections 2023-10-18T12:31:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 1 -- 172.21.15.70:0/2669934693 wait complete. 2023-10-18T12:31:50.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.511+0000 7fb4aa56db80 20 client.26366 trim_cache size 0 max 16384 2023-10-18T12:31:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 Processor -- start 2023-10-18T12:31:50.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 -- start start 2023-10-18T12:31:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 0x558706ee2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706ee69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ee6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706cb5920 con 0x558706ee7be0 2023-10-18T12:31:50.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.530+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706eb18b0 con 0x558706ee25d0 2023-10-18T12:31:50.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706e39ef0 con 0x558706ee8500 2023-10-18T12:31:50.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 0x558706ee2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 0x558706ee2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48914/0 (socket says 172.21.15.70:48914) 2023-10-18T12:31:50.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 -- 172.21.15.70:0/3982029478 learned_addr learned my addr 172.21.15.70:0/3982029478 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:50.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb48ffff700 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706ee69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49519c700 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ee6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 -- 172.21.15.70:0/3982029478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 msgr2=0x558706ee6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ee6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 -- 172.21.15.70:0/3982029478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 msgr2=0x558706ee69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706ee69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 -- 172.21.15.70:0/3982029478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558706c5fde0 con 0x558706ee25d0 2023-10-18T12:31:50.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb48ffff700 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706ee69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:50.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.531+0000 7fb49499b700 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 0x558706ee2090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb48400fb00 tx=0x7fb484009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0667be7ef4504b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.532+0000 7fb47b7fe700 1 -- 172.21.15.70:0/3982029478 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484033570 con 0x558706ee25d0 2023-10-18T12:31:50.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.532+0000 7fb47b7fe700 1 -- 172.21.15.70:0/3982029478 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb48400fb70 con 0x558706ee25d0 2023-10-18T12:31:50.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.532+0000 7fb47b7fe700 1 -- 172.21.15.70:0/3982029478 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484003e20 con 0x558706ee25d0 2023-10-18T12:31:50.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.532+0000 7fb4aa56db80 1 -- 172.21.15.70:0/3982029478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 msgr2=0x558706ee2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.532+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 0x558706ee2090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb48400fb00 tx=0x7fb484009d50 comp rx=0 tx=0).stop 2023-10-18T12:31:50.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 -- 172.21.15.70:0/3982029478 shutdown_connections 2023-10-18T12:31:50.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ee6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee25d0 0x558706ee2090 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/3982029478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee7be0 0x558706ee69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 -- 172.21.15.70:0/3982029478 >> 172.21.15.70:0/3982029478 conn(0x558706e56720 msgr2=0x558706ee5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 -- 172.21.15.70:0/3982029478 shutdown_connections 2023-10-18T12:31:50.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 -- 172.21.15.70:0/3982029478 wait complete. 2023-10-18T12:31:50.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 -- start start 2023-10-18T12:31:50.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.533+0000 7fb4aa56db80 1 Processor -- start 2023-10-18T12:31:50.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee25d0 0x558706ef2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ef1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ef1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706ef1e50 con 0x558706ee25d0 2023-10-18T12:31:50.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706ef1fc0 con 0x558706ee7be0 2023-10-18T12:31:50.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558706f4fe50 con 0x558706ee8500 2023-10-18T12:31:50.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb4aa56db80 10 client.0 _ll_register_callbacks cb 0x558706d1ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:50.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ef1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49519c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ef1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee25d0 0x558706ef2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ef1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48928/0 (socket says 172.21.15.70:48928) 2023-10-18T12:31:50.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb48ffff700 1 -- 172.21.15.70:0/912836722 learned_addr learned my addr 172.21.15.70:0/912836722 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:50.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49499b700 1 -- 172.21.15.70:0/912836722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 msgr2=0x558706ef1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49499b700 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ef1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49499b700 1 -- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 msgr2=0x558706ef1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:50.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49499b700 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ef1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb49499b700 1 -- 172.21.15.70:0/912836722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4901a9c70 con 0x558706ee25d0 2023-10-18T12:31:50.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.534+0000 7fb48ffff700 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ef1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:50.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb49499b700 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee25d0 0x558706ef2e70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb484000c00 tx=0x7fb48400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd08052c5f2a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb46ffff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:31:50.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb46ffff700 1 -- 172.21.15.70:0/912836722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484003d60 con 0x558706ee25d0 2023-10-18T12:31:50.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb46ffff700 1 -- 172.21.15.70:0/912836722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb484031070 con 0x558706ee25d0 2023-10-18T12:31:50.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb484015070 con 0x558706ee25d0 2023-10-18T12:31:50.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb46ffff700 1 -- 172.21.15.70:0/912836722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb484003ec0 con 0x558706ee25d0 2023-10-18T12:31:50.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb4aa56db80 10 client.16347 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:50.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558706df23f0 con 0x558706ee25d0 2023-10-18T12:31:50.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.535+0000 7fb46ffff700 1 -- 172.21.15.70:0/912836722 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb484015070 con 0x558706ee25d0 2023-10-18T12:31:50.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb46ffff700 1 -- 172.21.15.70:0/912836722 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb48403cb30 con 0x558706ee25d0 2023-10-18T12:31:50.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb4aa56db80 10 client.16347 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:50.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb4aa56db80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:50.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb4aa56db80 20 client.16347 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:50.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706f51260 0x558706f53650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb4aa56db80 4 client.16347 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:31:50.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558706e2de50 con 0x558706f51260 2023-10-18T12:31:50.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb48ffff700 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706f51260 0x558706f53650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb46ffff700 10 client.16347 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:50.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.536+0000 7fb48ffff700 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706f51260 0x558706f53650 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a58f3864255ac10 server_cookie=3a0341266f48476f in_seq=0 out_seq=0 2023-10-18T12:31:50.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb46ffff700 1 -- 172.21.15.70:0/912836722 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558706e2de50 con 0x558706f51260 2023-10-18T12:31:50.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb46ffff700 10 client.16347 handle_command_reply: tid=0 2023-10-18T12:31:50.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 client.16347 shutdown 2023-10-18T12:31:50.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee25d0 msgr2=0x558706ef2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee25d0 0x558706ef2e70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb484000c00 tx=0x7fb48400fce0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 shutdown_connections 2023-10-18T12:31:50.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558706f51260 0x558706f53650 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558706ee8500 0x558706ef1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558706ee25d0 0x558706ef2e70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 --2- 172.21.15.70:0/912836722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558706ee7be0 0x558706ef1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 >> 172.21.15.70:0/912836722 conn(0x558706e56720 msgr2=0x558706ee69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb46ffff700 0 client.16347 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:50.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 shutdown_connections 2023-10-18T12:31:50.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 1 -- 172.21.15.70:0/912836722 wait complete. 2023-10-18T12:31:50.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.537+0000 7fb4aa56db80 20 client.16347 trim_cache size 0 max 16384 2023-10-18T12:31:50.554 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:31:50.554 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:31:50.555 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:31:50.555 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:31:50.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4800a1670 msgr2=0x7fb4800a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4800a1670 0x7fb4800a3b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb484011660 tx=0x7fb484011a20 comp rx=0 tx=0).stop 2023-10-18T12:31:50.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 msgr2=0x7fb490109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 0x7fb490109c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005910 tx=0x7fb47c004730 comp rx=0 tx=0).stop 2023-10-18T12:31:50.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 shutdown_connections 2023-10-18T12:31:50.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb490106730 0x7fb49010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb490100720 0x7fb4901096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb4800a1670 0x7fb4800a3b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 --2- 172.21.15.70:0/3236149021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4901010e0 0x7fb490109c10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 >> 172.21.15.70:0/3236149021 conn(0x7fb4900fc1b0 msgr2=0x7fb490104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 shutdown_connections 2023-10-18T12:31:50.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.538+0000 7fb496bff700 1 -- 172.21.15.70:0/3236149021 wait complete. 2023-10-18T12:31:50.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:31:50.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:31:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 -- 172.21.15.70:0/2820212628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c1066d0 msgr2=0x7fc07c106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 --2- 172.21.15.70:0/2820212628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c1066d0 0x7fc07c106ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0099c0 tx=0x7fc06c009cd0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 -- 172.21.15.70:0/2820212628 shutdown_connections 2023-10-18T12:31:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 --2- 172.21.15.70:0/2820212628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c0681f0 0x7fc07c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 --2- 172.21.15.70:0/2820212628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c1066d0 0x7fc07c106ab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 --2- 172.21.15.70:0/2820212628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07c067830 0x7fc07c067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 -- 172.21.15.70:0/2820212628 >> 172.21.15.70:0/2820212628 conn(0x7fc07c076400 msgr2=0x7fc07c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 -- 172.21.15.70:0/2820212628 shutdown_connections 2023-10-18T12:31:50.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.773+0000 7fc081bff700 1 -- 172.21.15.70:0/2820212628 wait complete. 2023-10-18T12:31:50.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.774+0000 7fc081bff700 1 Processor -- start 2023-10-18T12:31:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.774+0000 7fc081bff700 1 -- start start 2023-10-18T12:31:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc081bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 0x7fc07c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc081bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07c0681f0 0x7fc07c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc081bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c1066d0 0x7fc07c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc081bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1991e0 con 0x7fc07c0681f0 2023-10-18T12:31:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc081bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c199350 con 0x7fc07c067830 2023-10-18T12:31:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc081bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c199510 con 0x7fc07c1066d0 2023-10-18T12:31:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 0x7fc07c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 0x7fc07c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48952/0 (socket says 172.21.15.70:48952) 2023-10-18T12:31:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 -- 172.21.15.70:0/1991212862 learned_addr learned my addr 172.21.15.70:0/1991212862 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07affd700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07c0681f0 0x7fc07c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07bfff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c1066d0 0x7fc07c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 -- 172.21.15.70:0/1991212862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c1066d0 msgr2=0x7fc07c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c1066d0 0x7fc07c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 -- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07c0681f0 msgr2=0x7fc07c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07c0681f0 0x7fc07c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.775+0000 7fc07b7fe700 1 -- 172.21.15.70:0/1991212862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0095f0 con 0x7fc07c067830 2023-10-18T12:31:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.776+0000 7fc07b7fe700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 0x7fc07c19ea50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00f690 tx=0x7fc06c00f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.776+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c01d070 con 0x7fc07c067830 2023-10-18T12:31:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.776+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c199790 con 0x7fc07c067830 2023-10-18T12:31:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.776+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c005050 con 0x7fc07c067830 2023-10-18T12:31:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.776+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c004a50 con 0x7fc07c067830 2023-10-18T12:31:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.776+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c1a5a40 con 0x7fc07c067830 2023-10-18T12:31:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.777+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c0fec10 con 0x7fc07c067830 2023-10-18T12:31:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.779+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc06c0051c0 con 0x7fc07c067830 2023-10-18T12:31:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.781+0000 7fc078ff9700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0640a15f0 0x7fc0640a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.781+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fc06c0ce540 con 0x7fc07c067830 2023-10-18T12:31:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.781+0000 7fc07affd700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0640a15f0 0x7fc0640a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.782+0000 7fc07affd700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0640a15f0 0x7fc0640a3ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc07c19a220 tx=0x7fc070009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:50.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:50.784+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc06c097590 con 0x7fc07c067830 2023-10-18T12:31:50.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 625316 12.39 MB/sec execute 139 sec latency 12.638 ms 2023-10-18T12:31:51.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.053+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc07c199fd0 con 0x7fc07c067830 2023-10-18T12:31:51.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.054+0000 7fc078ff9700 1 -- 172.21.15.70:0/1991212862 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fc06c095e60 con 0x7fc07c067830 2023-10-18T12:31:51.057 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:31:51.057 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:31:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0640a15f0 msgr2=0x7fc0640a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0640a15f0 0x7fc0640a3ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc07c19a220 tx=0x7fc070009260 comp rx=0 tx=0).stop 2023-10-18T12:31:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 msgr2=0x7fc07c19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 0x7fc07c19ea50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00f690 tx=0x7fc06c00f6c0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 shutdown_connections 2023-10-18T12:31:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc07c1066d0 0x7fc07c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc0640a15f0 0x7fc0640a3ab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc07c067830 0x7fc07c19ea50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 --2- 172.21.15.70:0/1991212862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07c0681f0 0x7fc07c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 >> 172.21.15.70:0/1991212862 conn(0x7fc07c076400 msgr2=0x7fc07c10bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 shutdown_connections 2023-10-18T12:31:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.056+0000 7fc081bff700 1 -- 172.21.15.70:0/1991212862 wait complete. 2023-10-18T12:31:51.062 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:31:51.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:31:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 -- 172.21.15.70:0/2616785293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe664072ba0 msgr2=0x7fe664072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 --2- 172.21.15.70:0/2616785293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe664072ba0 0x7fe664072f80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe660009990 tx=0x7fe660009ca0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 -- 172.21.15.70:0/2616785293 shutdown_connections 2023-10-18T12:31:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 --2- 172.21.15.70:0/2616785293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66410cfd0 0x7fe66410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 --2- 172.21.15.70:0/2616785293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe664072ba0 0x7fe664072f80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 --2- 172.21.15.70:0/2616785293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6640734c0 0x7fe66410ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 -- 172.21.15.70:0/2616785293 >> 172.21.15.70:0/2616785293 conn(0x7fe6640fc200 msgr2=0x7fe6640fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 -- 172.21.15.70:0/2616785293 shutdown_connections 2023-10-18T12:31:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.302+0000 7fe66adff700 1 -- 172.21.15.70:0/2616785293 wait complete. 2023-10-18T12:31:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.303+0000 7fe66adff700 1 Processor -- start 2023-10-18T12:31:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 -- start start 2023-10-18T12:31:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 0x7fe664198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640734c0 0x7fe664199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe66410cfd0 0x7fe66419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419dcf0 con 0x7fe66410cfd0 2023-10-18T12:31:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419de60 con 0x7fe664072ba0 2023-10-18T12:31:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe66adff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419e020 con 0x7fe6640734c0 2023-10-18T12:31:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 0x7fe664198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 0x7fe664198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48966/0 (socket says 172.21.15.70:48966) 2023-10-18T12:31:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe668b9b700 1 -- 172.21.15.70:0/267858702 learned_addr learned my addr 172.21.15.70:0/267858702 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.304+0000 7fe65bfff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640734c0 0x7fe664199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe66939c700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe66410cfd0 0x7fe66419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe668b9b700 1 -- 172.21.15.70:0/267858702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640734c0 msgr2=0x7fe664199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe668b9b700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640734c0 0x7fe664199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe668b9b700 1 -- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe66410cfd0 msgr2=0x7fe66419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe668b9b700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe66410cfd0 0x7fe66419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe668b9b700 1 -- 172.21.15.70:0/267858702 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654009520 con 0x7fe664072ba0 2023-10-18T12:31:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe66939c700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe66410cfd0 0x7fe66419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe668b9b700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 0x7fe664198dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe660005910 tx=0x7fe660004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe659ffb700 1 -- 172.21.15.70:0/267858702 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66001d070 con 0x7fe664072ba0 2023-10-18T12:31:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.305+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6600095f0 con 0x7fe664072ba0 2023-10-18T12:31:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.306+0000 7fe659ffb700 1 -- 172.21.15.70:0/267858702 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe660022470 con 0x7fe664072ba0 2023-10-18T12:31:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.306+0000 7fe659ffb700 1 -- 172.21.15.70:0/267858702 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660017650 con 0x7fe664072ba0 2023-10-18T12:31:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.306+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6641a5960 con 0x7fe664072ba0 2023-10-18T12:31:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.309+0000 7fe659ffb700 1 -- 172.21.15.70:0/267858702 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe6600177b0 con 0x7fe664072ba0 2023-10-18T12:31:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.310+0000 7fe659ffb700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe64c0a15f0 0x7fe64c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.310+0000 7fe67e66ab80 1 Processor -- start 2023-10-18T12:31:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.310+0000 7fe659ffb700 1 -- 172.21.15.70:0/267858702 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe6600ce640 con 0x7fe664072ba0 2023-10-18T12:31:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 -- start start 2023-10-18T12:31:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe65bfff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe64c0a15f0 0x7fe64c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cacfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cad500 0x55ecf0ca7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0ca7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0c1a6e0 con 0x55ecf0cacbe0 2023-10-18T12:31:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0c00c60 con 0x55ecf0cad500 2023-10-18T12:31:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.311+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0bfe580 con 0x55ecf0ca75d0 2023-10-18T12:31:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe66939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0ca7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cacfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe66939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0ca7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58870/0 (socket says 172.21.15.70:58870) 2023-10-18T12:31:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cacfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34746/0 (socket says 172.21.15.70:34746) 2023-10-18T12:31:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe66939c700 1 -- 172.21.15.70:0/1146958192 learned_addr learned my addr 172.21.15.70:0/1146958192 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cad500 0x55ecf0ca7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe64c0a15f0 0x7fe64c0a3ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe66419a320 tx=0x7fe654009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 -- 172.21.15.70:0/1146958192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 msgr2=0x55ecf0ca7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0ca7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 -- 172.21.15.70:0/1146958192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 msgr2=0x55ecf0cacfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cacfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe65bfff700 1 -- 172.21.15.70:0/1146958192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf0bb73f0 con 0x55ecf0cad500 2023-10-18T12:31:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe668b9b700 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cacfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.312+0000 7fe66939c700 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0ca7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.313+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cad500 0x55ecf0ca7090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e4b0 tx=0x7fe654014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e388640890b26fa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.313+0000 7fe64affd700 1 -- 172.21.15.70:0/1146958192 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654026430 con 0x55ecf0cad500 2023-10-18T12:31:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.313+0000 7fe64affd700 1 -- 172.21.15.70:0/1146958192 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe654021070 con 0x55ecf0cad500 2023-10-18T12:31:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.313+0000 7fe64affd700 1 -- 172.21.15.70:0/1146958192 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654025650 con 0x55ecf0cad500 2023-10-18T12:31:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1146958192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cad500 msgr2=0x55ecf0ca7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cad500 0x55ecf0ca7090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e4b0 tx=0x7fe654014000 comp rx=0 tx=0).stop 2023-10-18T12:31:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1146958192 shutdown_connections 2023-10-18T12:31:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0ca7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cacfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1146958192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cad500 0x55ecf0ca7090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1146958192 >> 172.21.15.70:0/1146958192 conn(0x55ecf0c1b720 msgr2=0x55ecf0ca1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1146958192 shutdown_connections 2023-10-18T12:31:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.314+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1146958192 wait complete. 2023-10-18T12:31:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.315+0000 7fe67e66ab80 1 -- start start 2023-10-18T12:31:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.315+0000 7fe67e66ab80 1 Processor -- start 2023-10-18T12:31:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.315+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.315+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.315+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0cb0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0bf9e70 con 0x55ecf0cad500 2023-10-18T12:31:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0b96710 con 0x55ecf0ca75d0 2023-10-18T12:31:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0b963b0 con 0x55ecf0cacbe0 2023-10-18T12:31:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe67e66ab80 10 client.0 _ll_register_callbacks cb 0x55ecf0ae0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48982/0 (socket says 172.21.15.70:48982) 2023-10-18T12:31:51.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 -- 172.21.15.70:0/1453332866 learned_addr learned my addr 172.21.15.70:0/1453332866 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:51.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe66939c700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0cb0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 -- 172.21.15.70:0/1453332866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cacbe0 msgr2=0x55ecf0cb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 -- 172.21.15.70:0/1453332866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 msgr2=0x55ecf0cb0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0cb0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 -- 172.21.15.70:0/1453332866 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6641a5960 con 0x55ecf0ca75d0 2023-10-18T12:31:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe668b9b700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb34e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe66009b050 tx=0x7fe66009ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d99434c5bcb3ef server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.316+0000 7fe643fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe643fff700 1 -- 172.21.15.70:0/1453332866 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66009f4a0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe643fff700 1 -- 172.21.15.70:0/1453332866 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe6600a3650 con 0x55ecf0ca75d0 2023-10-18T12:31:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecf0bfeef0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe643fff700 1 -- 172.21.15.70:0/1453332866 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66009f120 con 0x55ecf0ca75d0 2023-10-18T12:31:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 10 client.26390 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fe654017070 con 0x55ecf0ca75d0 2023-10-18T12:31:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe643fff700 1 -- 172.21.15.70:0/1453332866 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecf0bfeef0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe643fff700 1 -- 172.21.15.70:0/1453332866 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe66009a8d0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 10 client.26390 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 10 client.26390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 20 client.26390 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0cc0b60 0x55ecf0cc2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 4 client.26390 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.317+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecf0c768b0 con 0x55ecf0cc0b60 2023-10-18T12:31:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.318+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0cc0b60 0x55ecf0cc2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.318+0000 7fe65bfff700 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0cc0b60 0x55ecf0cc2f80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc29f7c07e14c3e server_cookie=6b9af453e5d352e7 in_seq=0 out_seq=0 2023-10-18T12:31:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.318+0000 7fe643fff700 10 client.26390 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe643fff700 1 -- 172.21.15.70:0/1453332866 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ecf0c768b0 con 0x55ecf0cc0b60 2023-10-18T12:31:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe643fff700 10 client.26390 handle_command_reply: tid=0 2023-10-18T12:31:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 client.26390 shutdown 2023-10-18T12:31:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 msgr2=0x55ecf0cb34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb34e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe66009b050 tx=0x7fe66009ad30 comp rx=0 tx=0).stop 2023-10-18T12:31:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 shutdown_connections 2023-10-18T12:31:51.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0cc0b60 0x55ecf0cc2f80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0cb0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe643fff700 0 client.26390 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:51.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb34e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/1453332866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 >> 172.21.15.70:0/1453332866 conn(0x55ecf0c1b720 msgr2=0x55ecf0ca6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 shutdown_connections 2023-10-18T12:31:51.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 1 -- 172.21.15.70:0/1453332866 wait complete. 2023-10-18T12:31:51.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.319+0000 7fe67e66ab80 20 client.26390 trim_cache size 0 max 16384 2023-10-18T12:31:51.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.333+0000 7fe67e66ab80 1 Processor -- start 2023-10-18T12:31:51.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.333+0000 7fe67e66ab80 1 -- start start 2023-10-18T12:31:51.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0caadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cad500 0x55ecf0cb3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0a7a920 con 0x55ecf0cacbe0 2023-10-18T12:31:51.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0cb1dd0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0ca7460 con 0x55ecf0cad500 2023-10-18T12:31:51.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0caadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe65bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0caadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48984/0 (socket says 172.21.15.70:48984) 2023-10-18T12:31:51.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 -- 172.21.15.70:0/341240544 learned_addr learned my addr 172.21.15.70:0/341240544 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:51.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe66939c700 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cad500 0x55ecf0cb3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 -- 172.21.15.70:0/341240544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cad500 msgr2=0x55ecf0cb3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cad500 0x55ecf0cb3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 -- 172.21.15.70:0/341240544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 msgr2=0x55ecf0cb7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.334+0000 7fe668b9b700 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.335+0000 7fe668b9b700 1 -- 172.21.15.70:0/341240544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf0a24de0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.335+0000 7fe668b9b700 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0caadc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe660097dc0 tx=0x7fe660005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cfcdcd8661c581f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:51.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.335+0000 7fe64b7fe700 1 -- 172.21.15.70:0/341240544 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6600a3650 con 0x55ecf0ca75d0 2023-10-18T12:31:51.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.335+0000 7fe64b7fe700 1 -- 172.21.15.70:0/341240544 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe66009f4a0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.335+0000 7fe64b7fe700 1 -- 172.21.15.70:0/341240544 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66009f2e0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 -- 172.21.15.70:0/341240544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 msgr2=0x55ecf0caadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0caadc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe660097dc0 tx=0x7fe660005fd0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 -- 172.21.15.70:0/341240544 shutdown_connections 2023-10-18T12:31:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0cad500 0x55ecf0cb3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0ca75d0 0x55ecf0caadc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/341240544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 -- 172.21.15.70:0/341240544 >> 172.21.15.70:0/341240544 conn(0x55ecf0c1b720 msgr2=0x55ecf0caa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 -- 172.21.15.70:0/341240544 shutdown_connections 2023-10-18T12:31:51.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 -- 172.21.15.70:0/341240544 wait complete. 2023-10-18T12:31:51.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.336+0000 7fe67e66ab80 1 -- start start 2023-10-18T12:31:51.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 Processor -- start 2023-10-18T12:31:51.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0d13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0d13ed0 con 0x55ecf0cad500 2023-10-18T12:31:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0d14010 con 0x55ecf0cacbe0 2023-10-18T12:31:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0d141a0 con 0x55ecf0ca75d0 2023-10-18T12:31:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.337+0000 7fe67e66ab80 10 client.0 _ll_register_callbacks cb 0x55ecf0ae0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48998/0 (socket says 172.21.15.70:48998) 2023-10-18T12:31:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58896/0 (socket says 172.21.15.70:58896) 2023-10-18T12:31:51.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 -- 172.21.15.70:0/847569953 learned_addr learned my addr 172.21.15.70:0/847569953 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:51.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe66939c700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0d13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 -- 172.21.15.70:0/847569953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 msgr2=0x55ecf0cb6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 -- 172.21.15.70:0/847569953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 msgr2=0x55ecf0d13990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:51.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0d13990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 -- 172.21.15.70:0/847569953 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe654017070 con 0x55ecf0cacbe0 2023-10-18T12:31:51.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe668b9b700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:51.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe66939c700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0d13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:51.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe65bfff700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb6650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e4b0 tx=0x7fe654009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5983b474456b2998 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:51.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.338+0000 7fe643fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe643fff700 1 -- 172.21.15.70:0/847569953 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654033a90 con 0x55ecf0cacbe0 2023-10-18T12:31:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe643fff700 1 -- 172.21.15.70:0/847569953 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe654033bf0 con 0x55ecf0cacbe0 2023-10-18T12:31:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe654021070 con 0x55ecf0cacbe0 2023-10-18T12:31:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe643fff700 1 -- 172.21.15.70:0/847569953 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe654033db0 con 0x55ecf0cacbe0 2023-10-18T12:31:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe67e66ab80 10 client.26396 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ecf0a0dc60 con 0x55ecf0cacbe0 2023-10-18T12:31:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe643fff700 1 -- 172.21.15.70:0/847569953 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe654021070 con 0x55ecf0cacbe0 2023-10-18T12:31:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.339+0000 7fe643fff700 1 -- 172.21.15.70:0/847569953 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe654003680 con 0x55ecf0cacbe0 2023-10-18T12:31:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe67e66ab80 10 client.26396 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe67e66ab80 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe67e66ab80 20 client.26396 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0d159c0 0x55ecf0d17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe67e66ab80 4 client.26396 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:31:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ecf0bf2e50 con 0x55ecf0d159c0 2023-10-18T12:31:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe668b9b700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0d159c0 0x55ecf0d17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe643fff700 10 client.26396 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.340+0000 7fe668b9b700 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0d159c0 0x55ecf0d17de0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329e2f8a140e131c server_cookie=bda75b667628aedb in_seq=0 out_seq=0 2023-10-18T12:31:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe643fff700 1 -- 172.21.15.70:0/847569953 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ecf0bf2e50 con 0x55ecf0d159c0 2023-10-18T12:31:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe643fff700 10 client.26396 handle_command_reply: tid=0 2023-10-18T12:31:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 client.26396 shutdown 2023-10-18T12:31:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 msgr2=0x55ecf0cb6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb6650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e4b0 tx=0x7fe654009800 comp rx=0 tx=0).stop 2023-10-18T12:31:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 shutdown_connections 2023-10-18T12:31:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ecf0d159c0 0x55ecf0d17de0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ecf0cad500 0x55ecf0d13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe643fff700 0 client.26396 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ecf0ca75d0 0x55ecf0cb6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 --2- 172.21.15.70:0/847569953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecf0cacbe0 0x55ecf0cb6650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 >> 172.21.15.70:0/847569953 conn(0x55ecf0c1b720 msgr2=0x55ecf0c9b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 shutdown_connections 2023-10-18T12:31:51.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 1 -- 172.21.15.70:0/847569953 wait complete. 2023-10-18T12:31:51.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.342+0000 7fe67e66ab80 20 client.26396 trim_cache size 0 max 16384 2023-10-18T12:31:51.363 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:31:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe64c0a15f0 msgr2=0x7fe64c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe64c0a15f0 0x7fe64c0a3ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe66419a320 tx=0x7fe654009260 comp rx=0 tx=0).stop 2023-10-18T12:31:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 msgr2=0x7fe664198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 0x7fe664198dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe660005910 tx=0x7fe660004730 comp rx=0 tx=0).stop 2023-10-18T12:31:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 shutdown_connections 2023-10-18T12:31:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe66410cfd0 0x7fe66419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe64c0a15f0 0x7fe64c0a3ab0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe664072ba0 0x7fe664198dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 --2- 172.21.15.70:0/267858702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640734c0 0x7fe664199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 >> 172.21.15.70:0/267858702 conn(0x7fe6640fc200 msgr2=0x7fe6640fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:51.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 shutdown_connections 2023-10-18T12:31:51.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:51.343+0000 7fe66adff700 1 -- 172.21.15.70:0/267858702 wait complete. 2023-10-18T12:31:51.366 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:51 smithi070 ceph-mon[98639]: pgmap v489: 129 pgs: 129 active+clean; 374 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 232 op/s 2023-10-18T12:31:51.366 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:51 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1991212862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:31:51.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:51 smithi152 ceph-mon[134767]: pgmap v489: 129 pgs: 129 active+clean; 374 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 232 op/s 2023-10-18T12:31:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:51 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1991212862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:31:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:51 smithi186 ceph-mon[135253]: pgmap v489: 129 pgs: 129 active+clean; 374 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 232 op/s 2023-10-18T12:31:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:51 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1991212862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:31:51.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 627445 12.35 MB/sec execute 140 sec latency 14.967 ms 2023-10-18T12:31:52.368 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:31:52.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:31:52.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.577+0000 7fe453dff700 1 -- 172.21.15.70:0/2166713240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c106730 msgr2=0x7fe44c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.577+0000 7fe453dff700 1 --2- 172.21.15.70:0/2166713240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c106730 0x7fe44c106b10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe4480098f0 tx=0x7fe448009c00 comp rx=0 tx=0).stop 2023-10-18T12:31:52.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.577+0000 7fe453dff700 1 -- 172.21.15.70:0/2166713240 shutdown_connections 2023-10-18T12:31:52.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.577+0000 7fe453dff700 1 --2- 172.21.15.70:0/2166713240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 0x7fe44c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.577+0000 7fe453dff700 1 --2- 172.21.15.70:0/2166713240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c106730 0x7fe44c106b10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.578+0000 7fe453dff700 1 --2- 172.21.15.70:0/2166713240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe44c100720 0x7fe44c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.578+0000 7fe453dff700 1 -- 172.21.15.70:0/2166713240 >> 172.21.15.70:0/2166713240 conn(0x7fe44c0fc1b0 msgr2=0x7fe44c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.578+0000 7fe453dff700 1 -- 172.21.15.70:0/2166713240 shutdown_connections 2023-10-18T12:31:52.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.578+0000 7fe453dff700 1 -- 172.21.15.70:0/2166713240 wait complete. 2023-10-18T12:31:52.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 Processor -- start 2023-10-18T12:31:52.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 -- start start 2023-10-18T12:31:52.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 0x7fe44c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 0x7fe44c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe44c106730 0x7fe44c196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c1970f0 con 0x7fe44c100720 2023-10-18T12:31:52.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c197260 con 0x7fe44c106730 2023-10-18T12:31:52.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe453dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c197420 con 0x7fe44c1010e0 2023-10-18T12:31:52.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 0x7fe44c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 0x7fe44c19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34786/0 (socket says 172.21.15.70:34786) 2023-10-18T12:31:52.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe451b9b700 1 -- 172.21.15.70:0/1176023016 learned_addr learned my addr 172.21.15.70:0/1176023016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe45239c700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe44c106730 0x7fe44c196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.579+0000 7fe45139a700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 0x7fe44c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe451b9b700 1 -- 172.21.15.70:0/1176023016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 msgr2=0x7fe44c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe451b9b700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 0x7fe44c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe451b9b700 1 -- 172.21.15.70:0/1176023016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe44c106730 msgr2=0x7fe44c196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe451b9b700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe44c106730 0x7fe44c196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe451b9b700 1 -- 172.21.15.70:0/1176023016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0095f0 con 0x7fe44c100720 2023-10-18T12:31:52.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe45139a700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 0x7fe44c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:31:52.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe451b9b700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 0x7fe44c19c960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe448005950 tx=0x7fe44800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe442ffd700 1 -- 172.21.15.70:0/1176023016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44801c070 con 0x7fe44c100720 2023-10-18T12:31:52.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe442ffd700 1 -- 172.21.15.70:0/1176023016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe44800bbd0 con 0x7fe44c100720 2023-10-18T12:31:52.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe448009520 con 0x7fe44c100720 2023-10-18T12:31:52.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe442ffd700 1 -- 172.21.15.70:0/1176023016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448003a00 con 0x7fe44c100720 2023-10-18T12:31:52.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.580+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c1a3770 con 0x7fe44c100720 2023-10-18T12:31:52.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.583+0000 7fe442ffd700 1 -- 172.21.15.70:0/1176023016 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe448020030 con 0x7fe44c100720 2023-10-18T12:31:52.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe442ffd700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe4380a15f0 0x7fe4380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 Processor -- start 2023-10-18T12:31:52.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe442ffd700 1 -- 172.21.15.70:0/1176023016 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe4480ce2d0 con 0x7fe44c100720 2023-10-18T12:31:52.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 -- start start 2023-10-18T12:31:52.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe45139a700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe4380a15f0 0x7fe4380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb43f760 con 0x55bcdb4d6990 2023-10-18T12:31:52.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb425ce0 con 0x55bcdb4d0a60 2023-10-18T12:31:52.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.585+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb423600 con 0x55bcdb4d6070 2023-10-18T12:31:52.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49028/0 (socket says 172.21.15.70:49028) 2023-10-18T12:31:52.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 -- 172.21.15.70:0/2496344958 learned_addr learned my addr 172.21.15.70:0/2496344958 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45139a700 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45139a700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe4380a15f0 0x7fe4380a3ab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe44c197c50 tx=0x7fe43c009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 -- 172.21.15.70:0/2496344958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 msgr2=0x55bcdb4d6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 -- 172.21.15.70:0/2496344958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 msgr2=0x55bcdb4d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 -- 172.21.15.70:0/2496344958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb3dc470 con 0x55bcdb4d0a60 2023-10-18T12:31:52.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.586+0000 7fe45239c700 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d0ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe444003f30 tx=0x7fe444009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc48ba46446f0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe437fff700 1 -- 172.21.15.70:0/2496344958 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444003a00 con 0x55bcdb4d0a60 2023-10-18T12:31:52.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe437fff700 1 -- 172.21.15.70:0/2496344958 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe44400f040 con 0x55bcdb4d0a60 2023-10-18T12:31:52.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe437fff700 1 -- 172.21.15.70:0/2496344958 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444015680 con 0x55bcdb4d0a60 2023-10-18T12:31:52.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe4677d6b80 1 -- 172.21.15.70:0/2496344958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 msgr2=0x55bcdb4d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d0ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe444003f30 tx=0x7fe444009c20 comp rx=0 tx=0).stop 2023-10-18T12:31:52.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe4677d6b80 1 -- 172.21.15.70:0/2496344958 shutdown_connections 2023-10-18T12:31:52.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d0ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.587+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.588+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/2496344958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.588+0000 7fe4677d6b80 1 -- 172.21.15.70:0/2496344958 >> 172.21.15.70:0/2496344958 conn(0x55bcdb4407a0 msgr2=0x55bcdb4c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.588+0000 7fe4677d6b80 1 -- 172.21.15.70:0/2496344958 shutdown_connections 2023-10-18T12:31:52.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.588+0000 7fe4677d6b80 1 -- 172.21.15.70:0/2496344958 wait complete. 2023-10-18T12:31:52.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.588+0000 7fe4677d6b80 1 -- start start 2023-10-18T12:31:52.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.588+0000 7fe4677d6b80 1 Processor -- start 2023-10-18T12:31:52.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6990 0x55bcdb4da190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb41eef0 con 0x55bcdb4d6070 2023-10-18T12:31:52.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb3bb790 con 0x55bcdb4d0a60 2023-10-18T12:31:52.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb3bb430 con 0x55bcdb4d6990 2023-10-18T12:31:52.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe4677d6b80 10 client.0 _ll_register_callbacks cb 0x55bcdb305b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:52.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe45139a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4dc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49044/0 (socket says 172.21.15.70:49044) 2023-10-18T12:31:52.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 -- 172.21.15.70:0/3695952895 learned_addr learned my addr 172.21.15.70:0/3695952895 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe45239c700 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6990 0x55bcdb4da190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 -- 172.21.15.70:0/3695952895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6990 msgr2=0x55bcdb4da190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6990 0x55bcdb4da190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 -- 172.21.15.70:0/3695952895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6070 msgr2=0x55bcdb4d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.589+0000 7fe451b9b700 1 -- 172.21.15.70:0/3695952895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe444003b60 con 0x55bcdb4d0a60 2023-10-18T12:31:52.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe451b9b700 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4dc890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe448009fd0 tx=0x7fe44809ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7941051952694a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4357fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:52.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4357fa700 1 -- 172.21.15.70:0/3695952895 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44809f130 con 0x55bcdb4d0a60 2023-10-18T12:31:52.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcdb423f70 con 0x55bcdb4d0a60 2023-10-18T12:31:52.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4357fa700 1 -- 172.21.15.70:0/3695952895 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe4480a32e0 con 0x55bcdb4d0a60 2023-10-18T12:31:52.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4357fa700 1 -- 172.21.15.70:0/3695952895 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4480b5e70 con 0x55bcdb4d0a60 2023-10-18T12:31:52.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4677d6b80 10 client.26408 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:52.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bcdb249e60 con 0x55bcdb4d0a60 2023-10-18T12:31:52.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.590+0000 7fe4357fa700 1 -- 172.21.15.70:0/3695952895 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4480b9030 con 0x55bcdb4d0a60 2023-10-18T12:31:52.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4357fa700 1 -- 172.21.15.70:0/3695952895 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe4480052b0 con 0x55bcdb4d0a60 2023-10-18T12:31:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4677d6b80 10 client.26408 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4677d6b80 10 client.26408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4677d6b80 20 client.26408 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb4e9e10 0x55bcdb4ec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4677d6b80 4 client.26408 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcdb49b930 con 0x55bcdb4e9e10 2023-10-18T12:31:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe45139a700 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb4e9e10 0x55bcdb4ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe45139a700 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb4e9e10 0x55bcdb4ec230 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c526b6d7cff0a1 server_cookie=8cb73531ed305256 in_seq=0 out_seq=0 2023-10-18T12:31:52.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.591+0000 7fe4357fa700 10 client.26408 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4357fa700 1 -- 172.21.15.70:0/3695952895 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bcdb49b930 con 0x55bcdb4e9e10 2023-10-18T12:31:52.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4357fa700 10 client.26408 handle_command_reply: tid=0 2023-10-18T12:31:52.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 client.26408 shutdown 2023-10-18T12:31:52.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 msgr2=0x55bcdb4dc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4dc890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe448009fd0 tx=0x7fe44809ac50 comp rx=0 tx=0).stop 2023-10-18T12:31:52.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 shutdown_connections 2023-10-18T12:31:52.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb4e9e10 0x55bcdb4ec230 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6990 0x55bcdb4da190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4357fa700 0 client.26408 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4dc890 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/3695952895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6070 0x55bcdb4d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 >> 172.21.15.70:0/3695952895 conn(0x55bcdb4407a0 msgr2=0x55bcdb4cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 shutdown_connections 2023-10-18T12:31:52.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.592+0000 7fe4677d6b80 1 -- 172.21.15.70:0/3695952895 wait complete. 2023-10-18T12:31:52.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.593+0000 7fe4677d6b80 20 client.26408 trim_cache size 0 max 16384 2023-10-18T12:31:52.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.611+0000 7fe4677d6b80 1 Processor -- start 2023-10-18T12:31:52.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 -- start start 2023-10-18T12:31:52.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb29f9a0 con 0x55bcdb4d6990 2023-10-18T12:31:52.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb4da770 con 0x55bcdb4d0a60 2023-10-18T12:31:52.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb4c76d0 con 0x55bcdb4d6070 2023-10-18T12:31:52.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe45239c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe45139a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49060/0 (socket says 172.21.15.70:49060) 2023-10-18T12:31:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe451b9b700 1 -- 172.21.15.70:0/1572993696 learned_addr learned my addr 172.21.15.70:0/1572993696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.612+0000 7fe45239c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34798/0 (socket says 172.21.15.70:34798) 2023-10-18T12:31:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe451b9b700 1 -- 172.21.15.70:0/1572993696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 msgr2=0x55bcdb4e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe451b9b700 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe451b9b700 1 -- 172.21.15.70:0/1572993696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 msgr2=0x55bcdb4d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe451b9b700 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe451b9b700 1 -- 172.21.15.70:0/1572993696 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb232ce0 con 0x55bcdb4d0a60 2023-10-18T12:31:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe45139a700 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe45239c700 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:52.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe451b9b700 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d9b20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe44809af70 tx=0x7fe448005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e40db4faef963bf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe440bff700 1 -- 172.21.15.70:0/1572993696 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480a32e0 con 0x55bcdb4d0a60 2023-10-18T12:31:52.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe440bff700 1 -- 172.21.15.70:0/1572993696 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4480b9040 con 0x55bcdb4d0a60 2023-10-18T12:31:52.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.613+0000 7fe440bff700 1 -- 172.21.15.70:0/1572993696 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480cb940 con 0x55bcdb4d0a60 2023-10-18T12:31:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 -- 172.21.15.70:0/1572993696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 msgr2=0x55bcdb4d9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d9b20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe44809af70 tx=0x7fe448005650 comp rx=0 tx=0).stop 2023-10-18T12:31:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 -- 172.21.15.70:0/1572993696 shutdown_connections 2023-10-18T12:31:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d6990 0x55bcdb4d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d0a60 0x55bcdb4d9b20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/1572993696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb4e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 -- 172.21.15.70:0/1572993696 >> 172.21.15.70:0/1572993696 conn(0x55bcdb4407a0 msgr2=0x55bcdb4d9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 -- 172.21.15.70:0/1572993696 shutdown_connections 2023-10-18T12:31:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 -- 172.21.15.70:0/1572993696 wait complete. 2023-10-18T12:31:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 -- start start 2023-10-18T12:31:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.614+0000 7fe4677d6b80 1 Processor -- start 2023-10-18T12:31:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d0a60 0x55bcdb538240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb538780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 0x55bcdb4e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb4e0bf0 con 0x55bcdb4d0a60 2023-10-18T12:31:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb4e0d60 con 0x55bcdb4d6990 2023-10-18T12:31:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb4e0f20 con 0x55bcdb4d6070 2023-10-18T12:31:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe4677d6b80 10 client.0 _ll_register_callbacks cb 0x55bcdb305b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe45239c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 0x55bcdb4e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe451b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d0a60 0x55bcdb538240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe45239c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 0x55bcdb4e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49066/0 (socket says 172.21.15.70:49066) 2023-10-18T12:31:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe45239c700 1 -- 172.21.15.70:0/333518990 learned_addr learned my addr 172.21.15.70:0/333518990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.615+0000 7fe45139a700 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb538780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe45239c700 1 -- 172.21.15.70:0/333518990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 msgr2=0x55bcdb538780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe45239c700 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb538780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe45239c700 1 -- 172.21.15.70:0/333518990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d0a60 msgr2=0x55bcdb538240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe45239c700 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d0a60 0x55bcdb538240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe45239c700 1 -- 172.21.15.70:0/333518990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c0125d0 con 0x55bcdb4d6990 2023-10-18T12:31:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe45239c700 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 0x55bcdb4e05d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe444007d50 tx=0x7fe444004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a23eb46efc8ba95 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4357fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4357fa700 1 -- 172.21.15.70:0/333518990 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444025070 con 0x55bcdb4d6990 2023-10-18T12:31:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4357fa700 1 -- 172.21.15.70:0/333518990 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe444004700 con 0x55bcdb4d6990 2023-10-18T12:31:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe44400f040 con 0x55bcdb4d6990 2023-10-18T12:31:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4357fa700 1 -- 172.21.15.70:0/333518990 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe444007500 con 0x55bcdb4d6990 2023-10-18T12:31:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4677d6b80 10 client.26414 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.616+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bcdb3dc470 con 0x55bcdb4d6990 2023-10-18T12:31:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4357fa700 1 -- 172.21.15.70:0/333518990 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe44400f040 con 0x55bcdb4d6990 2023-10-18T12:31:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4357fa700 1 -- 172.21.15.70:0/333518990 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe444015a40 con 0x55bcdb4d6990 2023-10-18T12:31:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4677d6b80 10 client.26414 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4677d6b80 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4677d6b80 20 client.26414 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb540120 0x55bcdb542540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4677d6b80 4 client.26414 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:31:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bcdb417ed0 con 0x55bcdb540120 2023-10-18T12:31:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.617+0000 7fe451b9b700 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb540120 0x55bcdb542540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.618+0000 7fe451b9b700 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb540120 0x55bcdb542540 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c7ddec28583df6 server_cookie=a29de0304b5e11de in_seq=0 out_seq=0 2023-10-18T12:31:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.618+0000 7fe4357fa700 10 client.26414 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.619+0000 7fe4357fa700 1 -- 172.21.15.70:0/333518990 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bcdb417ed0 con 0x55bcdb540120 2023-10-18T12:31:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.619+0000 7fe4357fa700 10 client.26414 handle_command_reply: tid=0 2023-10-18T12:31:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.619+0000 7fe4677d6b80 1 client.26414 shutdown 2023-10-18T12:31:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.619+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 msgr2=0x55bcdb4e05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.619+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 0x55bcdb4e05d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe444007d50 tx=0x7fe444004f90 comp rx=0 tx=0).stop 2023-10-18T12:31:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 shutdown_connections 2023-10-18T12:31:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bcdb540120 0x55bcdb542540 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcdb4d6990 0x55bcdb4e05d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bcdb4d0a60 0x55bcdb538240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4357fa700 0 client.26414 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 --2- 172.21.15.70:0/333518990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bcdb4d6070 0x55bcdb538780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 >> 172.21.15.70:0/333518990 conn(0x55bcdb4407a0 msgr2=0x55bcdb4d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 shutdown_connections 2023-10-18T12:31:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 1 -- 172.21.15.70:0/333518990 wait complete. 2023-10-18T12:31:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe4677d6b80 20 client.26414 trim_cache size 0 max 16384 2023-10-18T12:31:52.636 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:31:52.636 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:31:52.636 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "aadedcdd-acc6-4215-9dec-7b70f1025b27", 2023-10-18T12:31:52.637 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:31:52.637 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:31:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe4380a15f0 msgr2=0x7fe4380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe453dff700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe4380a15f0 0x7fe4380a3ab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe44c197c50 tx=0x7fe43c009310 comp rx=0 tx=0).stop 2023-10-18T12:31:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 msgr2=0x7fe44c19c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.620+0000 7fe453dff700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 0x7fe44c19c960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe448005950 tx=0x7fe44800fa00 comp rx=0 tx=0).stop 2023-10-18T12:31:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 shutdown_connections 2023-10-18T12:31:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe4380a15f0 0x7fe4380a3ab0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe44c106730 0x7fe44c196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe44c100720 0x7fe44c19c960 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 --2- 172.21.15.70:0/1176023016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe44c1010e0 0x7fe44c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 >> 172.21.15.70:0/1176023016 conn(0x7fe44c0fc1b0 msgr2=0x7fe44c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 shutdown_connections 2023-10-18T12:31:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.621+0000 7fe453dff700 1 -- 172.21.15.70:0/1176023016 wait complete. 2023-10-18T12:31:52.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:31:52.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 629417 12.36 MB/sec execute 141 sec latency 21.235 ms 2023-10-18T12:31:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 -- 172.21.15.70:0/1183781550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c106660 msgr2=0x7f071c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 --2- 172.21.15.70:0/1183781550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c106660 0x7f071c106a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0710009990 tx=0x7f0710009ca0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 -- 172.21.15.70:0/1183781550 shutdown_connections 2023-10-18T12:31:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 --2- 172.21.15.70:0/1183781550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 0x7f071c10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 --2- 172.21.15.70:0/1183781550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c106660 0x7f071c106a40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 --2- 172.21.15.70:0/1183781550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f071c1005c0 0x7f071c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 -- 172.21.15.70:0/1183781550 >> 172.21.15.70:0/1183781550 conn(0x7f071c0fc0f0 msgr2=0x7f071c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 -- 172.21.15.70:0/1183781550 shutdown_connections 2023-10-18T12:31:52.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.866+0000 7f0721d9d700 1 -- 172.21.15.70:0/1183781550 wait complete. 2023-10-18T12:31:52.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 Processor -- start 2023-10-18T12:31:52.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 -- start start 2023-10-18T12:31:52.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 0x7f071c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 0x7f071c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f071c106660 0x7f071c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1990c0 con 0x7f071c106660 2023-10-18T12:31:52.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c199230 con 0x7f071c1005c0 2023-10-18T12:31:52.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.867+0000 7f0721d9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1993f0 con 0x7f071c100f80 2023-10-18T12:31:52.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f071bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 0x7f071c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 0x7f071c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f072159c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f071c106660 0x7f071c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 0x7f071c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49086/0 (socket says 172.21.15.70:49086) 2023-10-18T12:31:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f071bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 0x7f071c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:58978/0 (socket says 172.21.15.70:58978) 2023-10-18T12:31:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 -- 172.21.15.70:0/4214767169 learned_addr learned my addr 172.21.15.70:0/4214767169 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 -- 172.21.15.70:0/4214767169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 msgr2=0x7f071c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 0x7f071c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 -- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f071c106660 msgr2=0x7f071c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f071c106660 0x7f071c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 -- 172.21.15.70:0/4214767169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100095f0 con 0x7f071c1005c0 2023-10-18T12:31:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.868+0000 7f0720d9b700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 0x7f071c19e930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0710000c00 tx=0x7f0710004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.869+0000 7f0719ffb700 1 -- 172.21.15.70:0/4214767169 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071001d070 con 0x7f071c1005c0 2023-10-18T12:31:52.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.869+0000 7f0719ffb700 1 -- 172.21.15.70:0/4214767169 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0710021410 con 0x7f071c1005c0 2023-10-18T12:31:52.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.869+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c199670 con 0x7f071c1005c0 2023-10-18T12:31:52.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.869+0000 7f0719ffb700 1 -- 172.21.15.70:0/4214767169 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710017650 con 0x7f071c1005c0 2023-10-18T12:31:52.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.869+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c1a5920 con 0x7f071c1005c0 2023-10-18T12:31:52.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.872+0000 7f0719ffb700 1 -- 172.21.15.70:0/4214767169 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0710021580 con 0x7f071c1005c0 2023-10-18T12:31:52.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.873+0000 7f0736a14b80 1 Processor -- start 2023-10-18T12:31:52.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.873+0000 7f0719ffb700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f070c0a15f0 0x7f070c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.873+0000 7f0736a14b80 1 -- start start 2023-10-18T12:31:52.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.873+0000 7f0719ffb700 1 -- 172.21.15.70:0/4214767169 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f07100ce630 con 0x7f071c1005c0 2023-10-18T12:31:52.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.873+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357607140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f072159c700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f070c0a15f0 0x7f070c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357607680 0x5643576012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357601820 0x564357601c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0736a14b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643575746e0 con 0x564357601820 2023-10-18T12:31:52.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0736a14b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435755ac60 con 0x564357606d60 2023-10-18T12:31:52.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0736a14b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564357558580 con 0x564357607680 2023-10-18T12:31:52.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357607140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357607140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49098/0 (socket says 172.21.15.70:49098) 2023-10-18T12:31:52.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 -- 172.21.15.70:0/1267778740 learned_addr learned my addr 172.21.15.70:0/1267778740 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f072159c700 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357601820 0x564357601c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f071bfff700 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357607680 0x5643576012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 -- 172.21.15.70:0/1267778740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357607680 msgr2=0x5643576012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357607680 0x5643576012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 -- 172.21.15.70:0/1267778740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357601820 msgr2=0x564357601c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357601820 0x564357601c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f0720d9b700 1 -- 172.21.15.70:0/1267778740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643575113f0 con 0x564357606d60 2023-10-18T12:31:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.874+0000 7f072159c700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f070c0a15f0 0x7f070c0a3ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ec0 tx=0x7f0714016450 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0720d9b700 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357607140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f071000f740 tx=0x7f07100977a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2b9fef2b754156 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0706ffd700 1 -- 172.21.15.70:0/1267778740 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071009ba40 con 0x564357606d60 2023-10-18T12:31:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0706ffd700 1 -- 172.21.15.70:0/1267778740 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071009b340 con 0x564357606d60 2023-10-18T12:31:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0706ffd700 1 -- 172.21.15.70:0/1267778740 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071009f130 con 0x564357606d60 2023-10-18T12:31:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 -- 172.21.15.70:0/1267778740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 msgr2=0x564357607140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357607140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f071000f740 tx=0x7f07100977a0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 -- 172.21.15.70:0/1267778740 shutdown_connections 2023-10-18T12:31:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357601820 0x564357601c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357607140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 --2- 172.21.15.70:0/1267778740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357607680 0x5643576012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.875+0000 7f0736a14b80 1 -- 172.21.15.70:0/1267778740 >> 172.21.15.70:0/1267778740 conn(0x564357575720 msgr2=0x5643575fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.876+0000 7f0736a14b80 1 -- 172.21.15.70:0/1267778740 shutdown_connections 2023-10-18T12:31:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.876+0000 7f0736a14b80 1 -- 172.21.15.70:0/1267778740 wait complete. 2023-10-18T12:31:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.876+0000 7f0736a14b80 1 -- start start 2023-10-18T12:31:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.876+0000 7f0736a14b80 1 Processor -- start 2023-10-18T12:31:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 0x56435760d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357606d60 0x56435760aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x56435760b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564357553e70 con 0x564357607680 2023-10-18T12:31:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643574f0710 con 0x564357601820 2023-10-18T12:31:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643574f03b0 con 0x564357606d60 2023-10-18T12:31:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0736a14b80 10 client.0 _ll_register_callbacks cb 0x56435743aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 0x56435760d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 0x56435760d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49100/0 (socket says 172.21.15.70:49100) 2023-10-18T12:31:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 -- 172.21.15.70:0/3862254957 learned_addr learned my addr 172.21.15.70:0/3862254957 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f071bfff700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357606d60 0x56435760aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f072159c700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x56435760b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 -- 172.21.15.70:0/3862254957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357606d60 msgr2=0x56435760aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357606d60 0x56435760aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 -- 172.21.15.70:0/3862254957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 msgr2=0x56435760b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x56435760b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.877+0000 7f0720d9b700 1 -- 172.21.15.70:0/3862254957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c1a5920 con 0x564357601820 2023-10-18T12:31:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f0720d9b700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 0x56435760d6c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f07100970c0 tx=0x7f071000bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ce9bec1f849333 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f06fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f06fbfff700 1 -- 172.21.15.70:0/3862254957 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071009f490 con 0x564357601820 2023-10-18T12:31:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564357558ef0 con 0x564357601820 2023-10-18T12:31:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f06fbfff700 1 -- 172.21.15.70:0/3862254957 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f07100a3640 con 0x564357601820 2023-10-18T12:31:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f06fbfff700 1 -- 172.21.15.70:0/3862254957 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f071009f130 con 0x564357601820 2023-10-18T12:31:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f0736a14b80 10 client.26426 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56435737ede0 con 0x564357601820 2023-10-18T12:31:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.878+0000 7f06fbfff700 1 -- 172.21.15.70:0/3862254957 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564357558ef0 con 0x564357601820 2023-10-18T12:31:52.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f06fbfff700 1 -- 172.21.15.70:0/3862254957 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f071007d070 con 0x564357601820 2023-10-18T12:31:52.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f0736a14b80 10 client.26426 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:52.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f0736a14b80 10 client.26426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:52.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f0736a14b80 20 client.26426 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:52.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f0736a14b80 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56435761a960 0x56435761cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f0736a14b80 4 client.26426 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:31:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643575d08b0 con 0x56435761a960 2023-10-18T12:31:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f071bfff700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56435761a960 0x56435761cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f06fbfff700 10 client.26426 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.879+0000 7f071bfff700 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56435761a960 0x56435761cd80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fbafb5d9ac5dde server_cookie=aa81bd852d7e9388 in_seq=0 out_seq=0 2023-10-18T12:31:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f06fbfff700 1 -- 172.21.15.70:0/3862254957 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5643575d08b0 con 0x56435761a960 2023-10-18T12:31:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f06fbfff700 10 client.26426 handle_command_reply: tid=0 2023-10-18T12:31:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 client.26426 shutdown 2023-10-18T12:31:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 msgr2=0x56435760d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 0x56435760d6c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f07100970c0 tx=0x7f071000bfb0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 shutdown_connections 2023-10-18T12:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56435761a960 0x56435761cd80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x56435760b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f06fbfff700 0 client.26426 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357601820 0x56435760d6c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 --2- 172.21.15.70:0/3862254957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357606d60 0x56435760aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 >> 172.21.15.70:0/3862254957 conn(0x564357575720 msgr2=0x5643576002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.880+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 shutdown_connections 2023-10-18T12:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.881+0000 7f0736a14b80 1 -- 172.21.15.70:0/3862254957 wait complete. 2023-10-18T12:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.881+0000 7f0736a14b80 20 client.26426 trim_cache size 0 max 16384 2023-10-18T12:31:52.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.923+0000 7f0736a14b80 1 Processor -- start 2023-10-18T12:31:52.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.923+0000 7f0736a14b80 1 -- start start 2023-10-18T12:31:52.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x5643575f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357611db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x5643576002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f0736a14b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643573d4920 con 0x564357607680 2023-10-18T12:31:52.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f0736a14b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643575fcc40 con 0x564357606d60 2023-10-18T12:31:52.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f0736a14b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564357612370 con 0x564357601820 2023-10-18T12:31:52.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.924+0000 7f072159c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x5643576002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.925+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x5643575f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.925+0000 7f0720d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x5643575f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:59012/0 (socket says 172.21.15.70:59012) 2023-10-18T12:31:52.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.925+0000 7f0720d9b700 1 -- 172.21.15.70:0/4115723798 learned_addr learned my addr 172.21.15.70:0/4115723798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.925+0000 7f0720d9b700 1 -- 172.21.15.70:0/4115723798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 msgr2=0x564357611db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T12:31:52.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.925+0000 7f0720d9b700 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357611db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.925+0000 7f0720d9b700 1 -- 172.21.15.70:0/4115723798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 msgr2=0x5643576002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:31:52.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.926+0000 7f0720d9b700 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x5643576002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.926+0000 7f0720d9b700 1 -- 172.21.15.70:0/4115723798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564357367c60 con 0x564357601820 2023-10-18T12:31:52.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.926+0000 7f072159c700 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x5643576002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:31:52.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.927+0000 7f0720d9b700 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x5643575f6820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f071000b2e0 tx=0x7f07100cbf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b5359dced522b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.928+0000 7f07077fe700 1 -- 172.21.15.70:0/4115723798 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071009ba40 con 0x564357601820 2023-10-18T12:31:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.929+0000 7f07077fe700 1 -- 172.21.15.70:0/4115723798 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071009b110 con 0x564357601820 2023-10-18T12:31:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.929+0000 7f07077fe700 1 -- 172.21.15.70:0/4115723798 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071009f130 con 0x564357601820 2023-10-18T12:31:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 -- 172.21.15.70:0/4115723798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 msgr2=0x5643575f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x5643575f6820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f071000b2e0 tx=0x7f07100cbf80 comp rx=0 tx=0).stop 2023-10-18T12:31:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 -- 172.21.15.70:0/4115723798 shutdown_connections 2023-10-18T12:31:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357607680 0x5643576002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x5643575f6820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 --2- 172.21.15.70:0/4115723798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357606d60 0x564357611db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.930+0000 7f0736a14b80 1 -- 172.21.15.70:0/4115723798 >> 172.21.15.70:0/4115723798 conn(0x564357575720 msgr2=0x564357601c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.931+0000 7f0736a14b80 1 -- 172.21.15.70:0/4115723798 shutdown_connections 2023-10-18T12:31:52.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.931+0000 7f0736a14b80 1 -- 172.21.15.70:0/4115723798 wait complete. 2023-10-18T12:31:52.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.931+0000 7f0736a14b80 1 -- start start 2023-10-18T12:31:52.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.932+0000 7f0736a14b80 1 Processor -- start 2023-10-18T12:31:52.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.934+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x564357611b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.935+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357606d60 0x56435760ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.935+0000 7f0736a14b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 0x5643576104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.936+0000 7f072159c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 0x5643576104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.936+0000 7f072159c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 0x5643576104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:49124/0 (socket says 172.21.15.70:49124) 2023-10-18T12:31:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.936+0000 7f072159c700 1 -- 172.21.15.70:0/256572800 learned_addr learned my addr 172.21.15.70:0/256572800 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:31:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.936+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564357610af0 con 0x564357606d60 2023-10-18T12:31:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.936+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564357610c30 con 0x564357607680 2023-10-18T12:31:52.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.936+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564357610df0 con 0x564357601820 2023-10-18T12:31:52.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.937+0000 7f072159c700 1 -- 172.21.15.70:0/256572800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 msgr2=0x564357611b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T12:31:52.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.937+0000 7f072159c700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x564357611b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.937+0000 7f072159c700 1 -- 172.21.15.70:0/256572800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357606d60 msgr2=0x56435760ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T12:31:52.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.937+0000 7f0736a14b80 10 client.0 _ll_register_callbacks cb 0x56435743aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:31:52.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.937+0000 7f071bfff700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357606d60 0x56435760ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.938+0000 7f072159c700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357606d60 0x56435760ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.938+0000 7f072159c700 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071401abe0 con 0x564357607680 2023-10-18T12:31:52.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.938+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f071009b340 con 0x564357607680 2023-10-18T12:31:52.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.938+0000 7f071bfff700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357606d60 0x56435760ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:31:52.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.939+0000 7f072159c700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 0x5643576104d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0714016390 tx=0x7f0714004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f496ef97f283a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:31:52.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.939+0000 7f06fbfff700 10 client.26429 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:31:52.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.939+0000 7f06fbfff700 1 -- 172.21.15.70:0/256572800 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071400e040 con 0x564357607680 2023-10-18T12:31:52.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.940+0000 7f06fbfff700 1 -- 172.21.15.70:0/256572800 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0714021040 con 0x564357607680 2023-10-18T12:31:52.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.940+0000 7f06fbfff700 1 -- 172.21.15.70:0/256572800 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0714028480 con 0x564357607680 2023-10-18T12:31:52.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.940+0000 7f0736a14b80 10 client.26429 fetch_fsmap learned FSMap version 20 2023-10-18T12:31:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.940+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({fsmap=20}) v3 -- 0x56435737ede0 con 0x564357607680 2023-10-18T12:31:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.941+0000 7f06fbfff700 1 -- 172.21.15.70:0/256572800 <== mon.2 v2:172.21.15.152:3300/0 4 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07140285e0 con 0x564357607680 2023-10-18T12:31:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0736a14b80 10 client.26429 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:31:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0736a14b80 10 client.26429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:31:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0736a14b80 20 client.26429 populate_metadata read hostname 'smithi070' 2023-10-18T12:31:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0736a14b80 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5643576783d0 0x56435767a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:31:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0736a14b80 4 client.26429 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:31:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56435754ce50 con 0x5643576783d0 2023-10-18T12:31:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.942+0000 7f0720d9b700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5643576783d0 0x56435767a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:31:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.944+0000 7f0720d9b700 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5643576783d0 0x56435767a850 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676621eea6e48cbf server_cookie=1116af048c1e37fb in_seq=0 out_seq=0 2023-10-18T12:31:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.944+0000 7f06fbfff700 10 client.26429 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.945+0000 7f06fbfff700 1 -- 172.21.15.70:0/256572800 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56435754ce50 con 0x5643576783d0 2023-10-18T12:31:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.945+0000 7f06fbfff700 10 client.26429 handle_command_reply: tid=0 2023-10-18T12:31:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.945+0000 7f0736a14b80 1 client.26429 shutdown 2023-10-18T12:31:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 msgr2=0x5643576104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 0x5643576104d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0714016390 tx=0x7f0714004b40 comp rx=0 tx=0).stop 2023-10-18T12:31:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 shutdown_connections 2023-10-18T12:31:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5643576783d0 0x56435767a850 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564357607680 0x5643576104d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564357601820 0x564357611b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 --2- 172.21.15.70:0/256572800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564357606d60 0x56435760ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f06fbfff700 0 client.26429 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:31:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 >> 172.21.15.70:0/256572800 conn(0x564357575720 msgr2=0x5643576048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 shutdown_connections 2023-10-18T12:31:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.946+0000 7f0736a14b80 1 -- 172.21.15.70:0/256572800 wait complete. 2023-10-18T12:31:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.947+0000 7f0736a14b80 20 client.26429 trim_cache size 0 max 16384 2023-10-18T12:31:52.952 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:31:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:31:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:31:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "aadedcdd-acc6-4215-9dec-7b70f1025b27": { 2023-10-18T12:31:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:31:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "aadedcdd-acc6-4215-9dec-7b70f1025b27", 2023-10-18T12:31:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:31:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:31:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:31:52.954 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:31:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f070c0a15f0 msgr2=0x7f070c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f070c0a15f0 0x7f070c0a3ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ec0 tx=0x7f0714016450 comp rx=0 tx=0).stop 2023-10-18T12:31:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 msgr2=0x7f071c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:31:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 0x7f071c19e930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0710000c00 tx=0x7f0710004730 comp rx=0 tx=0).stop 2023-10-18T12:31:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 shutdown_connections 2023-10-18T12:31:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f071c106660 0x7f071c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f070c0a15f0 0x7f070c0a3ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f071c1005c0 0x7f071c19e930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 --2- 172.21.15.70:0/4214767169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f071c100f80 0x7f071c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:31:52.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 >> 172.21.15.70:0/4214767169 conn(0x7f071c0fc0f0 msgr2=0x7f071c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:31:52.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 shutdown_connections 2023-10-18T12:31:52.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:31:52.949+0000 7f0721d9d700 1 -- 172.21.15.70:0/4214767169 wait complete. 2023-10-18T12:31:52.984 INFO:tasks.cephfs.filesystem:scrub status for tag:aadedcdd-acc6-4215-9dec-7b70f1025b27 - {'path': '/', 'tag': 'aadedcdd-acc6-4215-9dec-7b70f1025b27', 'options': 'recursive,force'} 2023-10-18T12:31:53.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:53 smithi070 ceph-mon[98639]: pgmap v490: 129 pgs: 129 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 234 op/s 2023-10-18T12:31:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:53 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:31:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:53 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:53 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:31:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:53 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:53.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:53 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:31:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:53 smithi152 ceph-mon[134767]: pgmap v490: 129 pgs: 129 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 234 op/s 2023-10-18T12:31:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:53 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:31:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:53 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:53 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:31:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:53 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:53 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:31:53.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:53 smithi186 ceph-mon[135253]: pgmap v490: 129 pgs: 129 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 234 op/s 2023-10-18T12:31:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:53 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:31:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:53 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:53 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:31:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:53 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:31:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:53 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:31:53.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 631378 12.34 MB/sec execute 142 sec latency 56.416 ms 2023-10-18T12:31:54.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 633425 12.33 MB/sec execute 143 sec latency 13.253 ms 2023-10-18T12:31:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:55 smithi070 ceph-mon[98639]: pgmap v491: 129 pgs: 129 active+clean; 381 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2023-10-18T12:31:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:55 smithi152 ceph-mon[134767]: pgmap v491: 129 pgs: 129 active+clean; 381 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2023-10-18T12:31:55.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:55 smithi186 ceph-mon[135253]: pgmap v491: 129 pgs: 129 active+clean; 381 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2023-10-18T12:31:55.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 635931 12.36 MB/sec execute 144 sec latency 110.472 ms 2023-10-18T12:31:56.852 INFO:tasks.workunit.client.0.smithi070.stdout: 1 637910 12.32 MB/sec execute 145 sec latency 16.514 ms 2023-10-18T12:31:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:57 smithi070 ceph-mon[98639]: pgmap v492: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2023-10-18T12:31:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:57 smithi152 ceph-mon[134767]: pgmap v492: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2023-10-18T12:31:57.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:57 smithi186 ceph-mon[135253]: pgmap v492: 129 pgs: 129 active+clean; 383 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2023-10-18T12:31:57.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 640056 12.33 MB/sec execute 146 sec latency 14.188 ms 2023-10-18T12:31:58.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 642039 12.31 MB/sec execute 147 sec latency 73.705 ms 2023-10-18T12:31:59.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:31:59 smithi070 ceph-mon[98639]: pgmap v493: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2023-10-18T12:31:59.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:31:59 smithi152 ceph-mon[134767]: pgmap v493: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2023-10-18T12:31:59.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:31:59 smithi186 ceph-mon[135253]: pgmap v493: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2023-10-18T12:31:59.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 643964 12.30 MB/sec execute 148 sec latency 23.423 ms 2023-10-18T12:32:00.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 645089 12.25 MB/sec execute 149 sec latency 55.306 ms 2023-10-18T12:32:01.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:01 smithi070 ceph-mon[98639]: pgmap v494: 129 pgs: 129 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2023-10-18T12:32:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:01 smithi152 ceph-mon[134767]: pgmap v494: 129 pgs: 129 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2023-10-18T12:32:01.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:01 smithi186 ceph-mon[135253]: pgmap v494: 129 pgs: 129 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2023-10-18T12:32:01.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 646949 12.25 MB/sec execute 150 sec latency 47.837 ms 2023-10-18T12:32:02.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 649329 12.25 MB/sec execute 151 sec latency 30.448 ms 2023-10-18T12:32:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:03 smithi070 ceph-mon[98639]: pgmap v495: 129 pgs: 129 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2023-10-18T12:32:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:03 smithi152 ceph-mon[134767]: pgmap v495: 129 pgs: 129 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2023-10-18T12:32:03.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:03 smithi186 ceph-mon[135253]: pgmap v495: 129 pgs: 129 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2023-10-18T12:32:03.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 651014 12.23 MB/sec execute 152 sec latency 63.843 ms 2023-10-18T12:32:04.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 653555 12.25 MB/sec execute 153 sec latency 58.645 ms 2023-10-18T12:32:05.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:05 smithi070 ceph-mon[98639]: pgmap v496: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2023-10-18T12:32:05.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:05 smithi152 ceph-mon[134767]: pgmap v496: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2023-10-18T12:32:05.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:05 smithi186 ceph-mon[135253]: pgmap v496: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 235 op/s 2023-10-18T12:32:05.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 655304 12.21 MB/sec execute 154 sec latency 14.978 ms 2023-10-18T12:32:06.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 657392 12.22 MB/sec execute 155 sec latency 150.390 ms 2023-10-18T12:32:07.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:07 smithi070 ceph-mon[98639]: pgmap v497: 129 pgs: 129 active+clean; 371 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2023-10-18T12:32:07.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:07 smithi152 ceph-mon[134767]: pgmap v497: 129 pgs: 129 active+clean; 371 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2023-10-18T12:32:07.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:07 smithi186 ceph-mon[135253]: pgmap v497: 129 pgs: 129 active+clean; 371 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2023-10-18T12:32:07.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 659545 12.20 MB/sec execute 156 sec latency 16.477 ms 2023-10-18T12:32:08.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:08 smithi070 ceph-mon[98639]: pgmap v498: 129 pgs: 129 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2023-10-18T12:32:08.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:08 smithi152 ceph-mon[134767]: pgmap v498: 129 pgs: 129 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2023-10-18T12:32:08.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:08 smithi186 ceph-mon[135253]: pgmap v498: 129 pgs: 129 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2023-10-18T12:32:08.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 661262 12.19 MB/sec execute 157 sec latency 83.067 ms 2023-10-18T12:32:09.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 663553 12.19 MB/sec execute 158 sec latency 37.755 ms 2023-10-18T12:32:10.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 665458 12.19 MB/sec execute 159 sec latency 56.730 ms 2023-10-18T12:32:11.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:11 smithi070 ceph-mon[98639]: pgmap v499: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2023-10-18T12:32:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:11 smithi152 ceph-mon[134767]: pgmap v499: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2023-10-18T12:32:11.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:11 smithi186 ceph-mon[135253]: pgmap v499: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2023-10-18T12:32:11.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 667881 12.20 MB/sec execute 160 sec latency 13.632 ms 2023-10-18T12:32:12.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 669764 12.16 MB/sec execute 161 sec latency 38.945 ms 2023-10-18T12:32:13.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:13 smithi070 ceph-mon[98639]: pgmap v500: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2023-10-18T12:32:13.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:13 smithi152 ceph-mon[134767]: pgmap v500: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2023-10-18T12:32:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:13 smithi186 ceph-mon[135253]: pgmap v500: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2023-10-18T12:32:13.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 671649 12.17 MB/sec execute 162 sec latency 40.100 ms 2023-10-18T12:32:14.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 673672 12.14 MB/sec execute 163 sec latency 36.109 ms 2023-10-18T12:32:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:15 smithi070 ceph-mon[98639]: pgmap v501: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 203 op/s 2023-10-18T12:32:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:15 smithi152 ceph-mon[134767]: pgmap v501: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 203 op/s 2023-10-18T12:32:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:15 smithi186 ceph-mon[135253]: pgmap v501: 129 pgs: 129 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 203 op/s 2023-10-18T12:32:15.853 INFO:tasks.workunit.client.0.smithi070.stdout: 1 675710 12.14 MB/sec execute 164 sec latency 15.257 ms 2023-10-18T12:32:16.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:16 smithi070 ceph-mon[98639]: pgmap v502: 129 pgs: 129 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2023-10-18T12:32:16.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:16 smithi152 ceph-mon[134767]: pgmap v502: 129 pgs: 129 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2023-10-18T12:32:16.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:16 smithi186 ceph-mon[135253]: pgmap v502: 129 pgs: 129 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2023-10-18T12:32:16.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 678533 12.18 MB/sec execute 165 sec latency 15.470 ms 2023-10-18T12:32:17.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 680170 12.14 MB/sec execute 166 sec latency 20.338 ms 2023-10-18T12:32:18.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 682289 12.15 MB/sec execute 167 sec latency 62.475 ms 2023-10-18T12:32:19.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:18 smithi070 ceph-mon[98639]: pgmap v503: 129 pgs: 129 active+clean; 386 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2023-10-18T12:32:19.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:18 smithi152 ceph-mon[134767]: pgmap v503: 129 pgs: 129 active+clean; 386 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2023-10-18T12:32:19.222 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:18 smithi186 ceph-mon[135253]: pgmap v503: 129 pgs: 129 active+clean; 386 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2023-10-18T12:32:19.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 684306 12.12 MB/sec execute 168 sec latency 22.824 ms 2023-10-18T12:32:20.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 686368 12.12 MB/sec execute 169 sec latency 13.748 ms 2023-10-18T12:32:21.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:21 smithi070 ceph-mon[98639]: pgmap v504: 129 pgs: 129 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 246 op/s 2023-10-18T12:32:21.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:21 smithi152 ceph-mon[134767]: pgmap v504: 129 pgs: 129 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 246 op/s 2023-10-18T12:32:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:21 smithi186 ceph-mon[135253]: pgmap v504: 129 pgs: 129 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 246 op/s 2023-10-18T12:32:21.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 689095 12.15 MB/sec execute 170 sec latency 14.799 ms 2023-10-18T12:32:22.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 690810 12.12 MB/sec execute 171 sec latency 18.355 ms 2023-10-18T12:32:22.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:32:23.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 -- 172.21.15.70:0/3853479404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24106720 msgr2=0x7fda24106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/3853479404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24106720 0x7fda24106b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda20009990 tx=0x7fda20009ca0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 -- 172.21.15.70:0/3853479404 shutdown_connections 2023-10-18T12:32:23.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/3853479404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda241010d0 0x7fda2410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/3853479404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24106720 0x7fda24106b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/3853479404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda24100710 0x7fda24100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 -- 172.21.15.70:0/3853479404 >> 172.21.15.70:0/3853479404 conn(0x7fda240fc200 msgr2=0x7fda240fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.210+0000 7fda2b1ff700 1 -- 172.21.15.70:0/3853479404 shutdown_connections 2023-10-18T12:32:23.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.211+0000 7fda2b1ff700 1 -- 172.21.15.70:0/3853479404 wait complete. 2023-10-18T12:32:23.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.211+0000 7fda2b1ff700 1 Processor -- start 2023-10-18T12:32:23.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 -- start start 2023-10-18T12:32:23.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24100710 0x7fda24198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 0x7fda24199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda24106720 0x7fda2419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2419dc90 con 0x7fda24106720 2023-10-18T12:32:23.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2419de00 con 0x7fda241010d0 2023-10-18T12:32:23.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2419dfc0 con 0x7fda24100710 2023-10-18T12:32:23.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 0x7fda24199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 0x7fda24199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50842/0 (socket says 172.21.15.70:50842) 2023-10-18T12:32:23.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24100710 0x7fda24198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda1bfff700 1 -- 172.21.15.70:0/1163204840 learned_addr learned my addr 172.21.15.70:0/1163204840 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:23.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.212+0000 7fda2979c700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda24106720 0x7fda2419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda1bfff700 1 -- 172.21.15.70:0/1163204840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24100710 msgr2=0x7fda24198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda1bfff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24100710 0x7fda24198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda1bfff700 1 -- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda24106720 msgr2=0x7fda2419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda1bfff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda24106720 0x7fda2419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda1bfff700 1 -- 172.21.15.70:0/1163204840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009c40 con 0x7fda241010d0 2023-10-18T12:32:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda1bfff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 0x7fda24199300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda1400e9a0 tx=0x7fda1400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda19ffb700 1 -- 172.21.15.70:0/1163204840 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14009790 con 0x7fda241010d0 2023-10-18T12:32:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda200095f0 con 0x7fda241010d0 2023-10-18T12:32:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda19ffb700 1 -- 172.21.15.70:0/1163204840 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1400ce80 con 0x7fda241010d0 2023-10-18T12:32:23.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda19ffb700 1 -- 172.21.15.70:0/1163204840 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14018680 con 0x7fda241010d0 2023-10-18T12:32:23.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.213+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda241a5900 con 0x7fda241010d0 2023-10-18T12:32:23.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.216+0000 7fda19ffb700 1 -- 172.21.15.70:0/1163204840 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fda14010430 con 0x7fda241010d0 2023-10-18T12:32:23.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 Processor -- start 2023-10-18T12:32:23.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 -- start start 2023-10-18T12:32:23.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda19ffb700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda0c0a15f0 0x7fda0c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda19ffb700 1 -- 172.21.15.70:0/1163204840 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fda14014070 con 0x7fda241010d0 2023-10-18T12:32:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5040fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b503b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b503b5d0 0x5580b503ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4fae6e0 con 0x5580b5040be0 2023-10-18T12:32:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4f94c60 con 0x5580b5041500 2023-10-18T12:32:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.218+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4f92580 con 0x5580b503b5d0 2023-10-18T12:32:23.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda2979c700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda0c0a15f0 0x7fda0c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5040fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5040fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33156/0 (socket says 172.21.15.70:33156) 2023-10-18T12:32:23.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda28f9b700 1 -- 172.21.15.70:0/4030499954 learned_addr learned my addr 172.21.15.70:0/4030499954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:23.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda1bfff700 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b503b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda2979c700 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b503b5d0 0x5580b503ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda1bfff700 1 -- 172.21.15.70:0/4030499954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b503b5d0 msgr2=0x5580b503ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda1bfff700 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b503b5d0 0x5580b503ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda1bfff700 1 -- 172.21.15.70:0/4030499954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 msgr2=0x5580b5040fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda1bfff700 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5040fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.219+0000 7fda1bfff700 1 -- 172.21.15.70:0/4030499954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580b4f4b3f0 con 0x5580b5041500 2023-10-18T12:32:23.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.220+0000 7fda1bfff700 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b503b090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fda140caa40 tx=0x7fda1409a9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a265cce50d2361 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.220+0000 7fda2979c700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda0c0a15f0 0x7fda0c0a3ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009fd0 tx=0x7fda1c00f5a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.220+0000 7fda0affd700 1 -- 172.21.15.70:0/4030499954 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140a2660 con 0x5580b5041500 2023-10-18T12:32:23.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.220+0000 7fda0affd700 1 -- 172.21.15.70:0/4030499954 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1409e130 con 0x5580b5041500 2023-10-18T12:32:23.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.220+0000 7fda0affd700 1 -- 172.21.15.70:0/4030499954 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1409a790 con 0x5580b5041500 2023-10-18T12:32:23.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 -- 172.21.15.70:0/4030499954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 msgr2=0x5580b503b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b503b090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fda140caa40 tx=0x7fda1409a9d0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 -- 172.21.15.70:0/4030499954 shutdown_connections 2023-10-18T12:32:23.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b503b5d0 0x5580b503ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5040fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/4030499954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b503b090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 -- 172.21.15.70:0/4030499954 >> 172.21.15.70:0/4030499954 conn(0x5580b4faf720 msgr2=0x5580b5035b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 -- 172.21.15.70:0/4030499954 shutdown_connections 2023-10-18T12:32:23.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.221+0000 7fda3eb15b80 1 -- 172.21.15.70:0/4030499954 wait complete. 2023-10-18T12:32:23.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.222+0000 7fda3eb15b80 1 -- start start 2023-10-18T12:32:23.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.222+0000 7fda3eb15b80 1 Processor -- start 2023-10-18T12:32:23.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.222+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b503b5d0 0x5580b50474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b50447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b5044e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4f8de70 con 0x5580b503b5d0 2023-10-18T12:32:23.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4f2a710 con 0x5580b5041500 2023-10-18T12:32:23.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4f2a3b0 con 0x5580b5040be0 2023-10-18T12:32:23.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda3eb15b80 10 client.0 _ll_register_callbacks cb 0x5580b4e74a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:23.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b50447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b503b5d0 0x5580b50474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b50447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38460/0 (socket says 172.21.15.70:38460) 2023-10-18T12:32:23.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda2979c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b5044e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 -- 172.21.15.70:0/653681197 learned_addr learned my addr 172.21.15.70:0/653681197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:23.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 -- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 msgr2=0x5580b5044e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b5044e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 -- 172.21.15.70:0/653681197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b503b5d0 msgr2=0x5580b50474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b503b5d0 0x5580b50474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 -- 172.21.15.70:0/653681197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda241a5900 con 0x5580b5040be0 2023-10-18T12:32:23.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.223+0000 7fda1bfff700 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b50447e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda14099fd0 tx=0x7fda14010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e705b8bcb558888 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda03fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:23.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda03fff700 1 -- 172.21.15.70:0/653681197 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140999d0 con 0x5580b5040be0 2023-10-18T12:32:23.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda03fff700 1 -- 172.21.15.70:0/653681197 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fda140cc040 con 0x5580b5040be0 2023-10-18T12:32:23.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580b4f92ef0 con 0x5580b5040be0 2023-10-18T12:32:23.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda03fff700 1 -- 172.21.15.70:0/653681197 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda1409a530 con 0x5580b5040be0 2023-10-18T12:32:23.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda3eb15b80 10 client.26443 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:23.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5580b4db8de0 con 0x5580b5040be0 2023-10-18T12:32:23.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda03fff700 1 -- 172.21.15.70:0/653681197 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580b4f92ef0 con 0x5580b5040be0 2023-10-18T12:32:23.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.224+0000 7fda03fff700 1 -- 172.21.15.70:0/653681197 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda140b82c0 con 0x5580b5040be0 2023-10-18T12:32:23.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda3eb15b80 10 client.26443 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:23.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda3eb15b80 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:23.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda3eb15b80 20 client.26443 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b505d1e0 0x5580b505f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda3eb15b80 4 client.26443 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580b500a8b0 con 0x5580b505d1e0 2023-10-18T12:32:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda28f9b700 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b505d1e0 0x5580b505f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda03fff700 10 client.26443 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:23.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.225+0000 7fda28f9b700 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b505d1e0 0x5580b505f5d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc44d7e6a6646c15 server_cookie=834e3ffdff5fcba7 in_seq=0 out_seq=0 2023-10-18T12:32:23.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda03fff700 1 -- 172.21.15.70:0/653681197 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5580b500a8b0 con 0x5580b505d1e0 2023-10-18T12:32:23.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda03fff700 10 client.26443 handle_command_reply: tid=0 2023-10-18T12:32:23.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 client.26443 shutdown 2023-10-18T12:32:23.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 msgr2=0x5580b50447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b50447e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda14099fd0 tx=0x7fda14010a10 comp rx=0 tx=0).stop 2023-10-18T12:32:23.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 shutdown_connections 2023-10-18T12:32:23.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b505d1e0 0x5580b505f5d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b5041500 0x5580b5044e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b503b5d0 0x5580b50474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/653681197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b50447e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda03fff700 0 client.26443 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 >> 172.21.15.70:0/653681197 conn(0x5580b4faf720 msgr2=0x5580b5035ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 shutdown_connections 2023-10-18T12:32:23.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.227+0000 7fda3eb15b80 1 -- 172.21.15.70:0/653681197 wait complete. 2023-10-18T12:32:23.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.228+0000 7fda3eb15b80 20 client.26443 trim_cache size 0 max 16384 2023-10-18T12:32:23.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 Processor -- start 2023-10-18T12:32:23.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 -- start start 2023-10-18T12:32:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b503b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5054740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5041500 0x5580b503e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b5030af0 con 0x5580b5040be0 2023-10-18T12:32:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b4e0e920 con 0x5580b503b5d0 2023-10-18T12:32:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b50322f0 con 0x5580b5041500 2023-10-18T12:32:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b503b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b503b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50868/0 (socket says 172.21.15.70:50868) 2023-10-18T12:32:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda28f9b700 1 -- 172.21.15.70:0/3589673729 learned_addr learned my addr 172.21.15.70:0/3589673729 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda2979c700 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5041500 0x5580b503e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.247+0000 7fda1bfff700 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5054740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda28f9b700 1 -- 172.21.15.70:0/3589673729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5041500 msgr2=0x5580b503e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda28f9b700 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5041500 0x5580b503e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda28f9b700 1 -- 172.21.15.70:0/3589673729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 msgr2=0x5580b5054740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda28f9b700 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5054740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda28f9b700 1 -- 172.21.15.70:0/3589673729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580b4da1c60 con 0x5580b503b5d0 2023-10-18T12:32:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda28f9b700 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b503b090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fda20005b40 tx=0x7fda2000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeba118e38cad2f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda0b7fe700 1 -- 172.21.15.70:0/3589673729 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2002b070 con 0x5580b503b5d0 2023-10-18T12:32:23.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda0b7fe700 1 -- 172.21.15.70:0/3589673729 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2001e040 con 0x5580b503b5d0 2023-10-18T12:32:23.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.248+0000 7fda0b7fe700 1 -- 172.21.15.70:0/3589673729 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2001d040 con 0x5580b503b5d0 2023-10-18T12:32:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 -- 172.21.15.70:0/3589673729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 msgr2=0x5580b503b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b503b090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fda20005b40 tx=0x7fda2000b260 comp rx=0 tx=0).stop 2023-10-18T12:32:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 -- 172.21.15.70:0/3589673729 shutdown_connections 2023-10-18T12:32:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5041500 0x5580b503e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b503b090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/3589673729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5040be0 0x5580b5054740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 -- 172.21.15.70:0/3589673729 >> 172.21.15.70:0/3589673729 conn(0x5580b4faf720 msgr2=0x5580b5043d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 -- 172.21.15.70:0/3589673729 shutdown_connections 2023-10-18T12:32:23.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.249+0000 7fda3eb15b80 1 -- 172.21.15.70:0/3589673729 wait complete. 2023-10-18T12:32:23.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 -- start start 2023-10-18T12:32:23.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 Processor -- start 2023-10-18T12:32:23.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b50546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b5052b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5041500 0x5580b5053160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b5053870 con 0x5580b5041500 2023-10-18T12:32:23.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b50539e0 con 0x5580b503b5d0 2023-10-18T12:32:23.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580b50b1070 con 0x5580b5040be0 2023-10-18T12:32:23.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.250+0000 7fda3eb15b80 10 client.0 _ll_register_callbacks cb 0x5580b4e74a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:23.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b50546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b50546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50874/0 (socket says 172.21.15.70:50874) 2023-10-18T12:32:23.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 -- 172.21.15.70:0/2238664650 learned_addr learned my addr 172.21.15.70:0/2238664650 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:23.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda2979c700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5041500 0x5580b5053160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda1bfff700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b5052b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 -- 172.21.15.70:0/2238664650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 msgr2=0x5580b5052b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b5052b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 -- 172.21.15.70:0/2238664650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5041500 msgr2=0x5580b5053160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5041500 0x5580b5053160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 -- 172.21.15.70:0/2238664650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2410dee0 con 0x5580b503b5d0 2023-10-18T12:32:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda28f9b700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b50546c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda20009340 tx=0x7fda200058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d643fc22b9b0377 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.251+0000 7fda03fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda03fff700 1 -- 172.21.15.70:0/2238664650 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20029070 con 0x5580b503b5d0 2023-10-18T12:32:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580b503c040 con 0x5580b503b5d0 2023-10-18T12:32:23.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda03fff700 1 -- 172.21.15.70:0/2238664650 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fda2000b950 con 0x5580b503b5d0 2023-10-18T12:32:23.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda03fff700 1 -- 172.21.15.70:0/2238664650 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2002b040 con 0x5580b503b5d0 2023-10-18T12:32:23.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 10 client.26447 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:23.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5580b4ddf250 con 0x5580b503b5d0 2023-10-18T12:32:23.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda03fff700 1 -- 172.21.15.70:0/2238664650 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580b503c040 con 0x5580b503b5d0 2023-10-18T12:32:23.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda03fff700 1 -- 172.21.15.70:0/2238664650 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda20036070 con 0x5580b503b5d0 2023-10-18T12:32:23.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:23.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:23.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 20 client.26447 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:23.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b50b2350 0x5580b50b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 4 client.26447 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:32:23.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.252+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fda1409e130 con 0x5580b50b2350 2023-10-18T12:32:23.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda1bfff700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b50b2350 0x5580b50b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda1bfff700 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b50b2350 0x5580b50b4740 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bab8c6dc8754d05 server_cookie=149ede75c1c306ea in_seq=0 out_seq=0 2023-10-18T12:32:23.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda03fff700 10 client.26447 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:23.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda03fff700 1 -- 172.21.15.70:0/2238664650 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fda1409e130 con 0x5580b50b2350 2023-10-18T12:32:23.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda03fff700 10 client.26447 handle_command_reply: tid=0 2023-10-18T12:32:23.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 client.26447 shutdown 2023-10-18T12:32:23.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 msgr2=0x5580b50546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b50546c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda20009340 tx=0x7fda200058e0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 shutdown_connections 2023-10-18T12:32:23.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5580b50b2350 0x5580b50b4740 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5580b5041500 0x5580b5053160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580b503b5d0 0x5580b50546c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda03fff700 0 client.26447 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:23.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 --2- 172.21.15.70:0/2238664650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580b5040be0 0x5580b5052b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.253+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 >> 172.21.15.70:0/2238664650 conn(0x5580b4faf720 msgr2=0x5580b5043d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 shutdown_connections 2023-10-18T12:32:23.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda3eb15b80 1 -- 172.21.15.70:0/2238664650 wait complete. 2023-10-18T12:32:23.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda3eb15b80 20 client.26447 trim_cache size 0 max 16384 2023-10-18T12:32:23.270 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:32:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:32:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:32:23.270 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:32:23.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda0c0a15f0 msgr2=0x7fda0c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda0c0a15f0 0x7fda0c0a3ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009fd0 tx=0x7fda1c00f5a0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 msgr2=0x7fda24199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.254+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 0x7fda24199300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda1400e9a0 tx=0x7fda1400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 shutdown_connections 2023-10-18T12:32:23.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda24106720 0x7fda2419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda24100710 0x7fda24198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fda0c0a15f0 0x7fda0c0a3ab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 --2- 172.21.15.70:0/1163204840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda241010d0 0x7fda24199300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 >> 172.21.15.70:0/1163204840 conn(0x7fda240fc200 msgr2=0x7fda240fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 shutdown_connections 2023-10-18T12:32:23.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.255+0000 7fda2b1ff700 1 -- 172.21.15.70:0/1163204840 wait complete. 2023-10-18T12:32:23.276 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:32:23.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:32:23.341 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:23 smithi070 ceph-mon[98639]: pgmap v505: 129 pgs: 129 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2023-10-18T12:32:23.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 -- 172.21.15.70:0/3413511112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 msgr2=0x7fd340073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 --2- 172.21.15.70:0/3413511112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 0x7fd340073000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009990 tx=0x7fd33c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 -- 172.21.15.70:0/3413511112 shutdown_connections 2023-10-18T12:32:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 --2- 172.21.15.70:0/3413511112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 --2- 172.21.15.70:0/3413511112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 0x7fd340073000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 --2- 172.21.15.70:0/3413511112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd34010c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 -- 172.21.15.70:0/3413511112 >> 172.21.15.70:0/3413511112 conn(0x7fd3400fc170 msgr2=0x7fd3400fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 -- 172.21.15.70:0/3413511112 shutdown_connections 2023-10-18T12:32:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.489+0000 7fd3485ff700 1 -- 172.21.15.70:0/3413511112 wait complete. 2023-10-18T12:32:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.490+0000 7fd3485ff700 1 Processor -- start 2023-10-18T12:32:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.490+0000 7fd3485ff700 1 -- start start 2023-10-18T12:32:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd3485ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 0x7fd340198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd3485ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd3401992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd3485ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd3485ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34019dcd0 con 0x7fd34010cee0 2023-10-18T12:32:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd3485ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34019de40 con 0x7fd340073540 2023-10-18T12:32:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd3485ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34019e000 con 0x7fd340072c20 2023-10-18T12:32:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd345b9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd3401992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33202/0 (socket says 172.21.15.70:33202) 2023-10-18T12:32:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd345b9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd3401992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50890/0 (socket says 172.21.15.70:50890) 2023-10-18T12:32:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 -- 172.21.15.70:0/713544796 learned_addr learned my addr 172.21.15.70:0/713544796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd34639b700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 0x7fd340198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 -- 172.21.15.70:0/713544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 msgr2=0x7fd340198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 0x7fd340198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 -- 172.21.15.70:0/713544796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 msgr2=0x7fd3401992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd3401992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd346b9c700 1 -- 172.21.15.70:0/713544796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0095f0 con 0x7fd34010cee0 2023-10-18T12:32:23.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.491+0000 7fd345b9a700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd3401992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:23.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.492+0000 7fd346b9c700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34019d690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd33800ef30 tx=0x7fd33800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.492+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338018070 con 0x7fd34010cee0 2023-10-18T12:32:23.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.492+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34019e280 con 0x7fd34010cee0 2023-10-18T12:32:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.492+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33800f040 con 0x7fd34010cee0 2023-10-18T12:32:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.492+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800cd00 con 0x7fd34010cee0 2023-10-18T12:32:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.492+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401a5b20 con 0x7fd34010cee0 2023-10-18T12:32:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.493+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3401080f0 con 0x7fd34010cee0 2023-10-18T12:32:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.495+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd338003a00 con 0x7fd34010cee0 2023-10-18T12:32:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.496+0000 7fd3377fe700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd32c0a15f0 0x7fd32c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.496+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd3380ccf30 con 0x7fd34010cee0 2023-10-18T12:32:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.496+0000 7fd34639b700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd32c0a15f0 0x7fd32c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.497+0000 7fd34639b700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd32c0a15f0 0x7fd32c0a3ab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd33c006010 tx=0x7fd33c00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:23.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.501+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd338095f80 con 0x7fd34010cee0 2023-10-18T12:32:23.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:23 smithi152 ceph-mon[134767]: pgmap v505: 129 pgs: 129 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2023-10-18T12:32:23.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:23 smithi186 ceph-mon[135253]: pgmap v505: 129 pgs: 129 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2023-10-18T12:32:23.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.815+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd34019a0b0 con 0x7fd34010cee0 2023-10-18T12:32:23.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.815+0000 7fd3377fe700 1 -- 172.21.15.70:0/713544796 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fd3380995a0 con 0x7fd34010cee0 2023-10-18T12:32:23.817 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:32:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:32:23.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd32c0a15f0 msgr2=0x7fd32c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd32c0a15f0 0x7fd32c0a3ab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd33c006010 tx=0x7fd33c00b260 comp rx=0 tx=0).stop 2023-10-18T12:32:23.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 msgr2=0x7fd34019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:23.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34019d690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd33800ef30 tx=0x7fd33800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 shutdown_connections 2023-10-18T12:32:23.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34010cee0 0x7fd34019d690 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd340072c20 0x7fd340198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd32c0a15f0 0x7fd32c0a3ab0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 --2- 172.21.15.70:0/713544796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd340073540 0x7fd3401992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:23.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 >> 172.21.15.70:0/713544796 conn(0x7fd3400fc170 msgr2=0x7fd3400fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:23.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 shutdown_connections 2023-10-18T12:32:23.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:23.818+0000 7fd3485ff700 1 -- 172.21.15.70:0/713544796 wait complete. 2023-10-18T12:32:23.822 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:32:23.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:32:23.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 692957 12.13 MB/sec execute 172 sec latency 78.683 ms 2023-10-18T12:32:24.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 -- 172.21.15.70:0/1294210893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48111730 msgr2=0x7f4c48113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/1294210893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48111730 0x7f4c48113b20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000b0a0 tx=0x7f4c4000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 -- 172.21.15.70:0/1294210893 shutdown_connections 2023-10-18T12:32:24.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/1294210893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48111730 0x7f4c48113b20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/1294210893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c48072b90 0x7f4c48072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/1294210893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c480734b0 0x7f4c481111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 -- 172.21.15.70:0/1294210893 >> 172.21.15.70:0/1294210893 conn(0x7f4c480fc1b0 msgr2=0x7f4c480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 -- 172.21.15.70:0/1294210893 shutdown_connections 2023-10-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.057+0000 7f4c4cbff700 1 -- 172.21.15.70:0/1294210893 wait complete. 2023-10-18T12:32:24.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.058+0000 7f4c4cbff700 1 Processor -- start 2023-10-18T12:32:24.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.058+0000 7f4c4cbff700 1 -- start start 2023-10-18T12:32:24.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4cbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 0x7f4c4819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4cbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c480734b0 0x7f4c4819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c48111730 0x7f4c481a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4cbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c481a2290 con 0x7f4c480734b0 2023-10-18T12:32:24.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4cbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c481a2400 con 0x7f4c48072b90 2023-10-18T12:32:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c481a25c0 con 0x7f4c48111730 2023-10-18T12:32:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 0x7f4c4819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c45d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c480734b0 0x7f4c4819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 0x7f4c4819d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50910/0 (socket says 172.21.15.70:50910) 2023-10-18T12:32:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c4659c700 1 -- 172.21.15.70:0/2657582851 learned_addr learned my addr 172.21.15.70:0/2657582851 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.059+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c48111730 0x7f4c481a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4659c700 1 -- 172.21.15.70:0/2657582851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c48111730 msgr2=0x7f4c481a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4659c700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c48111730 0x7f4c481a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4659c700 1 -- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c480734b0 msgr2=0x7f4c4819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4659c700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c480734b0 0x7f4c4819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4659c700 1 -- 172.21.15.70:0/2657582851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c38009520 con 0x7f4c48072b90 2023-10-18T12:32:24.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4659c700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 0x7f4c4819d370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800eb30 tx=0x7f4c3800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:24.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c377fe700 1 -- 172.21.15.70:0/2657582851 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38009880 con 0x7f4c48072b90 2023-10-18T12:32:24.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c377fe700 1 -- 172.21.15.70:0/2657582851 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c380044e0 con 0x7f4c48072b90 2023-10-18T12:32:24.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c40009d00 con 0x7f4c48072b90 2023-10-18T12:32:24.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c377fe700 1 -- 172.21.15.70:0/2657582851 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38003e90 con 0x7f4c48072b90 2023-10-18T12:32:24.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.060+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c481a9f00 con 0x7f4c48072b90 2023-10-18T12:32:24.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.063+0000 7f4c377fe700 1 -- 172.21.15.70:0/2657582851 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4c38004000 con 0x7f4c48072b90 2023-10-18T12:32:24.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 Processor -- start 2023-10-18T12:32:24.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 -- start start 2023-10-18T12:32:24.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c377fe700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c300a15f0 0x7f4c300a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb16409fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 0x55cb16404090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16404a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb163776e0 con 0x55cb16409be0 2023-10-18T12:32:24.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1635dc60 con 0x55cb164045d0 2023-10-18T12:32:24.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1635b580 con 0x55cb1640a500 2023-10-18T12:32:24.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c377fe700 1 -- 172.21.15.70:0/2657582851 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4c38014070 con 0x7f4c48072b90 2023-10-18T12:32:24.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c45d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 0x55cb16404090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c45d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 0x55cb16404090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38532/0 (socket says 172.21.15.70:38532) 2023-10-18T12:32:24.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c45d9b700 1 -- 172.21.15.70:0/937579476 learned_addr learned my addr 172.21.15.70:0/937579476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:24.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.065+0000 7f4c4659c700 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb16409fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16404a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c45d9b700 1 -- 172.21.15.70:0/937579476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 msgr2=0x55cb16404a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:24.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16404a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c45d9b700 1 -- 172.21.15.70:0/937579476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 msgr2=0x55cb16409fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:24.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb16409fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c45d9b700 1 -- 172.21.15.70:0/937579476 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb163143f0 con 0x55cb1640a500 2023-10-18T12:32:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16404a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:32:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c300a15f0 0x7f4c300a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 0x55cb16404090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009ca0 tx=0x7f4c3c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b838467b0c581ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c34ff9700 1 -- 172.21.15.70:0/937579476 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c011d80 con 0x55cb1640a500 2023-10-18T12:32:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c34ff9700 1 -- 172.21.15.70:0/937579476 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3c004cf0 con 0x55cb1640a500 2023-10-18T12:32:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.066+0000 7f4c34ff9700 1 -- 172.21.15.70:0/937579476 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c019690 con 0x55cb1640a500 2023-10-18T12:32:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 -- 172.21.15.70:0/937579476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 msgr2=0x55cb16404090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c300a15f0 0x7f4c300a3ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009cd0 tx=0x7f4c400061f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 0x55cb16404090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009ca0 tx=0x7f4c3c011440 comp rx=0 tx=0).stop 2023-10-18T12:32:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 -- 172.21.15.70:0/937579476 shutdown_connections 2023-10-18T12:32:24.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16404a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb16409fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/937579476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb1640a500 0x55cb16404090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 -- 172.21.15.70:0/937579476 >> 172.21.15.70:0/937579476 conn(0x55cb16378720 msgr2=0x55cb163feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:24.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 -- 172.21.15.70:0/937579476 shutdown_connections 2023-10-18T12:32:24.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.067+0000 7f4c6061cb80 1 -- 172.21.15.70:0/937579476 wait complete. 2023-10-18T12:32:24.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.068+0000 7f4c6061cb80 1 -- start start 2023-10-18T12:32:24.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.068+0000 7f4c6061cb80 1 Processor -- start 2023-10-18T12:32:24.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 0x55cb164104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb1640d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb1640df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb16356e70 con 0x55cb16409be0 2023-10-18T12:32:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb162f3710 con 0x55cb1640a500 2023-10-18T12:32:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb162f33b0 con 0x55cb164045d0 2023-10-18T12:32:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c6061cb80 10 client.0 _ll_register_callbacks cb 0x55cb1623da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 0x55cb164104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 0x55cb164104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38544/0 (socket says 172.21.15.70:38544) 2023-10-18T12:32:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 -- 172.21.15.70:0/3200560677 learned_addr learned my addr 172.21.15.70:0/3200560677 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb1640d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 -- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 msgr2=0x55cb1640df20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T12:32:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb1640df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 -- 172.21.15.70:0/3200560677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 msgr2=0x55cb1640d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb1640d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.069+0000 7f4c4659c700 1 -- 172.21.15.70:0/3200560677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c009900 con 0x55cb164045d0 2023-10-18T12:32:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c4659c700 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 0x55cb164104f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c3809a290 tx=0x7f4c3809a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97de579d2d1d535a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c2dffb700 1 -- 172.21.15.70:0/3200560677 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380a9ec0 con 0x55cb164045d0 2023-10-18T12:32:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c2dffb700 1 -- 172.21.15.70:0/3200560677 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4c38016330 con 0x55cb164045d0 2023-10-18T12:32:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb1635bef0 con 0x55cb164045d0 2023-10-18T12:32:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c2dffb700 1 -- 172.21.15.70:0/3200560677 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c380a7330 con 0x55cb164045d0 2023-10-18T12:32:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c6061cb80 10 client.26467 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.070+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cb16181de0 con 0x55cb164045d0 2023-10-18T12:32:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c2dffb700 1 -- 172.21.15.70:0/3200560677 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb1635bef0 con 0x55cb164045d0 2023-10-18T12:32:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c2dffb700 1 -- 172.21.15.70:0/3200560677 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4c3809a7f0 con 0x55cb164045d0 2023-10-18T12:32:24.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c6061cb80 10 client.26467 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:24.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c6061cb80 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:24.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c6061cb80 20 client.26467 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:24.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb1641dea0 0x55cb16420290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c6061cb80 4 client.26467 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb163d38b0 con 0x55cb1641dea0 2023-10-18T12:32:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.071+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb1641dea0 0x55cb16420290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.072+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb1641dea0 0x55cb16420290 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870ca3bb81729cc5 server_cookie=3e6c6e4407c5b0a2 in_seq=0 out_seq=0 2023-10-18T12:32:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.072+0000 7f4c2dffb700 10 client.26467 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.072+0000 7f4c2dffb700 1 -- 172.21.15.70:0/3200560677 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cb163d38b0 con 0x55cb1641dea0 2023-10-18T12:32:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c2dffb700 10 client.26467 handle_command_reply: tid=0 2023-10-18T12:32:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 client.26467 shutdown 2023-10-18T12:32:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 msgr2=0x55cb164104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 0x55cb164104f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c3809a290 tx=0x7f4c3809a0d0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 shutdown_connections 2023-10-18T12:32:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb1641dea0 0x55cb16420290 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb1640df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb164045d0 0x55cb164104f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/3200560677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb16409be0 0x55cb1640d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 >> 172.21.15.70:0/3200560677 conn(0x55cb16378720 msgr2=0x55cb163feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c2dffb700 0 client.26467 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 shutdown_connections 2023-10-18T12:32:24.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 1 -- 172.21.15.70:0/3200560677 wait complete. 2023-10-18T12:32:24.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.073+0000 7f4c6061cb80 20 client.26467 trim_cache size 0 max 16384 2023-10-18T12:32:24.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.090+0000 7f4c6061cb80 1 Processor -- start 2023-10-18T12:32:24.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 -- start start 2023-10-18T12:32:24.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 0x55cb1640d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb163f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb16414eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb161d7920 con 0x55cb164045d0 2023-10-18T12:32:24.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb164003b0 con 0x55cb1640a500 2023-10-18T12:32:24.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb164153f0 con 0x55cb16409be0 2023-10-18T12:32:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 0x55cb1640d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 0x55cb1640d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33256/0 (socket says 172.21.15.70:33256) 2023-10-18T12:32:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.091+0000 7f4c4659c700 1 -- 172.21.15.70:0/4280547918 learned_addr learned my addr 172.21.15.70:0/4280547918 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb16414eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c4659c700 1 -- 172.21.15.70:0/4280547918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 msgr2=0x55cb163f95d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T12:32:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c4659c700 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb163f95d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c4659c700 1 -- 172.21.15.70:0/4280547918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 msgr2=0x55cb16414eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c4659c700 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb16414eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c4659c700 1 -- 172.21.15.70:0/4280547918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb1616ac60 con 0x55cb164045d0 2023-10-18T12:32:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.092+0000 7f4c46d9d700 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb16414eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.093+0000 7f4c4659c700 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 0x55cb1640d780 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4c380851a0 tx=0x7f4c38098b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ae09e29f615de server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.093+0000 7f4c357fa700 1 -- 172.21.15.70:0/4280547918 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380a2920 con 0x55cb164045d0 2023-10-18T12:32:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.093+0000 7f4c357fa700 1 -- 172.21.15.70:0/4280547918 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c38016330 con 0x55cb164045d0 2023-10-18T12:32:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.093+0000 7f4c357fa700 1 -- 172.21.15.70:0/4280547918 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380a26f0 con 0x55cb164045d0 2023-10-18T12:32:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 -- 172.21.15.70:0/4280547918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 msgr2=0x55cb1640d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 0x55cb1640d780 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4c380851a0 tx=0x7f4c38098b40 comp rx=0 tx=0).stop 2023-10-18T12:32:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 -- 172.21.15.70:0/4280547918 shutdown_connections 2023-10-18T12:32:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb1640a500 0x55cb16414eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb164045d0 0x55cb1640d780 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/4280547918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb163f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 -- 172.21.15.70:0/4280547918 >> 172.21.15.70:0/4280547918 conn(0x55cb16378720 msgr2=0x55cb1640d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 -- 172.21.15.70:0/4280547918 shutdown_connections 2023-10-18T12:32:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.094+0000 7f4c6061cb80 1 -- 172.21.15.70:0/4280547918 wait complete. 2023-10-18T12:32:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 -- start start 2023-10-18T12:32:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 Processor -- start 2023-10-18T12:32:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16405c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb164061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb1640a500 0x55cb16406700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb16407490 con 0x55cb1640a500 2023-10-18T12:32:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb16471c90 con 0x55cb164045d0 2023-10-18T12:32:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb16471dd0 con 0x55cb16409be0 2023-10-18T12:32:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c6061cb80 10 client.0 _ll_register_callbacks cb 0x55cb1623da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16405c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c45d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb164061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c46d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb1640a500 0x55cb16406700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c4659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16405c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50934/0 (socket says 172.21.15.70:50934) 2023-10-18T12:32:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.095+0000 7f4c4659c700 1 -- 172.21.15.70:0/944914552 learned_addr learned my addr 172.21.15.70:0/944914552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c4659c700 1 -- 172.21.15.70:0/944914552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 msgr2=0x55cb164061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c4659c700 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb164061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c4659c700 1 -- 172.21.15.70:0/944914552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb1640a500 msgr2=0x55cb16406700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c4659c700 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb1640a500 0x55cb16406700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c4659c700 1 -- 172.21.15.70:0/944914552 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb16181de0 con 0x55cb164045d0 2023-10-18T12:32:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb164061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c4659c700 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16405c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4c38096a90 tx=0x7f4c3809a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdaafd99f3367ff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c2dffb700 1 -- 172.21.15.70:0/944914552 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3809e6e0 con 0x55cb164045d0 2023-10-18T12:32:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c2dffb700 1 -- 172.21.15.70:0/944914552 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4c3800cc90 con 0x55cb164045d0 2023-10-18T12:32:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c3c004cf0 con 0x55cb164045d0 2023-10-18T12:32:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c2dffb700 1 -- 172.21.15.70:0/944914552 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c380a2920 con 0x55cb164045d0 2023-10-18T12:32:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c6061cb80 10 client.26465 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.096+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cb161a8250 con 0x55cb164045d0 2023-10-18T12:32:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c2dffb700 1 -- 172.21.15.70:0/944914552 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3c004cf0 con 0x55cb164045d0 2023-10-18T12:32:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c2dffb700 1 -- 172.21.15.70:0/944914552 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4c38083c70 con 0x55cb164045d0 2023-10-18T12:32:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c6061cb80 10 client.26465 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c6061cb80 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c6061cb80 20 client.26465 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb16473070 0x55cb16475460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c6061cb80 4 client.26465 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:32:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb1634fe50 con 0x55cb16473070 2023-10-18T12:32:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.097+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb16473070 0x55cb16475460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.098+0000 7f4c2dffb700 10 client.26465 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.098+0000 7f4c45d9b700 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb16473070 0x55cb16475460 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd553a77d7cad10 server_cookie=96d990ea700e4990 in_seq=0 out_seq=0 2023-10-18T12:32:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.098+0000 7f4c2dffb700 1 -- 172.21.15.70:0/944914552 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb1634fe50 con 0x55cb16473070 2023-10-18T12:32:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.098+0000 7f4c2dffb700 10 client.26465 handle_command_reply: tid=0 2023-10-18T12:32:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.098+0000 7f4c6061cb80 1 client.26465 shutdown 2023-10-18T12:32:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 msgr2=0x55cb16405c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16405c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4c38096a90 tx=0x7f4c3809a0d0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 shutdown_connections 2023-10-18T12:32:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cb16473070 0x55cb16475460 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cb1640a500 0x55cb16406700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cb164045d0 0x55cb16405c80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c2dffb700 0 client.26465 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 --2- 172.21.15.70:0/944914552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb16409be0 0x55cb164061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 >> 172.21.15.70:0/944914552 conn(0x55cb16378720 msgr2=0x55cb164088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 shutdown_connections 2023-10-18T12:32:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 1 -- 172.21.15.70:0/944914552 wait complete. 2023-10-18T12:32:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.099+0000 7f4c6061cb80 20 client.26465 trim_cache size 0 max 16384 2023-10-18T12:32:24.114 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:32:24.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c300a15f0 msgr2=0x7f4c300a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c300a15f0 0x7f4c300a3ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009cd0 tx=0x7f4c400061f0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 msgr2=0x7f4c4819d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:24.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 0x7f4c4819d370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800eb30 tx=0x7f4c3800ee40 comp rx=0 tx=0).stop 2023-10-18T12:32:24.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 shutdown_connections 2023-10-18T12:32:24.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c48111730 0x7f4c481a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4c300a15f0 0x7f4c300a3ab0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c48072b90 0x7f4c4819d370 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 --2- 172.21.15.70:0/2657582851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c480734b0 0x7f4c4819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 >> 172.21.15.70:0/2657582851 conn(0x7f4c480fc1b0 msgr2=0x7f4c480fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 shutdown_connections 2023-10-18T12:32:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:24.100+0000 7f4c4cbff700 1 -- 172.21.15.70:0/2657582851 wait complete. 2023-10-18T12:32:24.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:24 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/713544796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:32:24.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:24 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/713544796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:32:24.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:24 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/713544796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:32:24.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 694886 12.10 MB/sec execute 173 sec latency 25.783 ms 2023-10-18T12:32:25.130 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:32:25.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:32:25.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 -- 172.21.15.70:0/1604047689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34381005d0 msgr2=0x7f3438100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 --2- 172.21.15.70:0/1604047689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34381005d0 0x7f3438100a50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3428009920 tx=0x7f3428009c30 comp rx=0 tx=0).stop 2023-10-18T12:32:25.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 -- 172.21.15.70:0/1604047689 shutdown_connections 2023-10-18T12:32:25.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 --2- 172.21.15.70:0/1604047689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438100f90 0x7f343810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 --2- 172.21.15.70:0/1604047689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3438106690 0x7f3438106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 --2- 172.21.15.70:0/1604047689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34381005d0 0x7f3438100a50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 -- 172.21.15.70:0/1604047689 >> 172.21.15.70:0/1604047689 conn(0x7f34380fc0c0 msgr2=0x7f34380fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.366+0000 7f343ef9d700 1 -- 172.21.15.70:0/1604047689 shutdown_connections 2023-10-18T12:32:25.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.367+0000 7f343ef9d700 1 -- 172.21.15.70:0/1604047689 wait complete. 2023-10-18T12:32:25.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.367+0000 7f343ef9d700 1 Processor -- start 2023-10-18T12:32:25.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 -- start start 2023-10-18T12:32:25.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 0x7f343819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438100f90 0x7f343819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3438106690 0x7f3438196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438196f70 con 0x7f3438106690 2023-10-18T12:32:25.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34381970e0 con 0x7f3438100f90 2023-10-18T12:32:25.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343ef9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34381972a0 con 0x7f34381005d0 2023-10-18T12:32:25.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343df9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 0x7f343819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343df9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 0x7f343819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38576/0 (socket says 172.21.15.70:38576) 2023-10-18T12:32:25.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343df9b700 1 -- 172.21.15.70:0/2854889514 learned_addr learned my addr 172.21.15.70:0/2854889514 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343d79a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438100f90 0x7f343819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.368+0000 7f343e79c700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3438106690 0x7f3438196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343df9b700 1 -- 172.21.15.70:0/2854889514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438100f90 msgr2=0x7f343819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343df9b700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438100f90 0x7f343819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343df9b700 1 -- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3438106690 msgr2=0x7f3438196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343df9b700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3438106690 0x7f3438196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343df9b700 1 -- 172.21.15.70:0/2854889514 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430009d00 con 0x7f34381005d0 2023-10-18T12:32:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343df9b700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 0x7f343819c710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f343400e9a0 tx=0x7f343400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f342effd700 1 -- 172.21.15.70:0/2854889514 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434009790 con 0x7f34381005d0 2023-10-18T12:32:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f342effd700 1 -- 172.21.15.70:0/2854889514 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3434010430 con 0x7f34381005d0 2023-10-18T12:32:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3428009520 con 0x7f34381005d0 2023-10-18T12:32:25.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f342effd700 1 -- 172.21.15.70:0/2854889514 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434018680 con 0x7f34381005d0 2023-10-18T12:32:25.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.369+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34381a35d0 con 0x7f34381005d0 2023-10-18T12:32:25.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.372+0000 7f342effd700 1 -- 172.21.15.70:0/2854889514 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f34340105a0 con 0x7f34381005d0 2023-10-18T12:32:25.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 Processor -- start 2023-10-18T12:32:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 -- start start 2023-10-18T12:32:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f342effd700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34240a1740 0x7f34240a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f342effd700 1 -- 172.21.15.70:0/2854889514 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3434014070 con 0x7f34381005d0 2023-10-18T12:32:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd0762cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07630df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07631790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0759b760 con 0x55fd07631330 2023-10-18T12:32:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07581ce0 con 0x55fd0762c960 2023-10-18T12:32:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0757f600 con 0x55fd0762d280 2023-10-18T12:32:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f343e79c700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34240a1740 0x7f34240a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.374+0000 7f343df9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd0762cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd0762cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50958/0 (socket says 172.21.15.70:50958) 2023-10-18T12:32:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 -- 172.21.15.70:0/1722593670 learned_addr learned my addr 172.21.15.70:0/1722593670 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343d79a700 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07630df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343e79c700 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07631790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 -- 172.21.15.70:0/1722593670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 msgr2=0x55fd07630df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07630df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 -- 172.21.15.70:0/1722593670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 msgr2=0x55fd07631790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07631790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 -- 172.21.15.70:0/1722593670 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd07538470 con 0x55fd0762c960 2023-10-18T12:32:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343e79c700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34240a1740 0x7f34240a3c00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3430009fd0 tx=0x7f3430007b80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f343df9b700 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd0762cd40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3434096c50 tx=0x7f3434099e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f15b50378205ca server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.375+0000 7f3423fff700 1 -- 172.21.15.70:0/1722593670 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343409e160 con 0x55fd0762c960 2023-10-18T12:32:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3423fff700 1 -- 172.21.15.70:0/1722593670 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34340cd050 con 0x55fd0762c960 2023-10-18T12:32:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3423fff700 1 -- 172.21.15.70:0/1722593670 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340109c0 con 0x55fd0762c960 2023-10-18T12:32:25.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1722593670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 msgr2=0x55fd0762cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd0762cd40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3434096c50 tx=0x7f3434099e50 comp rx=0 tx=0).stop 2023-10-18T12:32:25.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1722593670 shutdown_connections 2023-10-18T12:32:25.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07631790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd0762cd40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1722593670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07630df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1722593670 >> 172.21.15.70:0/1722593670 conn(0x55fd0759c7a0 msgr2=0x55fd07622d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.376+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1722593670 shutdown_connections 2023-10-18T12:32:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.377+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1722593670 wait complete. 2023-10-18T12:32:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.377+0000 7f3453ad3b80 1 -- start start 2023-10-18T12:32:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.377+0000 7f3453ad3b80 1 Processor -- start 2023-10-18T12:32:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd07638b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07635ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd076365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0757aef0 con 0x55fd07631330 2023-10-18T12:32:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07517790 con 0x55fd0762c960 2023-10-18T12:32:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07517430 con 0x55fd0762d280 2023-10-18T12:32:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f3453ad3b80 10 client.0 _ll_register_callbacks cb 0x55fd07461b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f343d79a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07635ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f343d79a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07635ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38598/0 (socket says 172.21.15.70:38598) 2023-10-18T12:32:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f343df9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd07638b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f343d79a700 1 -- 172.21.15.70:0/1642092223 learned_addr learned my addr 172.21.15.70:0/1642092223 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.378+0000 7f343e79c700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd076365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343d79a700 1 -- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 msgr2=0x55fd07638b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343d79a700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd07638b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343d79a700 1 -- 172.21.15.70:0/1642092223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 msgr2=0x55fd076365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343d79a700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd076365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343d79a700 1 -- 172.21.15.70:0/1642092223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34381a35d0 con 0x55fd0762d280 2023-10-18T12:32:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343e79c700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd076365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f343d79a700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07635ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3428009920 tx=0x7f342800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c47c8aa48715a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f34217fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f34217fa700 1 -- 172.21.15.70:0/1642092223 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800b760 con 0x55fd0762d280 2023-10-18T12:32:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f34217fa700 1 -- 172.21.15.70:0/1642092223 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f342800b8c0 con 0x55fd0762d280 2023-10-18T12:32:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd0757ff70 con 0x55fd0762d280 2023-10-18T12:32:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f34217fa700 1 -- 172.21.15.70:0/1642092223 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f342800ba80 con 0x55fd0762d280 2023-10-18T12:32:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f3453ad3b80 10 client.26482 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.379+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fd073a5e60 con 0x55fd0762d280 2023-10-18T12:32:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f34217fa700 1 -- 172.21.15.70:0/1642092223 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd0757ff70 con 0x55fd0762d280 2023-10-18T12:32:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f34217fa700 1 -- 172.21.15.70:0/1642092223 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3428015020 con 0x55fd0762d280 2023-10-18T12:32:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f3453ad3b80 10 client.26482 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f3453ad3b80 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f3453ad3b80 20 client.26482 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd076466f0 0x55fd07648b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f3453ad3b80 4 client.26482 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd075f7930 con 0x55fd076466f0 2023-10-18T12:32:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.380+0000 7f343df9b700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd076466f0 0x55fd07648b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.381+0000 7f34217fa700 10 client.26482 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.381+0000 7f343df9b700 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd076466f0 0x55fd07648b10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52122e2706e0a70e server_cookie=d7e9652bd84030ae in_seq=0 out_seq=0 2023-10-18T12:32:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.381+0000 7f34217fa700 1 -- 172.21.15.70:0/1642092223 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd075f7930 con 0x55fd076466f0 2023-10-18T12:32:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.381+0000 7f34217fa700 10 client.26482 handle_command_reply: tid=0 2023-10-18T12:32:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.381+0000 7f3453ad3b80 1 client.26482 shutdown 2023-10-18T12:32:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 msgr2=0x55fd07635ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07635ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3428009920 tx=0x7f342800f690 comp rx=0 tx=0).stop 2023-10-18T12:32:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 shutdown_connections 2023-10-18T12:32:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd076466f0 0x55fd07648b10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd076365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762c960 0x55fd07638b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/1642092223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762d280 0x55fd07635ed0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f34217fa700 0 client.26482 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 >> 172.21.15.70:0/1642092223 conn(0x55fd0759c7a0 msgr2=0x55fd07622d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 shutdown_connections 2023-10-18T12:32:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 1 -- 172.21.15.70:0/1642092223 wait complete. 2023-10-18T12:32:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.382+0000 7f3453ad3b80 20 client.26482 trim_cache size 0 max 16384 2023-10-18T12:32:25.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.401+0000 7f3453ad3b80 1 Processor -- start 2023-10-18T12:32:25.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.401+0000 7f3453ad3b80 1 -- start start 2023-10-18T12:32:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.401+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd07630460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.401+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd0762d280 0x55fd076309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 0x55fd07641c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd073fb9a0 con 0x55fd0762d280 2023-10-18T12:32:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07623e40 con 0x55fd07631330 2023-10-18T12:32:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd076430a0 con 0x55fd0762c960 2023-10-18T12:32:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 0x55fd07641c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343df9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd07630460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343d79a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd0762d280 0x55fd076309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 0x55fd07641c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50972/0 (socket says 172.21.15.70:50972) 2023-10-18T12:32:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 -- 172.21.15.70:0/3651240645 learned_addr learned my addr 172.21.15.70:0/3651240645 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 -- 172.21.15.70:0/3651240645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 msgr2=0x55fd07630460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd07630460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 -- 172.21.15.70:0/3651240645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd0762d280 msgr2=0x55fd076309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd0762d280 0x55fd076309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.402+0000 7f343e79c700 1 -- 172.21.15.70:0/3651240645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd0738ece0 con 0x55fd07631330 2023-10-18T12:32:25.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.403+0000 7f343e79c700 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 0x55fd07641c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3430018790 tx=0x7f343001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c57ffb0394cba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.403+0000 7f342cff9700 1 -- 172.21.15.70:0/3651240645 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430020040 con 0x55fd07631330 2023-10-18T12:32:25.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.403+0000 7f342cff9700 1 -- 172.21.15.70:0/3651240645 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343000e070 con 0x55fd07631330 2023-10-18T12:32:25.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.403+0000 7f342cff9700 1 -- 172.21.15.70:0/3651240645 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343001a860 con 0x55fd07631330 2023-10-18T12:32:25.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.403+0000 7f3453ad3b80 1 -- 172.21.15.70:0/3651240645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 msgr2=0x55fd07641c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.403+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 0x55fd07641c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3430018790 tx=0x7f343001b000 comp rx=0 tx=0).stop 2023-10-18T12:32:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 -- 172.21.15.70:0/3651240645 shutdown_connections 2023-10-18T12:32:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd07631330 0x55fd07641c90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd07630460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/3651240645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd0762d280 0x55fd076309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 -- 172.21.15.70:0/3651240645 >> 172.21.15.70:0/3651240645 conn(0x55fd0759c7a0 msgr2=0x55fd076347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 -- 172.21.15.70:0/3651240645 shutdown_connections 2023-10-18T12:32:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 -- 172.21.15.70:0/3651240645 wait complete. 2023-10-18T12:32:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 -- start start 2023-10-18T12:32:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.404+0000 7f3453ad3b80 1 Processor -- start 2023-10-18T12:32:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd0763ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 0x55fd0763d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07637700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07637de0 con 0x55fd07631330 2023-10-18T12:32:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07637f50 con 0x55fd0762d280 2023-10-18T12:32:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07638110 con 0x55fd0762c960 2023-10-18T12:32:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f3453ad3b80 10 client.0 _ll_register_callbacks cb 0x7f342800fe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343d79a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 0x55fd0763d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343df9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd0763ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343d79a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 0x55fd0763d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:50988/0 (socket says 172.21.15.70:50988) 2023-10-18T12:32:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343e79c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07637700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343d79a700 1 -- 172.21.15.70:0/4220611039 learned_addr learned my addr 172.21.15.70:0/4220611039 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343df9b700 1 -- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 msgr2=0x55fd0763d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343df9b700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 0x55fd0763d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343df9b700 1 -- 172.21.15.70:0/4220611039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 msgr2=0x55fd07637700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.405+0000 7f343df9b700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07637700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f343df9b700 1 -- 172.21.15.70:0/4220611039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd073cc2d0 con 0x55fd0762c960 2023-10-18T12:32:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f343d79a700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 0x55fd0763d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f343e79c700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07637700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:25.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f343df9b700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd0763ce00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f34340a2040 tx=0x7f343409a0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe6bf8b4cba3063 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f34217fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f34217fa700 1 -- 172.21.15.70:0/4220611039 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434098580 con 0x55fd0762c960 2023-10-18T12:32:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f34217fa700 1 -- 172.21.15.70:0/4220611039 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3434010880 con 0x55fd0762c960 2023-10-18T12:32:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34340cd050 con 0x55fd0762c960 2023-10-18T12:32:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f34217fa700 1 -- 172.21.15.70:0/4220611039 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34340cd1c0 con 0x55fd0762c960 2023-10-18T12:32:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f3453ad3b80 10 client.26488 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:25.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.406+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fd073a4330 con 0x55fd0762c960 2023-10-18T12:32:25.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f34217fa700 1 -- 172.21.15.70:0/4220611039 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34340cd050 con 0x55fd0762c960 2023-10-18T12:32:25.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f34217fa700 1 -- 172.21.15.70:0/4220611039 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f34340996d0 con 0x55fd0762c960 2023-10-18T12:32:25.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f3453ad3b80 10 client.26488 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:25.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f3453ad3b80 10 client.26488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:25.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f3453ad3b80 20 client.26488 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:25.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd0769c2f0 0x55fd0769e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f3453ad3b80 4 client.26488 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:32:25.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fd07573ed0 con 0x55fd0769c2f0 2023-10-18T12:32:25.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f343d79a700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd0769c2f0 0x55fd0769e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f343d79a700 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd0769c2f0 0x55fd0769e710 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13fb69686d9d47db server_cookie=edff91a6136bd23f in_seq=0 out_seq=0 2023-10-18T12:32:25.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.407+0000 7f34217fa700 10 client.26488 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f34217fa700 1 -- 172.21.15.70:0/4220611039 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fd07573ed0 con 0x55fd0769c2f0 2023-10-18T12:32:25.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f34217fa700 10 client.26488 handle_command_reply: tid=0 2023-10-18T12:32:25.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 client.26488 shutdown 2023-10-18T12:32:25.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 msgr2=0x55fd0763ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd0763ce00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f34340a2040 tx=0x7f343409a0c0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 shutdown_connections 2023-10-18T12:32:25.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fd0769c2f0 0x55fd0769e710 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd07631330 0x55fd07637700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd0762c960 0x55fd0763ce00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f34217fa700 0 client.26488 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 --2- 172.21.15.70:0/4220611039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd0762d280 0x55fd0763d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 >> 172.21.15.70:0/4220611039 conn(0x55fd0759c7a0 msgr2=0x55fd07630460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 shutdown_connections 2023-10-18T12:32:25.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 1 -- 172.21.15.70:0/4220611039 wait complete. 2023-10-18T12:32:25.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.410+0000 7f3453ad3b80 20 client.26488 trim_cache size 0 max 16384 2023-10-18T12:32:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:32:25.427 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:32:25.428 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "1576edd9-c656-414d-a649-eff91f9a4d73", 2023-10-18T12:32:25.428 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:32:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:32:25.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34240a1740 msgr2=0x7f34240a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34240a1740 0x7f34240a3c00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3430009fd0 tx=0x7f3430007b80 comp rx=0 tx=0).stop 2023-10-18T12:32:25.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 msgr2=0x7f343819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 0x7f343819c710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f343400e9a0 tx=0x7f343400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 shutdown_connections 2023-10-18T12:32:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3438106690 0x7f3438196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34381005d0 0x7f343819c710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f34240a1740 0x7f34240a3c00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 --2- 172.21.15.70:0/2854889514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438100f90 0x7f343819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 >> 172.21.15.70:0/2854889514 conn(0x7f34380fc0c0 msgr2=0x7f34380fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 shutdown_connections 2023-10-18T12:32:25.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.411+0000 7f343ef9d700 1 -- 172.21.15.70:0/2854889514 wait complete. 2023-10-18T12:32:25.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:32:25.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:25 smithi070 ceph-mon[98639]: pgmap v506: 129 pgs: 129 active+clean; 399 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-18T12:32:25.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 -- 172.21.15.70:0/304797845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 msgr2=0x7f5e8c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 --2- 172.21.15.70:0/304797845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 0x7f5e8c106b10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009990 tx=0x7f5e7c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 -- 172.21.15.70:0/304797845 shutdown_connections 2023-10-18T12:32:25.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 --2- 172.21.15.70:0/304797845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 --2- 172.21.15.70:0/304797845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 0x7f5e8c106b10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 --2- 172.21.15.70:0/304797845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 0x7f5e8c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 -- 172.21.15.70:0/304797845 >> 172.21.15.70:0/304797845 conn(0x7f5e8c0fc1b0 msgr2=0x7f5e8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 -- 172.21.15.70:0/304797845 shutdown_connections 2023-10-18T12:32:25.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.645+0000 7f5e913ff700 1 -- 172.21.15.70:0/304797845 wait complete. 2023-10-18T12:32:25.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.646+0000 7f5e913ff700 1 Processor -- start 2023-10-18T12:32:25.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.646+0000 7f5e913ff700 1 -- start start 2023-10-18T12:32:25.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.646+0000 7f5e913ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 0x7f5e8c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.646+0000 7f5e913ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.646+0000 7f5e913ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 0x7f5e8c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e913ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1993a0 con 0x7f5e8c1010e0 2023-10-18T12:32:25.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e913ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c199510 con 0x7f5e8c100720 2023-10-18T12:32:25.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e913ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1996d0 con 0x7f5e8c106730 2023-10-18T12:32:25.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 0x7f5e8c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33326/0 (socket says 172.21.15.70:33326) 2023-10-18T12:32:25.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 -- 172.21.15.70:0/2356971083 learned_addr learned my addr 172.21.15.70:0/2356971083 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e8b7fe700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 0x7f5e8c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 -- 172.21.15.70:0/2356971083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 msgr2=0x7f5e8c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 0x7f5e8c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 -- 172.21.15.70:0/2356971083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 msgr2=0x7f5e8c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 0x7f5e8c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 -- 172.21.15.70:0/2356971083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0095f0 con 0x7f5e8c1010e0 2023-10-18T12:32:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e8affd700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 0x7f5e8c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.647+0000 7f5e83fff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c19f020 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400e9a0 tx=0x7f5e7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.648+0000 7f5e88ff9700 1 -- 172.21.15.70:0/2356971083 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74009790 con 0x7f5e8c1010e0 2023-10-18T12:32:25.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.648+0000 7f5e88ff9700 1 -- 172.21.15.70:0/2356971083 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e74010430 con 0x7f5e8c1010e0 2023-10-18T12:32:25.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.648+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c1999b0 con 0x7f5e8c1010e0 2023-10-18T12:32:25.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.648+0000 7f5e88ff9700 1 -- 172.21.15.70:0/2356971083 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74018680 con 0x7f5e8c1010e0 2023-10-18T12:32:25.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.648+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c1a5a60 con 0x7f5e8c1010e0 2023-10-18T12:32:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.651+0000 7f5e88ff9700 1 -- 172.21.15.70:0/2356971083 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5e740105a0 con 0x7f5e8c1010e0 2023-10-18T12:32:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 Processor -- start 2023-10-18T12:32:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5e88ff9700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e780a15f0 0x7f5e780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 -- start start 2023-10-18T12:32:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5e88ff9700 1 -- 172.21.15.70:0/2356971083 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5e74014070 con 0x7f5e8c1010e0 2023-10-18T12:32:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5e8b7fe700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e780a15f0 0x7f5e780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be41c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be41376e0 con 0x555be41c45d0 2023-10-18T12:32:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be411dc60 con 0x555be41ca500 2023-10-18T12:32:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be411b580 con 0x555be41c9be0 2023-10-18T12:32:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.653+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be41c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be41c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51016/0 (socket says 172.21.15.70:51016) 2023-10-18T12:32:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 -- 172.21.15.70:0/4024059658 learned_addr learned my addr 172.21.15.70:0/4024059658 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e8affd700 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 -- 172.21.15.70:0/4024059658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 msgr2=0x555be41c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 -- 172.21.15.70:0/4024059658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 msgr2=0x555be41c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 -- 172.21.15.70:0/4024059658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555be40d43f0 con 0x555be41ca500 2023-10-18T12:32:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e8b7fe700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e780a15f0 0x7f5e780a3ab0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c19f770 tx=0x7f5e84009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e83fff700 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be41c4090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e74096d10 tx=0x7f5e74099f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f35a00876670acf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e817fa700 1 -- 172.21.15.70:0/4024059658 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e740b1070 con 0x555be41ca500 2023-10-18T12:32:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e817fa700 1 -- 172.21.15.70:0/4024059658 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e7409e3f0 con 0x555be41ca500 2023-10-18T12:32:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.654+0000 7f5e817fa700 1 -- 172.21.15.70:0/4024059658 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74010970 con 0x555be41ca500 2023-10-18T12:32:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/4024059658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 msgr2=0x555be41c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be41c4090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e74096d10 tx=0x7f5e74099f10 comp rx=0 tx=0).stop 2023-10-18T12:32:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/4024059658 shutdown_connections 2023-10-18T12:32:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/4024059658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be41c4090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/4024059658 >> 172.21.15.70:0/4024059658 conn(0x555be4138720 msgr2=0x555be41beb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/4024059658 shutdown_connections 2023-10-18T12:32:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.655+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/4024059658 wait complete. 2023-10-18T12:32:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.656+0000 7f5ea4c8bb80 1 -- start start 2023-10-18T12:32:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.656+0000 7f5ea4c8bb80 1 Processor -- start 2023-10-18T12:32:25.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be4116e70 con 0x555be41ca500 2023-10-18T12:32:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be40b3710 con 0x555be41c45d0 2023-10-18T12:32:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be40b33b0 con 0x555be41c9be0 2023-10-18T12:32:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5ea4c8bb80 10 client.0 _ll_register_callbacks cb 0x555be3ffda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38650/0 (socket says 172.21.15.70:38650) 2023-10-18T12:32:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e83fff700 1 -- 172.21.15.70:0/2990808920 learned_addr learned my addr 172.21.15.70:0/2990808920 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8b7fe700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8affd700 1 -- 172.21.15.70:0/2990808920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 msgr2=0x555be41cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8affd700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8affd700 1 -- 172.21.15.70:0/2990808920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 msgr2=0x555be41ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8affd700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e8affd700 1 -- 172.21.15.70:0/2990808920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c1a5a60 con 0x555be41c45d0 2023-10-18T12:32:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.657+0000 7f5e83fff700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5e8affd700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41cd920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000c00 tx=0x7f5e7c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=853d7ce77ab2e5a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5e727fc700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5e727fc700 1 -- 172.21.15.70:0/2990808920 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01d030 con 0x555be41c45d0 2023-10-18T12:32:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5e727fc700 1 -- 172.21.15.70:0/2990808920 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5e7c004910 con 0x555be41c45d0 2023-10-18T12:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555be411bef0 con 0x555be41c45d0 2023-10-18T12:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5e727fc700 1 -- 172.21.15.70:0/2990808920 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e7c004ad0 con 0x555be41c45d0 2023-10-18T12:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5ea4c8bb80 10 client.26495 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555be3f41de0 con 0x555be41c45d0 2023-10-18T12:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.658+0000 7f5e727fc700 1 -- 172.21.15.70:0/2990808920 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555be411bef0 con 0x555be41c45d0 2023-10-18T12:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5e727fc700 1 -- 172.21.15.70:0/2990808920 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e7c00fb90 con 0x555be41c45d0 2023-10-18T12:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5ea4c8bb80 10 client.26495 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5ea4c8bb80 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5ea4c8bb80 20 client.26495 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be41ddf60 0x555be41e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5ea4c8bb80 4 client.26495 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:25.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555be41938b0 con 0x555be41ddf60 2023-10-18T12:32:25.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5e83fff700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be41ddf60 0x555be41e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5e83fff700 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be41ddf60 0x555be41e0380 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5383028a3032b8 server_cookie=58ef6663b69a26bb in_seq=0 out_seq=0 2023-10-18T12:32:25.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.659+0000 7f5e727fc700 10 client.26495 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5e727fc700 1 -- 172.21.15.70:0/2990808920 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555be41938b0 con 0x555be41ddf60 2023-10-18T12:32:25.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5e727fc700 10 client.26495 handle_command_reply: tid=0 2023-10-18T12:32:25.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 client.26495 shutdown 2023-10-18T12:32:25.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 msgr2=0x555be41cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41cd920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000c00 tx=0x7f5e7c005270 comp rx=0 tx=0).stop 2023-10-18T12:32:25.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 shutdown_connections 2023-10-18T12:32:25.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be41ddf60 0x555be41e0380 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41cd920 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/2990808920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5e727fc700 0 client.26495 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 >> 172.21.15.70:0/2990808920 conn(0x555be4138720 msgr2=0x555be41beac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.660+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 shutdown_connections 2023-10-18T12:32:25.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.661+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/2990808920 wait complete. 2023-10-18T12:32:25.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.661+0000 7f5ea4c8bb80 20 client.26495 trim_cache size 0 max 16384 2023-10-18T12:32:25.679 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:25 smithi152 ceph-mon[134767]: pgmap v506: 129 pgs: 129 active+clean; 399 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-18T12:32:25.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 Processor -- start 2023-10-18T12:32:25.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 -- start start 2023-10-18T12:32:25.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be41bf130 con 0x555be41ca500 2023-10-18T12:32:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be3f97920 con 0x555be41c45d0 2023-10-18T12:32:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.678+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be41cef00 con 0x555be41c9be0 2023-10-18T12:32:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51030/0 (socket says 172.21.15.70:51030) 2023-10-18T12:32:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 -- 172.21.15.70:0/1235955833 learned_addr learned my addr 172.21.15.70:0/1235955833 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8b7fe700 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 -- 172.21.15.70:0/1235955833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 msgr2=0x555be41c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 -- 172.21.15.70:0/1235955833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 msgr2=0x555be41ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 -- 172.21.15.70:0/1235955833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555be3f2ac60 con 0x555be41c45d0 2023-10-18T12:32:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e83fff700 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.679+0000 7f5e8affd700 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41c4090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c011030 tx=0x7f5e7c024710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2710ee7f74f0daba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5e81ffb700 1 -- 172.21.15.70:0/1235955833 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01d040 con 0x555be41c45d0 2023-10-18T12:32:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5e81ffb700 1 -- 172.21.15.70:0/1235955833 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e7c01e040 con 0x555be41c45d0 2023-10-18T12:32:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5e81ffb700 1 -- 172.21.15.70:0/1235955833 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c017980 con 0x555be41c45d0 2023-10-18T12:32:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/1235955833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 msgr2=0x555be41c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41c4090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c011030 tx=0x7f5e7c024710 comp rx=0 tx=0).stop 2023-10-18T12:32:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/1235955833 shutdown_connections 2023-10-18T12:32:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41ca500 0x555be41ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41c45d0 0x555be41c4090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/1235955833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.680+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/1235955833 >> 172.21.15.70:0/1235955833 conn(0x555be4138720 msgr2=0x555be41c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.681+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/1235955833 shutdown_connections 2023-10-18T12:32:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.681+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/1235955833 wait complete. 2023-10-18T12:32:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.681+0000 7f5ea4c8bb80 1 -- start start 2023-10-18T12:32:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.681+0000 7f5ea4c8bb80 1 Processor -- start 2023-10-18T12:32:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be422ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be422b2c0 con 0x555be41c45d0 2023-10-18T12:32:25.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be422b430 con 0x555be41ca500 2023-10-18T12:32:25.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be422b5f0 con 0x555be41c9be0 2023-10-18T12:32:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5ea4c8bb80 10 client.0 _ll_register_callbacks cb 0x555be3ffda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33358/0 (socket says 172.21.15.70:33358) 2023-10-18T12:32:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 -- 172.21.15.70:0/3939174728 learned_addr learned my addr 172.21.15.70:0/3939174728 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be422ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e83fff700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 -- 172.21.15.70:0/3939174728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 msgr2=0x555be41d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41d32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 -- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 msgr2=0x555be422ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:25.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be422ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e8affd700 1 -- 172.21.15.70:0/3939174728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555be40d43f0 con 0x555be41c45d0 2023-10-18T12:32:25.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.682+0000 7f5e83fff700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:25.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5e8affd700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41d2d70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00ff20 tx=0x7f5e7c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de48d1e1848cd3a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:25.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5e727fc700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:32:25.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5e727fc700 1 -- 172.21.15.70:0/3939174728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004910 con 0x555be41c45d0 2023-10-18T12:32:25.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e7409e3f0 con 0x555be41c45d0 2023-10-18T12:32:25.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5e727fc700 1 -- 172.21.15.70:0/3939174728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5e7c005940 con 0x555be41c45d0 2023-10-18T12:32:25.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5e727fc700 1 -- 172.21.15.70:0/3939174728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e7c024e70 con 0x555be41c45d0 2023-10-18T12:32:25.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5ea4c8bb80 10 client.16479 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:25.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555be3f41de0 con 0x555be41c45d0 2023-10-18T12:32:25.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.683+0000 7f5e727fc700 1 -- 172.21.15.70:0/3939174728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e7409e3f0 con 0x555be41c45d0 2023-10-18T12:32:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5e727fc700 1 -- 172.21.15.70:0/3939174728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e7c00f7b0 con 0x555be41c45d0 2023-10-18T12:32:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5ea4c8bb80 10 client.16479 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5ea4c8bb80 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5ea4c8bb80 20 client.16479 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be4232970 0x555be4234d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5ea4c8bb80 4 client.16479 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:32:25.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555be410fe50 con 0x555be4232970 2023-10-18T12:32:25.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5e83fff700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be4232970 0x555be4234d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:25.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5e727fc700 10 client.16479 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.684+0000 7f5e83fff700 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be4232970 0x555be4234d60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9d74230bf43b6b server_cookie=2db056d071667bef in_seq=0 out_seq=0 2023-10-18T12:32:25.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.687+0000 7f5e727fc700 1 -- 172.21.15.70:0/3939174728 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555be410fe50 con 0x555be4232970 2023-10-18T12:32:25.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.687+0000 7f5e727fc700 10 client.16479 handle_command_reply: tid=0 2023-10-18T12:32:25.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.687+0000 7f5ea4c8bb80 1 client.16479 shutdown 2023-10-18T12:32:25.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 msgr2=0x555be41d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41d2d70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00ff20 tx=0x7f5e7c005270 comp rx=0 tx=0).stop 2023-10-18T12:32:25.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 shutdown_connections 2023-10-18T12:32:25.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555be4232970 0x555be4234d60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555be41ca500 0x555be422ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555be41c45d0 0x555be41d2d70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 --2- 172.21.15.70:0/3939174728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555be41c9be0 0x555be41d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 >> 172.21.15.70:0/3939174728 conn(0x555be4138720 msgr2=0x555be41ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5e727fc700 0 client.16479 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:25.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 shutdown_connections 2023-10-18T12:32:25.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 1 -- 172.21.15.70:0/3939174728 wait complete. 2023-10-18T12:32:25.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.688+0000 7f5ea4c8bb80 20 client.16479 trim_cache size 0 max 16384 2023-10-18T12:32:25.703 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:32:25.703 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:32:25.703 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:32:25.703 INFO:teuthology.orchestra.run.smithi070.stdout: "1576edd9-c656-414d-a649-eff91f9a4d73": { 2023-10-18T12:32:25.703 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:32:25.703 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "1576edd9-c656-414d-a649-eff91f9a4d73", 2023-10-18T12:32:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:32:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:32:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:32:25.704 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:32:25.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e780a15f0 msgr2=0x7f5e780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e780a15f0 0x7f5e780a3ab0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c19f770 tx=0x7f5e84009920 comp rx=0 tx=0).stop 2023-10-18T12:32:25.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 msgr2=0x7f5e8c19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:25.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c19f020 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400e9a0 tx=0x7f5e7400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 shutdown_connections 2023-10-18T12:32:25.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c106730 0x7f5e8c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e780a15f0 0x7f5e780a3ab0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e8c100720 0x7f5e8c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 --2- 172.21.15.70:0/2356971083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e8c1010e0 0x7f5e8c19f020 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:25.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 >> 172.21.15.70:0/2356971083 conn(0x7f5e8c0fc1b0 msgr2=0x7f5e8c10bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:25.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 shutdown_connections 2023-10-18T12:32:25.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:25.689+0000 7f5e913ff700 1 -- 172.21.15.70:0/2356971083 wait complete. 2023-10-18T12:32:25.711 INFO:tasks.cephfs.filesystem:scrub status for tag:1576edd9-c656-414d-a649-eff91f9a4d73 - {'path': '/', 'tag': '1576edd9-c656-414d-a649-eff91f9a4d73', 'options': 'recursive,force'} 2023-10-18T12:32:25.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:25 smithi186 ceph-mon[135253]: pgmap v506: 129 pgs: 129 active+clean; 399 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 240 op/s 2023-10-18T12:32:25.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 696843 12.10 MB/sec execute 174 sec latency 24.579 ms 2023-10-18T12:32:26.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:26 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:32:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:26 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:26 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:32:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:26 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:26.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:26 smithi070 ceph-mon[98639]: pgmap v507: 129 pgs: 129 active+clean; 398 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2023-10-18T12:32:26.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:26 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:32:26.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:26 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:26.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:26 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:32:26.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:26 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:26.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:26 smithi152 ceph-mon[134767]: pgmap v507: 129 pgs: 129 active+clean; 398 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2023-10-18T12:32:26.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:26 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:32:26.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:26 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:26.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:26 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:32:26.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:26 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:26.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:26 smithi186 ceph-mon[135253]: pgmap v507: 129 pgs: 129 active+clean; 398 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2023-10-18T12:32:26.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 699655 12.13 MB/sec execute 175 sec latency 16.481 ms 2023-10-18T12:32:27.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 701359 12.10 MB/sec execute 176 sec latency 35.583 ms 2023-10-18T12:32:28.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 703661 12.11 MB/sec execute 177 sec latency 26.006 ms 2023-10-18T12:32:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:28 smithi070 ceph-mon[98639]: pgmap v508: 129 pgs: 129 active+clean; 403 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2023-10-18T12:32:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:28 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:32:29.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:28 smithi152 ceph-mon[134767]: pgmap v508: 129 pgs: 129 active+clean; 403 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2023-10-18T12:32:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:28 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:32:29.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:28 smithi186 ceph-mon[135253]: pgmap v508: 129 pgs: 129 active+clean; 403 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2023-10-18T12:32:29.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:28 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:32:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 705548 12.08 MB/sec execute 178 sec latency 45.859 ms 2023-10-18T12:32:30.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 707732 12.09 MB/sec execute 179 sec latency 13.055 ms 2023-10-18T12:32:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:31 smithi070 ceph-mon[98639]: pgmap v509: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2023-10-18T12:32:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:31 smithi152 ceph-mon[134767]: pgmap v509: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2023-10-18T12:32:31.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:31 smithi186 ceph-mon[135253]: pgmap v509: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2023-10-18T12:32:31.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 710483 12.12 MB/sec execute 180 sec latency 44.650 ms 2023-10-18T12:32:32.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 712613 12.09 MB/sec execute 181 sec latency 15.583 ms 2023-10-18T12:32:33.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:33 smithi070 ceph-mon[98639]: pgmap v510: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2023-10-18T12:32:33.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:33 smithi152 ceph-mon[134767]: pgmap v510: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2023-10-18T12:32:33.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:33 smithi186 ceph-mon[135253]: pgmap v510: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2023-10-18T12:32:33.854 INFO:tasks.workunit.client.0.smithi070.stdout: 1 714651 12.10 MB/sec execute 182 sec latency 13.323 ms 2023-10-18T12:32:34.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 716694 12.09 MB/sec execute 183 sec latency 71.846 ms 2023-10-18T12:32:35.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:35 smithi070 ceph-mon[98639]: pgmap v511: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 262 op/s 2023-10-18T12:32:35.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:35 smithi152 ceph-mon[134767]: pgmap v511: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 262 op/s 2023-10-18T12:32:35.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:35 smithi186 ceph-mon[135253]: pgmap v511: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 262 op/s 2023-10-18T12:32:35.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 718663 12.09 MB/sec execute 184 sec latency 11.014 ms 2023-10-18T12:32:36.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:36 smithi070 ceph-mon[98639]: pgmap v512: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-18T12:32:36.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:36 smithi152 ceph-mon[134767]: pgmap v512: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-18T12:32:36.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:36 smithi186 ceph-mon[135253]: pgmap v512: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-10-18T12:32:36.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 721126 12.10 MB/sec execute 185 sec latency 15.208 ms 2023-10-18T12:32:37.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:37 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:32:37.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:37 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:32:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:37 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:32:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 722632 12.07 MB/sec execute 186 sec latency 32.249 ms 2023-10-18T12:32:38.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:32:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:32:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:32:38.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:38 smithi070 ceph-mon[98639]: pgmap v513: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 236 op/s 2023-10-18T12:32:38.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:32:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:32:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:32:38.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:38 smithi152 ceph-mon[134767]: pgmap v513: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 236 op/s 2023-10-18T12:32:38.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:32:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:32:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:32:38.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:38 smithi186 ceph-mon[135253]: pgmap v513: 129 pgs: 129 active+clean; 419 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 236 op/s 2023-10-18T12:32:38.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 725071 12.08 MB/sec execute 187 sec latency 14.557 ms 2023-10-18T12:32:39.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 726743 12.05 MB/sec execute 188 sec latency 72.426 ms 2023-10-18T12:32:40.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 728555 12.05 MB/sec execute 189 sec latency 131.030 ms 2023-10-18T12:32:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:41 smithi070 ceph-mon[98639]: pgmap v514: 129 pgs: 129 active+clean; 411 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 242 op/s 2023-10-18T12:32:41.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:41 smithi152 ceph-mon[134767]: pgmap v514: 129 pgs: 129 active+clean; 411 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 242 op/s 2023-10-18T12:32:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:41 smithi186 ceph-mon[135253]: pgmap v514: 129 pgs: 129 active+clean; 411 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 242 op/s 2023-10-18T12:32:41.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 731141 12.06 MB/sec execute 190 sec latency 15.509 ms 2023-10-18T12:32:42.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:42 smithi070 ceph-mon[98639]: pgmap v515: 129 pgs: 129 active+clean; 407 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-18T12:32:42.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:42 smithi152 ceph-mon[134767]: pgmap v515: 129 pgs: 129 active+clean; 407 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-18T12:32:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:42 smithi186 ceph-mon[135253]: pgmap v515: 129 pgs: 129 active+clean; 407 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2023-10-18T12:32:42.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 732991 12.05 MB/sec execute 191 sec latency 20.072 ms 2023-10-18T12:32:43.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 735518 12.06 MB/sec execute 192 sec latency 12.446 ms 2023-10-18T12:32:44.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 737128 12.03 MB/sec execute 193 sec latency 52.988 ms 2023-10-18T12:32:45.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:45 smithi070 ceph-mon[98639]: pgmap v516: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 221 op/s 2023-10-18T12:32:45.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:45 smithi152 ceph-mon[134767]: pgmap v516: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 221 op/s 2023-10-18T12:32:45.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:45 smithi186 ceph-mon[135253]: pgmap v516: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 221 op/s 2023-10-18T12:32:45.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 739338 12.04 MB/sec execute 194 sec latency 14.041 ms 2023-10-18T12:32:46.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 742212 12.06 MB/sec execute 195 sec latency 12.155 ms 2023-10-18T12:32:47.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:47 smithi070 ceph-mon[98639]: pgmap v517: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 194 op/s 2023-10-18T12:32:47.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:47 smithi152 ceph-mon[134767]: pgmap v517: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 194 op/s 2023-10-18T12:32:47.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:47 smithi186 ceph-mon[135253]: pgmap v517: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 194 op/s 2023-10-18T12:32:47.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 743832 12.04 MB/sec execute 196 sec latency 33.529 ms 2023-10-18T12:32:48.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:48 smithi070 ceph-mon[98639]: pgmap v518: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2023-10-18T12:32:48.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:48 smithi152 ceph-mon[134767]: pgmap v518: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2023-10-18T12:32:48.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:48 smithi186 ceph-mon[135253]: pgmap v518: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2023-10-18T12:32:48.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 746271 12.04 MB/sec execute 197 sec latency 19.389 ms 2023-10-18T12:32:49.855 INFO:tasks.workunit.client.0.smithi070.stdout: 1 748162 12.02 MB/sec execute 198 sec latency 139.994 ms 2023-10-18T12:32:50.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 750091 12.02 MB/sec execute 199 sec latency 26.588 ms 2023-10-18T12:32:51.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:51 smithi070 ceph-mon[98639]: pgmap v519: 129 pgs: 129 active+clean; 404 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 202 op/s 2023-10-18T12:32:51.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:51 smithi152 ceph-mon[134767]: pgmap v519: 129 pgs: 129 active+clean; 404 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 202 op/s 2023-10-18T12:32:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:51 smithi186 ceph-mon[135253]: pgmap v519: 129 pgs: 129 active+clean; 404 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 202 op/s 2023-10-18T12:32:51.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 752852 12.05 MB/sec execute 200 sec latency 15.192 ms 2023-10-18T12:32:52.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:52 smithi070 ceph-mon[98639]: pgmap v520: 129 pgs: 129 active+clean; 409 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 207 op/s 2023-10-18T12:32:52.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:52 smithi152 ceph-mon[134767]: pgmap v520: 129 pgs: 129 active+clean; 409 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 207 op/s 2023-10-18T12:32:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:52 smithi186 ceph-mon[135253]: pgmap v520: 129 pgs: 129 active+clean; 409 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 207 op/s 2023-10-18T12:32:52.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 754545 12.02 MB/sec execute 201 sec latency 29.039 ms 2023-10-18T12:32:53.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 756928 12.03 MB/sec execute 202 sec latency 28.482 ms 2023-10-18T12:32:54.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 759078 12.03 MB/sec execute 203 sec latency 128.735 ms 2023-10-18T12:32:55.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:55 smithi070 ceph-mon[98639]: pgmap v521: 129 pgs: 129 active+clean; 409 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 213 op/s 2023-10-18T12:32:55.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:55 smithi152 ceph-mon[134767]: pgmap v521: 129 pgs: 129 active+clean; 409 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 213 op/s 2023-10-18T12:32:55.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:32:55.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:55 smithi186 ceph-mon[135253]: pgmap v521: 129 pgs: 129 active+clean; 409 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 213 op/s 2023-10-18T12:32:55.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 761087 12.02 MB/sec execute 204 sec latency 28.496 ms 2023-10-18T12:32:55.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 -- 172.21.15.70:0/2625196228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784105170 msgr2=0x7f97841055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:55.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2625196228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784105170 0x7f97841055f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9774009920 tx=0x7f9774009c30 comp rx=0 tx=0).stop 2023-10-18T12:32:55.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 -- 172.21.15.70:0/2625196228 shutdown_connections 2023-10-18T12:32:55.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2625196228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105b30 0x7f978410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:55.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2625196228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784069290 0x7f9784069670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:55.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2625196228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784105170 0x7f97841055f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:55.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.938+0000 7f978c3ff700 1 -- 172.21.15.70:0/2625196228 >> 172.21.15.70:0/2625196228 conn(0x7f9784076790 msgr2=0x7f9784076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.939+0000 7f978c3ff700 1 -- 172.21.15.70:0/2625196228 shutdown_connections 2023-10-18T12:32:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.939+0000 7f978c3ff700 1 -- 172.21.15.70:0/2625196228 wait complete. 2023-10-18T12:32:55.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 Processor -- start 2023-10-18T12:32:55.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 -- start start 2023-10-18T12:32:55.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784069290 0x7f9784198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:55.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105170 0x7f9784199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:55.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 0x7f978419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:55.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419de50 con 0x7f9784105b30 2023-10-18T12:32:56.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419dfc0 con 0x7f9784105170 2023-10-18T12:32:56.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.940+0000 7f978c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e180 con 0x7f9784069290 2023-10-18T12:32:56.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 0x7f978419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 0x7f978419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60216/0 (socket says 172.21.15.70:60216) 2023-10-18T12:32:56.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 -- 172.21.15.70:0/2945203744 learned_addr learned my addr 172.21.15.70:0/2945203744 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978999a700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105170 0x7f9784199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a19b700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784069290 0x7f9784198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 -- 172.21.15.70:0/2945203744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784069290 msgr2=0x7f9784198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784069290 0x7f9784198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 -- 172.21.15.70:0/2945203744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105170 msgr2=0x7f9784199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105170 0x7f9784199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 -- 172.21.15.70:0/2945203744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780009520 con 0x7f9784105b30 2023-10-18T12:32:56.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978999a700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105170 0x7f9784199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f978a99c700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 0x7f978419d810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c110 tx=0x7f977c00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.941+0000 7f977b7fe700 1 -- 172.21.15.70:0/2945203744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c017070 con 0x7f9784105b30 2023-10-18T12:32:56.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.942+0000 7f977b7fe700 1 -- 172.21.15.70:0/2945203744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f977c00e050 con 0x7f9784105b30 2023-10-18T12:32:56.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.942+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774009520 con 0x7f9784105b30 2023-10-18T12:32:56.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.942+0000 7f977b7fe700 1 -- 172.21.15.70:0/2945203744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0078f0 con 0x7f9784105b30 2023-10-18T12:32:56.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.942+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97841a5ac0 con 0x7f9784105b30 2023-10-18T12:32:56.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.945+0000 7f977b7fe700 1 -- 172.21.15.70:0/2945203744 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f977c007a50 con 0x7f9784105b30 2023-10-18T12:32:56.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 Processor -- start 2023-10-18T12:32:56.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 -- start start 2023-10-18T12:32:56.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f977b7fe700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f97700a15f0 0x7f97700a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c6500 0x55a9f66c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f977b7fe700 1 -- 172.21.15.70:0/2945203744 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f977c0cce70 con 0x7f9784105b30 2023-10-18T12:32:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c05d0 0x55a9f66c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f66336e0 con 0x55a9f66c05d0 2023-10-18T12:32:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f6619c60 con 0x55a9f66c6500 2023-10-18T12:32:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.947+0000 7f979fe10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f6617580 con 0x55a9f66c5be0 2023-10-18T12:32:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978999a700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f97700a15f0 0x7f97700a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c6500 0x55a9f66c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:43792/0 (socket says 172.21.15.70:43792) 2023-10-18T12:32:56.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 -- 172.21.15.70:0/2662642453 learned_addr learned my addr 172.21.15.70:0/2662642453 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a99c700 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c05d0 0x55a9f66c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 -- 172.21.15.70:0/2662642453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c6500 msgr2=0x55a9f66c0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c6500 0x55a9f66c0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 -- 172.21.15.70:0/2662642453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c05d0 msgr2=0x55a9f66c0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c05d0 0x55a9f66c0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 -- 172.21.15.70:0/2662642453 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f65d03f0 con 0x55a9f66c5be0 2023-10-18T12:32:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978999a700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f97700a15f0 0x7f97700a3ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f978419a480 tx=0x7f9774010ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.948+0000 7f978a19b700 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66c5fc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9780009f10 tx=0x7f9780011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e9e89b45a4d899 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f9778bfe700 1 -- 172.21.15.70:0/2662642453 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780019630 con 0x55a9f66c5be0 2023-10-18T12:32:56.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f9778bfe700 1 -- 172.21.15.70:0/2662642453 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9780019c50 con 0x55a9f66c5be0 2023-10-18T12:32:56.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f9778bfe700 1 -- 172.21.15.70:0/2662642453 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978000ec40 con 0x55a9f66c5be0 2023-10-18T12:32:56.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f979fe10b80 1 -- 172.21.15.70:0/2662642453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 msgr2=0x55a9f66c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66c5fc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9780009f10 tx=0x7f9780011720 comp rx=0 tx=0).stop 2023-10-18T12:32:56.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f979fe10b80 1 -- 172.21.15.70:0/2662642453 shutdown_connections 2023-10-18T12:32:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c05d0 0x55a9f66c0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66c5fc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.949+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2662642453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c6500 0x55a9f66c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.950+0000 7f979fe10b80 1 -- 172.21.15.70:0/2662642453 >> 172.21.15.70:0/2662642453 conn(0x55a9f6634720 msgr2=0x55a9f66bab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.950+0000 7f979fe10b80 1 -- 172.21.15.70:0/2662642453 shutdown_connections 2023-10-18T12:32:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.950+0000 7f979fe10b80 1 -- 172.21.15.70:0/2662642453 wait complete. 2023-10-18T12:32:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.950+0000 7f979fe10b80 1 -- start start 2023-10-18T12:32:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 Processor -- start 2023-10-18T12:32:56.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c05d0 0x55a9f66cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 0x55a9f66c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f6612e70 con 0x55a9f66c6500 2023-10-18T12:32:56.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f65af710 con 0x55a9f66c5be0 2023-10-18T12:32:56.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f65af3b0 con 0x55a9f66c05d0 2023-10-18T12:32:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.951+0000 7f979fe10b80 10 client.0 _ll_register_callbacks cb 0x55a9f64f9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 0x55a9f66c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 0x55a9f66c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48024/0 (socket says 172.21.15.70:48024) 2023-10-18T12:32:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 -- 172.21.15.70:0/4100970205 learned_addr learned my addr 172.21.15.70:0/4100970205 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978a19b700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c05d0 0x55a9f66cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978a99c700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 -- 172.21.15.70:0/4100970205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c05d0 msgr2=0x55a9f66cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c05d0 0x55a9f66cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 -- 172.21.15.70:0/4100970205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 msgr2=0x55a9f66c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 -- 172.21.15.70:0/4100970205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9780009b70 con 0x55a9f66c5be0 2023-10-18T12:32:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978a99c700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978a19b700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c05d0 0x55a9f66cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f978999a700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 0x55a9f66c97c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9774016000 tx=0x7f9774009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbc2bde16db1037 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f976dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.952+0000 7f976dffb700 1 -- 172.21.15.70:0/4100970205 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977401c070 con 0x55a9f66c5be0 2023-10-18T12:32:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f6617ef0 con 0x55a9f66c5be0 2023-10-18T12:32:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f976dffb700 1 -- 172.21.15.70:0/4100970205 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9774005110 con 0x55a9f66c5be0 2023-10-18T12:32:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f976dffb700 1 -- 172.21.15.70:0/4100970205 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97740052d0 con 0x55a9f66c5be0 2023-10-18T12:32:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 10 client.26513 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a9f643dde0 con 0x55a9f66c5be0 2023-10-18T12:32:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f976dffb700 1 -- 172.21.15.70:0/4100970205 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9f6617ef0 con 0x55a9f66c5be0 2023-10-18T12:32:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f976dffb700 1 -- 172.21.15.70:0/4100970205 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9774028610 con 0x55a9f66c5be0 2023-10-18T12:32:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 20 client.26513 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f66d9ec0 0x55a9f66dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 4 client.26513 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.953+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f668f8b0 con 0x55a9f66d9ec0 2023-10-18T12:32:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.954+0000 7f978a19b700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f66d9ec0 0x55a9f66dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.954+0000 7f976dffb700 10 client.26513 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.954+0000 7f978a19b700 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f66d9ec0 0x55a9f66dc2e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6ac98e70039623 server_cookie=f716aa7a05cd84da in_seq=0 out_seq=0 2023-10-18T12:32:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.955+0000 7f976dffb700 1 -- 172.21.15.70:0/4100970205 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a9f668f8b0 con 0x55a9f66d9ec0 2023-10-18T12:32:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.955+0000 7f976dffb700 10 client.26513 handle_command_reply: tid=0 2023-10-18T12:32:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.955+0000 7f979fe10b80 1 client.26513 shutdown 2023-10-18T12:32:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 msgr2=0x55a9f66c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 0x55a9f66c97c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9774016000 tx=0x7f9774009c30 comp rx=0 tx=0).stop 2023-10-18T12:32:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 shutdown_connections 2023-10-18T12:32:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f66d9ec0 0x55a9f66dc2e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c05d0 0x55a9f66cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f976dffb700 0 client.26513 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 --2- 172.21.15.70:0/4100970205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c5be0 0x55a9f66c97c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 >> 172.21.15.70:0/4100970205 conn(0x55a9f6634720 msgr2=0x55a9f66bf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 shutdown_connections 2023-10-18T12:32:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 1 -- 172.21.15.70:0/4100970205 wait complete. 2023-10-18T12:32:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.956+0000 7f979fe10b80 20 client.26513 trim_cache size 0 max 16384 2023-10-18T12:32:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.974+0000 7f979fe10b80 1 Processor -- start 2023-10-18T12:32:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.974+0000 7f979fe10b80 1 -- start start 2023-10-18T12:32:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f979fe10b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f66b5af0 con 0x55a9f66c6500 2023-10-18T12:32:56.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f979fe10b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f6493920 con 0x55a9f66c05d0 2023-10-18T12:32:56.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f979fe10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f66c95e0 con 0x55a9f66c5be0 2023-10-18T12:32:56.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48028/0 (socket says 172.21.15.70:48028) 2023-10-18T12:32:56.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 -- 172.21.15.70:0/3853531635 learned_addr learned my addr 172.21.15.70:0/3853531635 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a99c700 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978999a700 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 -- 172.21.15.70:0/3853531635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 msgr2=0x55a9f66d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 -- 172.21.15.70:0/3853531635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 msgr2=0x55a9f66cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a19b700 1 -- 172.21.15.70:0/3853531635 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f6426c60 con 0x55a9f66c05d0 2023-10-18T12:32:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.975+0000 7f978a99c700 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.976+0000 7f978a19b700 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66c90a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9780009e60 tx=0x7f9780009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf21f0583dcfe664 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.976+0000 7f97793ff700 1 -- 172.21.15.70:0/3853531635 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780029070 con 0x55a9f66c05d0 2023-10-18T12:32:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.976+0000 7f97793ff700 1 -- 172.21.15.70:0/3853531635 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9780005050 con 0x55a9f66c05d0 2023-10-18T12:32:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.976+0000 7f97793ff700 1 -- 172.21.15.70:0/3853531635 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780019690 con 0x55a9f66c05d0 2023-10-18T12:32:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 -- 172.21.15.70:0/3853531635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 msgr2=0x55a9f66c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66c90a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9780009e60 tx=0x7f9780009260 comp rx=0 tx=0).stop 2023-10-18T12:32:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 -- 172.21.15.70:0/3853531635 shutdown_connections 2023-10-18T12:32:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c6500 0x55a9f66cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66c90a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 --2- 172.21.15.70:0/3853531635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c5be0 0x55a9f66d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 -- 172.21.15.70:0/3853531635 >> 172.21.15.70:0/3853531635 conn(0x55a9f6634720 msgr2=0x55a9f66c09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 -- 172.21.15.70:0/3853531635 shutdown_connections 2023-10-18T12:32:56.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 -- 172.21.15.70:0/3853531635 wait complete. 2023-10-18T12:32:56.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 -- start start 2023-10-18T12:32:56.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.977+0000 7f979fe10b80 1 Processor -- start 2023-10-18T12:32:56.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c5be0 0x55a9f66cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c6500 0x55a9f66cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f66cf9f0 con 0x55a9f66c5be0 2023-10-18T12:32:56.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f66cfb60 con 0x55a9f66c05d0 2023-10-18T12:32:56.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f66cfd20 con 0x55a9f66c6500 2023-10-18T12:32:56.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f979fe10b80 10 client.0 _ll_register_callbacks cb 0x55a9f64f9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:56.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f978a19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f978a19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48030/0 (socket says 172.21.15.70:48030) 2023-10-18T12:32:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f978a19b700 1 -- 172.21.15.70:0/2289524188 learned_addr learned my addr 172.21.15.70:0/2289524188 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f978999a700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c5be0 0x55a9f66cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.978+0000 7f978a99c700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c6500 0x55a9f66cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f978a19b700 1 -- 172.21.15.70:0/2289524188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c6500 msgr2=0x55a9f66cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f978a19b700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c6500 0x55a9f66cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f978a19b700 1 -- 172.21.15.70:0/2289524188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c5be0 msgr2=0x55a9f66cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f978a19b700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c5be0 0x55a9f66cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f978a19b700 1 -- 172.21.15.70:0/2289524188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9f65d03f0 con 0x55a9f66c05d0 2023-10-18T12:32:56.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f978a19b700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66d0bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9780005350 tx=0x7f9780005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74bb7ed9b01df900 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f976dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:56.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f976dffb700 1 -- 172.21.15.70:0/2289524188 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780029040 con 0x55a9f66c05d0 2023-10-18T12:32:56.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9780019c50 con 0x55a9f66c05d0 2023-10-18T12:32:56.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f976dffb700 1 -- 172.21.15.70:0/2289524188 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9780019430 con 0x55a9f66c05d0 2023-10-18T12:32:56.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f976dffb700 1 -- 172.21.15.70:0/2289524188 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9780019610 con 0x55a9f66c05d0 2023-10-18T12:32:56.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f979fe10b80 10 client.26519 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:56.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.979+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a9f6464250 con 0x55a9f66c05d0 2023-10-18T12:32:56.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f976dffb700 1 -- 172.21.15.70:0/2289524188 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9780019c50 con 0x55a9f66c05d0 2023-10-18T12:32:56.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f976dffb700 1 -- 172.21.15.70:0/2289524188 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9780003680 con 0x55a9f66c05d0 2023-10-18T12:32:56.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f979fe10b80 10 client.26519 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:56.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f979fe10b80 10 client.26519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:56.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f979fe10b80 20 client.26519 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:56.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f672ee20 0x55a9f6731210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f979fe10b80 4 client.26519 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:32:56.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9f660be50 con 0x55a9f672ee20 2023-10-18T12:32:56.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.980+0000 7f978999a700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f672ee20 0x55a9f6731210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f978999a700 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f672ee20 0x55a9f6731210 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663504e82cd7d425 server_cookie=989c400111ed6b5f in_seq=0 out_seq=0 2023-10-18T12:32:56.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f976dffb700 10 client.26519 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f976dffb700 1 -- 172.21.15.70:0/2289524188 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a9f660be50 con 0x55a9f672ee20 2023-10-18T12:32:56.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f976dffb700 10 client.26519 handle_command_reply: tid=0 2023-10-18T12:32:56.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 client.26519 shutdown 2023-10-18T12:32:56.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 msgr2=0x55a9f66d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66d0bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9780005350 tx=0x7f9780005380 comp rx=0 tx=0).stop 2023-10-18T12:32:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 shutdown_connections 2023-10-18T12:32:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a9f672ee20 0x55a9f6731210 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9f66c6500 0x55a9f66cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a9f66c05d0 0x55a9f66d0bf0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9780005350 tx=0x7f9780005380 comp rx=0 tx=0).stop 2023-10-18T12:32:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f976dffb700 0 client.26519 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 --2- 172.21.15.70:0/2289524188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a9f66c5be0 0x55a9f66cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 >> 172.21.15.70:0/2289524188 conn(0x55a9f6634720 msgr2=0x55a9f66c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.981+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 shutdown_connections 2023-10-18T12:32:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f979fe10b80 1 -- 172.21.15.70:0/2289524188 wait complete. 2023-10-18T12:32:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f979fe10b80 20 client.26519 trim_cache size 0 max 16384 2023-10-18T12:32:56.062 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:32:56.062 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:32:56.062 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:32:56.063 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:32:56.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f97700a15f0 msgr2=0x7f97700a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f97700a15f0 0x7f97700a3ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f978419a480 tx=0x7f9774010ed0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 msgr2=0x7f978419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 0x7f978419d810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c110 tx=0x7f977c00c420 comp rx=0 tx=0).stop 2023-10-18T12:32:56.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 shutdown_connections 2023-10-18T12:32:56.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9784105b30 0x7f978419d810 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9784069290 0x7f9784198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f97700a15f0 0x7f97700a3ab0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 --2- 172.21.15.70:0/2945203744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784105170 0x7f9784199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 >> 172.21.15.70:0/2945203744 conn(0x7f9784076790 msgr2=0x7f978410bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.982+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 shutdown_connections 2023-10-18T12:32:56.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:55.983+0000 7f978c3ff700 1 -- 172.21.15.70:0/2945203744 wait complete. 2023-10-18T12:32:56.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:32:56.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:32:56.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/354001916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd81066b0 msgr2=0x7f3cd8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/354001916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd81066b0 0x7f3cd8106a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098f0 tx=0x7f3cd4009c00 comp rx=0 tx=0).stop 2023-10-18T12:32:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/354001916 shutdown_connections 2023-10-18T12:32:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/354001916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd8101060 0x7f3cd810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/354001916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd81066b0 0x7f3cd8106a90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/354001916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cd81006a0 0x7f3cd8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/354001916 >> 172.21.15.70:0/354001916 conn(0x7f3cd80fc170 msgr2=0x7f3cd80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/354001916 shutdown_connections 2023-10-18T12:32:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.284+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/354001916 wait complete. 2023-10-18T12:32:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 Processor -- start 2023-10-18T12:32:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 -- start start 2023-10-18T12:32:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 0x7f3cd8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd8101060 0x7f3cd8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cd81066b0 0x7f3cd819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd819cb30 con 0x7f3cd81066b0 2023-10-18T12:32:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd819cca0 con 0x7f3cd8101060 2023-10-18T12:32:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.286+0000 7f3cdf7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd819ce60 con 0x7f3cd81006a0 2023-10-18T12:32:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 0x7f3cd8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 0x7f3cd8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:43838/0 (socket says 172.21.15.70:43838) 2023-10-18T12:32:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 -- 172.21.15.70:0/4287217437 learned_addr learned my addr 172.21.15.70:0/4287217437 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdcd9a700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd8101060 0x7f3cd8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cddd9c700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cd81066b0 0x7f3cd819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 -- 172.21.15.70:0/4287217437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd8101060 msgr2=0x7f3cd8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd8101060 0x7f3cd8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 -- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cd81066b0 msgr2=0x7f3cd819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cd81066b0 0x7f3cd819c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 -- 172.21.15.70:0/4287217437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009520 con 0x7f3cd81006a0 2023-10-18T12:32:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdcd9a700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd8101060 0x7f3cd8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.287+0000 7f3cdd59b700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 0x7f3cd8198d00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005950 tx=0x7f3cd4010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.288+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd401c070 con 0x7f3cd81006a0 2023-10-18T12:32:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.288+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd400bba0 con 0x7f3cd81006a0 2023-10-18T12:32:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.288+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4003a00 con 0x7f3cd81006a0 2023-10-18T12:32:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.288+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd819d0e0 con 0x7f3cd81006a0 2023-10-18T12:32:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.288+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd81a5aa0 con 0x7f3cd81006a0 2023-10-18T12:32:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.289+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd8101de0 con 0x7f3cd81006a0 2023-10-18T12:32:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.291+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3cd4020030 con 0x7f3cd81006a0 2023-10-18T12:32:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.292+0000 7f3cce7fc700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cc40a1670 0x7f3cc40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.292+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3cd40ce1e0 con 0x7f3cd81006a0 2023-10-18T12:32:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.292+0000 7f3cdcd9a700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cc40a1670 0x7f3cc40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.295+0000 7f3cdcd9a700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cc40a1670 0x7f3cc40a3b30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3cd819a210 tx=0x7f3cc8009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.302+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3cd4097230 con 0x7f3cd81006a0 2023-10-18T12:32:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.610+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd8199fc0 con 0x7f3cd81006a0 2023-10-18T12:32:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.611+0000 7f3cce7fc700 1 -- 172.21.15.70:0/4287217437 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f3cd409a850 con 0x7f3cd81006a0 2023-10-18T12:32:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:32:56.614 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:32:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cc40a1670 msgr2=0x7f3cc40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cc40a1670 0x7f3cc40a3b30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3cd819a210 tx=0x7f3cc8009310 comp rx=0 tx=0).stop 2023-10-18T12:32:56.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 msgr2=0x7f3cd8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 0x7f3cd8198d00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005950 tx=0x7f3cd4010ff0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 shutdown_connections 2023-10-18T12:32:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cd81066b0 0x7f3cd819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cd81006a0 0x7f3cd8198d00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3cc40a1670 0x7f3cc40a3b30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 --2- 172.21.15.70:0/4287217437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cd8101060 0x7f3cd8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 >> 172.21.15.70:0/4287217437 conn(0x7f3cd80fc170 msgr2=0x7f3cd810bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 shutdown_connections 2023-10-18T12:32:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.614+0000 7f3cdf7ff700 1 -- 172.21.15.70:0/4287217437 wait complete. 2023-10-18T12:32:56.619 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:32:56.636 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:32:56.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 763680 12.04 MB/sec execute 205 sec latency 25.676 ms 2023-10-18T12:32:56.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.857+0000 7f15649ff700 1 -- 172.21.15.70:0/2209247078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560101100 msgr2=0x7f156010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.857+0000 7f15649ff700 1 --2- 172.21.15.70:0/2209247078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560101100 0x7f156010d610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1558009f90 tx=0x7f155800b330 comp rx=0 tx=0).stop 2023-10-18T12:32:56.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 -- 172.21.15.70:0/2209247078 shutdown_connections 2023-10-18T12:32:56.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 --2- 172.21.15.70:0/2209247078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560101100 0x7f156010d610 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 --2- 172.21.15.70:0/2209247078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560106750 0x7f1560106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 --2- 172.21.15.70:0/2209247078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560100740 0x7f1560100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 -- 172.21.15.70:0/2209247078 >> 172.21.15.70:0/2209247078 conn(0x7f15600fc230 msgr2=0x7f15600fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 -- 172.21.15.70:0/2209247078 shutdown_connections 2023-10-18T12:32:56.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.858+0000 7f15649ff700 1 -- 172.21.15.70:0/2209247078 wait complete. 2023-10-18T12:32:56.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 Processor -- start 2023-10-18T12:32:56.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 -- start start 2023-10-18T12:32:56.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 0x7f156019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560101100 0x7f156019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 0x7f1560196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560196ff0 con 0x7f1560100740 2023-10-18T12:32:56.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560197160 con 0x7f1560106750 2023-10-18T12:32:56.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.859+0000 7f15649ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560197320 con 0x7f1560101100 2023-10-18T12:32:56.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 0x7f156019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 0x7f1560196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 0x7f156019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60290/0 (socket says 172.21.15.70:60290) 2023-10-18T12:32:56.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 0x7f1560196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48076/0 (socket says 172.21.15.70:48076) 2023-10-18T12:32:56.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155e59c700 1 -- 172.21.15.70:0/2487225050 learned_addr learned my addr 172.21.15.70:0/2487225050 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155dd9b700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560101100 0x7f156019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 -- 172.21.15.70:0/2487225050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560101100 msgr2=0x7f156019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560101100 0x7f156019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 -- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 msgr2=0x7f156019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 0x7f156019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 -- 172.21.15.70:0/2487225050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540095f0 con 0x7f1560106750 2023-10-18T12:32:56.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155e59c700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 0x7f156019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155dd9b700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560101100 0x7f156019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:56.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.860+0000 7f155ed9d700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 0x7f1560196990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f155800b6c0 tx=0x7f1558008fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.861+0000 7f154f7fe700 1 -- 172.21.15.70:0/2487225050 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155800e040 con 0x7f1560106750 2023-10-18T12:32:56.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.861+0000 7f154f7fe700 1 -- 172.21.15.70:0/2487225050 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f155801d470 con 0x7f1560106750 2023-10-18T12:32:56.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.861+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1558009c40 con 0x7f1560106750 2023-10-18T12:32:56.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.861+0000 7f154f7fe700 1 -- 172.21.15.70:0/2487225050 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580052a0 con 0x7f1560106750 2023-10-18T12:32:56.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.861+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15601a3620 con 0x7f1560106750 2023-10-18T12:32:56.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.865+0000 7f154f7fe700 1 -- 172.21.15.70:0/2487225050 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f155801d5e0 con 0x7f1560106750 2023-10-18T12:32:56.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 Processor -- start 2023-10-18T12:32:56.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f154f7fe700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15480a15f0 0x7f15480a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f154f7fe700 1 -- 172.21.15.70:0/2487225050 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f15580ce840 con 0x7f1560106750 2023-10-18T12:32:56.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 -- start start 2023-10-18T12:32:56.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f155dd9b700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15480a15f0 0x7f15480a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 0x55c9518f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f05d0 0x55c9518f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9518636e0 con 0x55c9518f05d0 2023-10-18T12:32:56.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c951849c60 con 0x55c9518f5be0 2023-10-18T12:32:56.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.867+0000 7f1578364b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c951847580 con 0x55c9518f6500 2023-10-18T12:32:56.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 0x55c9518f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f05d0 0x55c9518f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 0x55c9518f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48080/0 (socket says 172.21.15.70:48080) 2023-10-18T12:32:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 -- 172.21.15.70:0/752736166 learned_addr learned my addr 172.21.15.70:0/752736166 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155dd9b700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15480a15f0 0x7f15480a3ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1560197b00 tx=0x7f1554009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155dd9b700 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 -- 172.21.15.70:0/752736166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 msgr2=0x55c9518f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 -- 172.21.15.70:0/752736166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f05d0 msgr2=0x55c9518f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f05d0 0x55c9518f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 -- 172.21.15.70:0/752736166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9518003f0 con 0x55c9518f5be0 2023-10-18T12:32:56.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155ed9d700 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f05d0 0x55c9518f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f155e59c700 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 0x55c9518f5fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1550011650 tx=0x7f1550011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8c9775f225eb84 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.868+0000 7f154cff9700 1 -- 172.21.15.70:0/752736166 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550019630 con 0x55c9518f5be0 2023-10-18T12:32:56.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f154cff9700 1 -- 172.21.15.70:0/752736166 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1550019c50 con 0x55c9518f5be0 2023-10-18T12:32:56.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f154cff9700 1 -- 172.21.15.70:0/752736166 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000eca0 con 0x55c9518f5be0 2023-10-18T12:32:56.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 -- 172.21.15.70:0/752736166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 msgr2=0x55c9518f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 0x55c9518f5fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1550011650 tx=0x7f1550011960 comp rx=0 tx=0).stop 2023-10-18T12:32:56.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 -- 172.21.15.70:0/752736166 shutdown_connections 2023-10-18T12:32:56.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f05d0 0x55c9518f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f5be0 0x55c9518f5fc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 --2- 172.21.15.70:0/752736166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 -- 172.21.15.70:0/752736166 >> 172.21.15.70:0/752736166 conn(0x55c951864720 msgr2=0x55c9518eab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 -- 172.21.15.70:0/752736166 shutdown_connections 2023-10-18T12:32:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.869+0000 7f1578364b80 1 -- 172.21.15.70:0/752736166 wait complete. 2023-10-18T12:32:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.870+0000 7f1578364b80 1 -- start start 2023-10-18T12:32:56.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.870+0000 7f1578364b80 1 Processor -- start 2023-10-18T12:32:56.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c9518fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f5be0 0x55c9518f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f6500 0x55c9518f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c951842e70 con 0x55c9518f6500 2023-10-18T12:32:56.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9517df710 con 0x55c9518f05d0 2023-10-18T12:32:56.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9517df3b0 con 0x55c9518f5be0 2023-10-18T12:32:56.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f1578364b80 10 client.0 _ll_register_callbacks cb 0x55c951729a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:56.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f155e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c9518fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f155dd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f5be0 0x55c9518f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f155e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c9518fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48084/0 (socket says 172.21.15.70:48084) 2023-10-18T12:32:56.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.871+0000 7f155e59c700 1 -- 172.21.15.70:0/3708861830 learned_addr learned my addr 172.21.15.70:0/3708861830 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155ed9d700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f6500 0x55c9518f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155e59c700 1 -- 172.21.15.70:0/3708861830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f5be0 msgr2=0x55c9518f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155e59c700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f5be0 0x55c9518f97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155e59c700 1 -- 172.21.15.70:0/3708861830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f6500 msgr2=0x55c9518f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155e59c700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f6500 0x55c9518f9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155e59c700 1 -- 172.21.15.70:0/3708861830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15601a3620 con 0x55c9518f05d0 2023-10-18T12:32:56.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155ed9d700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f6500 0x55c9518f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:32:56.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f155e59c700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c9518fc4b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1550011cb0 tx=0x7f15500046a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655f45140de4e402 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1545ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:56.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1545ffb700 1 -- 172.21.15.70:0/3708861830 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000ed30 con 0x55c9518f05d0 2023-10-18T12:32:56.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1545ffb700 1 -- 172.21.15.70:0/3708861830 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1550019dc0 con 0x55c9518f05d0 2023-10-18T12:32:56.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c951847ef0 con 0x55c9518f05d0 2023-10-18T12:32:56.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1545ffb700 1 -- 172.21.15.70:0/3708861830 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155000ee90 con 0x55c9518f05d0 2023-10-18T12:32:56.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1578364b80 10 client.26537 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:56.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.872+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f1550009e10 con 0x55c9518f05d0 2023-10-18T12:32:56.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1545ffb700 1 -- 172.21.15.70:0/3708861830 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c951847ef0 con 0x55c9518f05d0 2023-10-18T12:32:56.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1545ffb700 1 -- 172.21.15.70:0/3708861830 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1550029020 con 0x55c9518f05d0 2023-10-18T12:32:56.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1578364b80 10 client.26537 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:56.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1578364b80 10 client.26537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:56.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1578364b80 20 client.26537 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:56.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1578364b80 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951912480 0x55c9519148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1578364b80 4 client.26537 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:56.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9518bf8b0 con 0x55c951912480 2023-10-18T12:32:56.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.873+0000 7f155dd9b700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951912480 0x55c9519148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.874+0000 7f1545ffb700 10 client.26537 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.874+0000 7f155dd9b700 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951912480 0x55c9519148a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865e2c7b87fe6392 server_cookie=fdfcf7455dabe141 in_seq=0 out_seq=0 2023-10-18T12:32:56.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.874+0000 7f1545ffb700 1 -- 172.21.15.70:0/3708861830 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9518bf8b0 con 0x55c951912480 2023-10-18T12:32:56.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.874+0000 7f1545ffb700 10 client.26537 handle_command_reply: tid=0 2023-10-18T12:32:56.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 client.26537 shutdown 2023-10-18T12:32:56.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 msgr2=0x55c9518fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c9518fc4b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1550011cb0 tx=0x7f15500046a0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 shutdown_connections 2023-10-18T12:32:56.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951912480 0x55c9519148a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f6500 0x55c9518f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1545ffb700 0 client.26537 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c9518fc4b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 --2- 172.21.15.70:0/3708861830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f5be0 0x55c9518f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 >> 172.21.15.70:0/3708861830 conn(0x55c951864720 msgr2=0x55c9518ef080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 shutdown_connections 2023-10-18T12:32:56.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 1 -- 172.21.15.70:0/3708861830 wait complete. 2023-10-18T12:32:56.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.875+0000 7f1578364b80 20 client.26537 trim_cache size 0 max 16384 2023-10-18T12:32:56.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 Processor -- start 2023-10-18T12:32:56.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 -- start start 2023-10-18T12:32:56.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c951914870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c9518ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9516c3920 con 0x55c9518f5be0 2023-10-18T12:32:56.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9518e7230 con 0x55c9518f05d0 2023-10-18T12:32:56.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.890+0000 7f1578364b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9518f4330 con 0x55c9518f6500 2023-10-18T12:32:56.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c951914870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c951914870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:48092/0 (socket says 172.21.15.70:48092) 2023-10-18T12:32:56.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 -- 172.21.15.70:0/4271705700 learned_addr learned my addr 172.21.15.70:0/4271705700 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155ed9d700 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155dd9b700 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c9518ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 -- 172.21.15.70:0/4271705700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 msgr2=0x55c9518f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 -- 172.21.15.70:0/4271705700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 msgr2=0x55c9518ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c9518ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 -- 172.21.15.70:0/4271705700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c95166dde0 con 0x55c9518f05d0 2023-10-18T12:32:56.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155dd9b700 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c9518ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:56.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f155e59c700 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c951914870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f155000e8f0 tx=0x7f1550005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f560ba23ed3a083d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f154d7fa700 1 -- 172.21.15.70:0/4271705700 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550011ce0 con 0x55c9518f05d0 2023-10-18T12:32:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f154d7fa700 1 -- 172.21.15.70:0/4271705700 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1550003680 con 0x55c9518f05d0 2023-10-18T12:32:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.891+0000 7f154d7fa700 1 -- 172.21.15.70:0/4271705700 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550019a70 con 0x55c9518f05d0 2023-10-18T12:32:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 -- 172.21.15.70:0/4271705700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 msgr2=0x55c951914870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c951914870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f155000e8f0 tx=0x7f1550005a00 comp rx=0 tx=0).stop 2023-10-18T12:32:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 -- 172.21.15.70:0/4271705700 shutdown_connections 2023-10-18T12:32:56.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f6500 0x55c9518f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f05d0 0x55c951914870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 --2- 172.21.15.70:0/4271705700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c9518ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 -- 172.21.15.70:0/4271705700 >> 172.21.15.70:0/4271705700 conn(0x55c951864720 msgr2=0x55c9518f36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 -- 172.21.15.70:0/4271705700 shutdown_connections 2023-10-18T12:32:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.892+0000 7f1578364b80 1 -- 172.21.15.70:0/4271705700 wait complete. 2023-10-18T12:32:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.893+0000 7f1578364b80 1 -- start start 2023-10-18T12:32:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.893+0000 7f1578364b80 1 Processor -- start 2023-10-18T12:32:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.893+0000 7f1578364b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 0x55c95190ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f1578364b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c95190f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f1578364b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f6500 0x55c95190bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f1578364b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95190c070 con 0x55c9518f5be0 2023-10-18T12:32:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f1578364b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95190c1e0 con 0x55c9518f6500 2023-10-18T12:32:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f1578364b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95190c3a0 con 0x55c9518f05d0 2023-10-18T12:32:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f1578364b80 10 client.0 _ll_register_callbacks cb 0x55c951729a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 0x55c95190ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 0x55c95190ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:43888/0 (socket says 172.21.15.70:43888) 2023-10-18T12:32:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f6500 0x55c95190bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 -- 172.21.15.70:0/1607259796 learned_addr learned my addr 172.21.15.70:0/1607259796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155dd9b700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c95190f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 -- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f6500 msgr2=0x55c95190bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f6500 0x55c95190bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 -- 172.21.15.70:0/1607259796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 msgr2=0x55c95190f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c95190f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155e59c700 1 -- 172.21.15.70:0/1607259796 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1550009e10 con 0x55c9518f05d0 2023-10-18T12:32:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.894+0000 7f155ed9d700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f6500 0x55c95190bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f155e59c700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 0x55c95190ed50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1550011650 tx=0x7f1550003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eecc140ef545406b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1545ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1545ffb700 1 -- 172.21.15.70:0/1607259796 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000ed30 con 0x55c9518f05d0 2023-10-18T12:32:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1545ffb700 1 -- 172.21.15.70:0/1607259796 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1550000e40 con 0x55c9518f05d0 2023-10-18T12:32:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1550019c50 con 0x55c9518f05d0 2023-10-18T12:32:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1545ffb700 1 -- 172.21.15.70:0/1607259796 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1550011cb0 con 0x55c9518f05d0 2023-10-18T12:32:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1578364b80 10 client.26548 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c951656c60 con 0x55c9518f05d0 2023-10-18T12:32:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.895+0000 7f1545ffb700 1 -- 172.21.15.70:0/1607259796 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1550019c50 con 0x55c9518f05d0 2023-10-18T12:32:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1545ffb700 1 -- 172.21.15.70:0/1607259796 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1550029070 con 0x55c9518f05d0 2023-10-18T12:32:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1578364b80 10 client.26548 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1578364b80 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1578364b80 20 client.26548 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1578364b80 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951966e20 0x55c951969240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1578364b80 4 client.26548 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:32:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c95183be50 con 0x55c951966e20 2023-10-18T12:32:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f155dd9b700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951966e20 0x55c951969240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f1545ffb700 10 client.26548 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.896+0000 7f155dd9b700 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951966e20 0x55c951969240 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dcb505166fa5a3 server_cookie=57e83b6c3f8b13a2 in_seq=0 out_seq=0 2023-10-18T12:32:56.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1545ffb700 1 -- 172.21.15.70:0/1607259796 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c95183be50 con 0x55c951966e20 2023-10-18T12:32:56.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1545ffb700 10 client.26548 handle_command_reply: tid=0 2023-10-18T12:32:56.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 client.26548 shutdown 2023-10-18T12:32:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 msgr2=0x55c95190ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 0x55c95190ed50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1550011650 tx=0x7f1550003db0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 shutdown_connections 2023-10-18T12:32:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c951966e20 0x55c951969240 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9518f6500 0x55c95190bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9518f05d0 0x55c95190ed50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 --2- 172.21.15.70:0/1607259796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9518f5be0 0x55c95190f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1545ffb700 0 client.26548 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 >> 172.21.15.70:0/1607259796 conn(0x55c951864720 msgr2=0x55c9518e44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 shutdown_connections 2023-10-18T12:32:56.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 1 -- 172.21.15.70:0/1607259796 wait complete. 2023-10-18T12:32:56.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.897+0000 7f1578364b80 20 client.26548 trim_cache size 0 max 16384 2023-10-18T12:32:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:32:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15480a15f0 msgr2=0x7f15480a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15480a15f0 0x7f15480a3ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1560197b00 tx=0x7f1554009310 comp rx=0 tx=0).stop 2023-10-18T12:32:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 msgr2=0x7f1560196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 0x7f1560196990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f155800b6c0 tx=0x7f1558008fb0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 shutdown_connections 2023-10-18T12:32:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f15480a15f0 0x7f15480a3ab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1560106750 0x7f1560196990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1560100740 0x7f156019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 --2- 172.21.15.70:0/2487225050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1560101100 0x7f156019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 >> 172.21.15.70:0/2487225050 conn(0x7f15600fc230 msgr2=0x7f15600fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 shutdown_connections 2023-10-18T12:32:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:56.898+0000 7f15649ff700 1 -- 172.21.15.70:0/2487225050 wait complete. 2023-10-18T12:32:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:57 smithi070 ceph-mon[98639]: pgmap v522: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 216 op/s 2023-10-18T12:32:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:57 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4287217437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:32:57.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:57 smithi152 ceph-mon[134767]: pgmap v522: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 216 op/s 2023-10-18T12:32:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:57 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4287217437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:32:57.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:57 smithi186 ceph-mon[135253]: pgmap v522: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 216 op/s 2023-10-18T12:32:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:57 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4287217437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:32:57.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 765324 12.01 MB/sec execute 206 sec latency 29.350 ms 2023-10-18T12:32:57.922 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:32:57.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:32:58.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 -- 172.21.15.70:0/3400836687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 msgr2=0x7faa100ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 --2- 172.21.15.70:0/3400836687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa100ffac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa00009990 tx=0x7faa00009ca0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 -- 172.21.15.70:0/3400836687 shutdown_connections 2023-10-18T12:32:58.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 --2- 172.21.15.70:0/3400836687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa10100000 0x7faa1010c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 --2- 172.21.15.70:0/3400836687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101056e0 0x7faa10105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 --2- 172.21.15.70:0/3400836687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa100ffac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 -- 172.21.15.70:0/3400836687 >> 172.21.15.70:0/3400836687 conn(0x7faa100767f0 msgr2=0x7faa10076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 -- 172.21.15.70:0/3400836687 shutdown_connections 2023-10-18T12:32:58.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.150+0000 7faa17bff700 1 -- 172.21.15.70:0/3400836687 wait complete. 2023-10-18T12:32:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.151+0000 7faa17bff700 1 Processor -- start 2023-10-18T12:32:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.151+0000 7faa17bff700 1 -- start start 2023-10-18T12:32:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa17bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa10198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa17bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa10100000 0x7faa10199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa17bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa101056e0 0x7faa1019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa17bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1019dd20 con 0x7faa10100000 2023-10-18T12:32:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa17bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1019de90 con 0x7faa101056e0 2023-10-18T12:32:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa17bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1019e050 con 0x7faa100ff640 2023-10-18T12:32:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa10198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa10198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35594/0 (socket says 172.21.15.70:35594) 2023-10-18T12:32:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 -- 172.21.15.70:0/695971790 learned_addr learned my addr 172.21.15.70:0/695971790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1519a700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa10100000 0x7faa10199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1619c700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa101056e0 0x7faa1019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 -- 172.21.15.70:0/695971790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa101056e0 msgr2=0x7faa1019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa101056e0 0x7faa1019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 -- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa10100000 msgr2=0x7faa10199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa10100000 0x7faa10199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1599b700 1 -- 172.21.15.70:0/695971790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000095f0 con 0x7faa100ff640 2023-10-18T12:32:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.152+0000 7faa1619c700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa101056e0 0x7faa1019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.153+0000 7faa1599b700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa10198e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9a0 tx=0x7faa0c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.153+0000 7faa06ffd700 1 -- 172.21.15.70:0/695971790 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c009790 con 0x7faa100ff640 2023-10-18T12:32:58.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.153+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1019e330 con 0x7faa100ff640 2023-10-18T12:32:58.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.153+0000 7faa06ffd700 1 -- 172.21.15.70:0/695971790 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0c00cbd0 con 0x7faa100ff640 2023-10-18T12:32:58.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.153+0000 7faa06ffd700 1 -- 172.21.15.70:0/695971790 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c018680 con 0x7faa100ff640 2023-10-18T12:32:58.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.153+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa101a5b70 con 0x7faa100ff640 2023-10-18T12:32:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.156+0000 7faa06ffd700 1 -- 172.21.15.70:0/695971790 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7faa0c00cd40 con 0x7faa100ff640 2023-10-18T12:32:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.157+0000 7faa06ffd700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9fc0a1670 0x7fa9fc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.158+0000 7faa06ffd700 1 -- 172.21.15.70:0/695971790 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7faa0c014070 con 0x7faa100ff640 2023-10-18T12:32:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.158+0000 7faa1519a700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9fc0a1670 0x7fa9fc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.158+0000 7faa2b5bbb80 1 Processor -- start 2023-10-18T12:32:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.158+0000 7faa2b5bbb80 1 -- start start 2023-10-18T12:32:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22136920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa1519a700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9fc0a1670 0x7fa9fc0a3b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa1019a350 tx=0x7faa0000b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c221372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c220a1760 con 0x557c22136e60 2023-10-18T12:32:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22087ce0 con 0x557c2213cd90 2023-10-18T12:32:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22085600 con 0x557c2213c470 2023-10-18T12:32:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa1599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa1519a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa1599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35598/0 (socket says 172.21.15.70:35598) 2023-10-18T12:32:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa1599b700 1 -- 172.21.15.70:0/1627243976 learned_addr learned my addr 172.21.15.70:0/1627243976 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.159+0000 7faa1619c700 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c221372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7faa1519a700 1 -- 172.21.15.70:0/1627243976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 msgr2=0x557c2213c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7faa1519a700 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7faa1519a700 1 -- 172.21.15.70:0/1627243976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 msgr2=0x557c221372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7faa1519a700 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c221372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7faa1519a700 1 -- 172.21.15.70:0/1627243976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2203e470 con 0x557c2213cd90 2023-10-18T12:32:58.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7faa1519a700 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22136920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa00006010 tx=0x7faa00020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d565be50c099f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.160+0000 7fa9fbfff700 1 -- 172.21.15.70:0/1627243976 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00004500 con 0x557c2213cd90 2023-10-18T12:32:58.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7fa9fbfff700 1 -- 172.21.15.70:0/1627243976 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0001e040 con 0x557c2213cd90 2023-10-18T12:32:58.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7fa9fbfff700 1 -- 172.21.15.70:0/1627243976 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000f930 con 0x557c2213cd90 2023-10-18T12:32:58.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1627243976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 msgr2=0x557c22136920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22136920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa00006010 tx=0x7faa00020000 comp rx=0 tx=0).stop 2023-10-18T12:32:58.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1627243976 shutdown_connections 2023-10-18T12:32:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c221372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1627243976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22136920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.161+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1627243976 >> 172.21.15.70:0/1627243976 conn(0x557c220a27a0 msgr2=0x557c22128be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.162+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1627243976 shutdown_connections 2023-10-18T12:32:58.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.162+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1627243976 wait complete. 2023-10-18T12:32:58.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.162+0000 7faa2b5bbb80 1 -- start start 2023-10-18T12:32:58.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.162+0000 7faa2b5bbb80 1 Processor -- start 2023-10-18T12:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22142d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213c470 0x557c22140000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c22140610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22080ef0 con 0x557c22136e60 2023-10-18T12:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2201d790 con 0x557c2213c470 2023-10-18T12:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2201d430 con 0x557c2213cd90 2023-10-18T12:32:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa2b5bbb80 10 client.0 _ll_register_callbacks cb 0x557c21f67b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c22140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c22140610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35614/0 (socket says 172.21.15.70:35614) 2023-10-18T12:32:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 -- 172.21.15.70:0/1044041406 learned_addr learned my addr 172.21.15.70:0/1044041406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1519a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213c470 0x557c22140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1599b700 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 -- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213c470 msgr2=0x557c22140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213c470 0x557c22140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 -- 172.21.15.70:0/1044041406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 msgr2=0x557c22142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.163+0000 7faa1619c700 1 -- 172.21.15.70:0/1044041406 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa10067830 con 0x557c2213cd90 2023-10-18T12:32:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7faa1619c700 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c22140610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa08007de0 tx=0x7faa08007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2ea0b4b5ba9820 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7fa9f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1044041406 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800e070 con 0x557c2213cd90 2023-10-18T12:32:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c22085f70 con 0x557c2213cd90 2023-10-18T12:32:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1044041406 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7faa08004db0 con 0x557c2213cd90 2023-10-18T12:32:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1044041406 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa080128c0 con 0x557c2213cd90 2023-10-18T12:32:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7faa2b5bbb80 10 client.26560 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:58.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557c21eabe60 con 0x557c2213cd90 2023-10-18T12:32:58.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1044041406 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c22085f70 con 0x557c2213cd90 2023-10-18T12:32:58.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.164+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1044041406 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa0801c020 con 0x557c2213cd90 2023-10-18T12:32:58.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa2b5bbb80 10 client.26560 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:58.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa2b5bbb80 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:58.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa2b5bbb80 20 client.26560 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:58.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221506d0 0x557c22152af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa2b5bbb80 4 client.26560 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:58.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c220fd930 con 0x557c221506d0 2023-10-18T12:32:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa1599b700 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221506d0 0x557c22152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7fa9f97fa700 10 client.26560 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.165+0000 7faa1599b700 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221506d0 0x557c22152af0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955cd2df2cbfb7dc server_cookie=3a326ff621e03fc0 in_seq=0 out_seq=0 2023-10-18T12:32:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.166+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1044041406 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c220fd930 con 0x557c221506d0 2023-10-18T12:32:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.166+0000 7fa9f97fa700 10 client.26560 handle_command_reply: tid=0 2023-10-18T12:32:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.166+0000 7faa2b5bbb80 1 client.26560 shutdown 2023-10-18T12:32:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.166+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 msgr2=0x557c22140610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.166+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c22140610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa08007de0 tx=0x7faa08007e10 comp rx=0 tx=0).stop 2023-10-18T12:32:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 shutdown_connections 2023-10-18T12:32:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221506d0 0x557c22152af0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c22140610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1044041406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213c470 0x557c22140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7fa9f97fa700 0 client.26560 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 >> 172.21.15.70:0/1044041406 conn(0x557c220a27a0 msgr2=0x557c22135910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 shutdown_connections 2023-10-18T12:32:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1044041406 wait complete. 2023-10-18T12:32:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.167+0000 7faa2b5bbb80 20 client.26560 trim_cache size 0 max 16384 2023-10-18T12:32:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 Processor -- start 2023-10-18T12:32:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 -- start start 2023-10-18T12:32:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22147710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21f019a0 con 0x557c22136e60 2023-10-18T12:32:58.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2214dd90 con 0x557c2213cd90 2023-10-18T12:32:58.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.186+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22140f20 con 0x557c2213c470 2023-10-18T12:32:58.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1519a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1519a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35616/0 (socket says 172.21.15.70:35616) 2023-10-18T12:32:58.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56220/0 (socket says 172.21.15.70:56220) 2023-10-18T12:32:58.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1519a700 1 -- 172.21.15.70:0/3026947525 learned_addr learned my addr 172.21.15.70:0/3026947525 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1599b700 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 -- 172.21.15.70:0/3026947525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 msgr2=0x557c2213ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 -- 172.21.15.70:0/3026947525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 msgr2=0x557c22147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 -- 172.21.15.70:0/3026947525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c221576e0 con 0x557c2213cd90 2023-10-18T12:32:58.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1519a700 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:58.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa1619c700 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22140500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa08002230 tx=0x7faa08009170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9cee9bd576ef92f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.187+0000 7faa04bff700 1 -- 172.21.15.70:0/3026947525 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800e040 con 0x557c2213cd90 2023-10-18T12:32:58.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa04bff700 1 -- 172.21.15.70:0/3026947525 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0800f040 con 0x557c2213cd90 2023-10-18T12:32:58.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa04bff700 1 -- 172.21.15.70:0/3026947525 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0801f440 con 0x557c2213cd90 2023-10-18T12:32:58.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/3026947525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 msgr2=0x557c22140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22140500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa08002230 tx=0x7faa08009170 comp rx=0 tx=0).stop 2023-10-18T12:32:58.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/3026947525 shutdown_connections 2023-10-18T12:32:58.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c2213cd90 0x557c22140500 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c22136e60 0x557c22147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/3026947525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213c470 0x557c2213ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/3026947525 >> 172.21.15.70:0/3026947525 conn(0x557c220a27a0 msgr2=0x557c22128890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/3026947525 shutdown_connections 2023-10-18T12:32:58.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.188+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/3026947525 wait complete. 2023-10-18T12:32:58.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 -- start start 2023-10-18T12:32:58.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 Processor -- start 2023-10-18T12:32:58.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 0x557c22146880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c2213c470 0x557c22146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c2219e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2219ea50 con 0x557c2213c470 2023-10-18T12:32:58.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2219ebc0 con 0x557c22136e60 2023-10-18T12:32:58.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2219ed80 con 0x557c2213cd90 2023-10-18T12:32:58.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.189+0000 7faa2b5bbb80 10 client.0 _ll_register_callbacks cb 0x557c21f67b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:58.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 0x557c22146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 0x557c22146880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56222/0 (socket says 172.21.15.70:56222) 2023-10-18T12:32:58.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 -- 172.21.15.70:0/1204745983 learned_addr learned my addr 172.21.15.70:0/1204745983 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1519a700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c2213c470 0x557c22146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1619c700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c2219e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 -- 172.21.15.70:0/1204745983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 msgr2=0x557c2219e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c2219e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 -- 172.21.15.70:0/1204745983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c2213c470 msgr2=0x557c22146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c2213c470 0x557c22146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 -- 172.21.15.70:0/1204745983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c2203e470 con 0x557c22136e60 2023-10-18T12:32:58.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7faa1599b700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 0x557c22146880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faa0c09a180 tx=0x7faa0c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f2017738b54eb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.190+0000 7fa9f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:32:58.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1204745983 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0cd3e0 con 0x557c22136e60 2023-10-18T12:32:58.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1204745983 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7faa0c09e150 con 0x557c22136e60 2023-10-18T12:32:58.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa0001e040 con 0x557c22136e60 2023-10-18T12:32:58.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1204745983 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa0c09a4a0 con 0x557c22136e60 2023-10-18T12:32:58.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7faa2b5bbb80 10 client.26561 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:58.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557c21eabe60 con 0x557c22136e60 2023-10-18T12:32:58.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1204745983 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0001e040 con 0x557c22136e60 2023-10-18T12:32:58.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1204745983 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa0c0a2140 con 0x557c22136e60 2023-10-18T12:32:58.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7faa2b5bbb80 10 client.26561 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:58.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7faa2b5bbb80 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:58.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.191+0000 7faa2b5bbb80 20 client.26561 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:58.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.192+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221a6650 0x557c221a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.192+0000 7faa2b5bbb80 4 client.26561 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:32:58.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.192+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557c22079ed0 con 0x557c221a6650 2023-10-18T12:32:58.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.192+0000 7faa1519a700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221a6650 0x557c221a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.192+0000 7fa9f97fa700 10 client.26561 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.192+0000 7faa1519a700 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221a6650 0x557c221a8a40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd318c3d0b5491a server_cookie=7de109740ae5228b in_seq=0 out_seq=0 2023-10-18T12:32:58.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7fa9f97fa700 1 -- 172.21.15.70:0/1204745983 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557c22079ed0 con 0x557c221a6650 2023-10-18T12:32:58.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7fa9f97fa700 10 client.26561 handle_command_reply: tid=0 2023-10-18T12:32:58.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 client.26561 shutdown 2023-10-18T12:32:58.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 msgr2=0x557c22146880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 0x557c22146880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faa0c09a180 tx=0x7faa0c00c6e0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 shutdown_connections 2023-10-18T12:32:58.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557c221a6650 0x557c221a8a40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c2213cd90 0x557c2219e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c22136e60 0x557c22146880 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 --2- 172.21.15.70:0/1204745983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c2213c470 0x557c22146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.197+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 >> 172.21.15.70:0/1204745983 conn(0x557c220a27a0 msgr2=0x557c22122530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.198+0000 7fa9f97fa700 0 client.26561 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.198+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 shutdown_connections 2023-10-18T12:32:58.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.198+0000 7faa2b5bbb80 1 -- 172.21.15.70:0/1204745983 wait complete. 2023-10-18T12:32:58.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.198+0000 7faa2b5bbb80 20 client.26561 trim_cache size 0 max 16384 2023-10-18T12:32:58.211 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:32:58.212 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:32:58.212 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "5d8d4585-e9c8-4180-822e-9e5b4e8c42e2", 2023-10-18T12:32:58.212 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:32:58.212 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:32:58.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.199+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9fc0a1670 msgr2=0x7fa9fc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.199+0000 7faa17bff700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9fc0a1670 0x7fa9fc0a3b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa1019a350 tx=0x7faa0000b260 comp rx=0 tx=0).stop 2023-10-18T12:32:58.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.199+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 msgr2=0x7faa10198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.199+0000 7faa17bff700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa10198e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9a0 tx=0x7faa0c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 shutdown_connections 2023-10-18T12:32:58.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9fc0a1670 0x7fa9fc0a3b30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa101056e0 0x7faa1019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa100ff640 0x7faa10198e00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 --2- 172.21.15.70:0/695971790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa10100000 0x7faa10199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 >> 172.21.15.70:0/695971790 conn(0x7faa100767f0 msgr2=0x7faa1010d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.200+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 shutdown_connections 2023-10-18T12:32:58.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.201+0000 7faa17bff700 1 -- 172.21.15.70:0/695971790 wait complete. 2023-10-18T12:32:58.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:32:58.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 -- 172.21.15.70:0/3319191644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728072c20 msgr2=0x7fa728073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 --2- 172.21.15.70:0/3319191644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728072c20 0x7fa728073000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa718009990 tx=0x7fa718009ca0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 -- 172.21.15.70:0/3319191644 shutdown_connections 2023-10-18T12:32:58.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 --2- 172.21.15.70:0/3319191644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72810cf60 0x7fa72810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 --2- 172.21.15.70:0/3319191644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728072c20 0x7fa728073000 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 --2- 172.21.15.70:0/3319191644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728073540 0x7fa72810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 -- 172.21.15.70:0/3319191644 >> 172.21.15.70:0/3319191644 conn(0x7fa7280fc1b0 msgr2=0x7fa7280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.445+0000 7fa72dbff700 1 -- 172.21.15.70:0/3319191644 shutdown_connections 2023-10-18T12:32:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.446+0000 7fa72dbff700 1 -- 172.21.15.70:0/3319191644 wait complete. 2023-10-18T12:32:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.446+0000 7fa72dbff700 1 Processor -- start 2023-10-18T12:32:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 -- start start 2023-10-18T12:32:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728072c20 0x7fa728198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 0x7fa7281993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72810cf60 0x7fa72819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819ddc0 con 0x7fa72810cf60 2023-10-18T12:32:58.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819df30 con 0x7fa728073540 2023-10-18T12:32:58.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa72dbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819e0f0 con 0x7fa728072c20 2023-10-18T12:32:58.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 0x7fa7281993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 0x7fa7281993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:56240/0 (socket says 172.21.15.70:56240) 2023-10-18T12:32:58.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa726ffd700 1 -- 172.21.15.70:0/1174328775 learned_addr learned my addr 172.21.15.70:0/1174328775 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa7277fe700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728072c20 0x7fa728198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.447+0000 7fa727fff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72810cf60 0x7fa72819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa726ffd700 1 -- 172.21.15.70:0/1174328775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728072c20 msgr2=0x7fa728198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa726ffd700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728072c20 0x7fa728198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa726ffd700 1 -- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72810cf60 msgr2=0x7fa72819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa726ffd700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72810cf60 0x7fa72819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa726ffd700 1 -- 172.21.15.70:0/1174328775 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180095f0 con 0x7fa728073540 2023-10-18T12:32:58.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa7277fe700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728072c20 0x7fa728198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:58.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa726ffd700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 0x7fa7281993e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00e9a0 tx=0x7fa71c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa724ff9700 1 -- 172.21.15.70:0/1174328775 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c009790 con 0x7fa728073540 2023-10-18T12:32:58.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa724ff9700 1 -- 172.21.15.70:0/1174328775 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71c010430 con 0x7fa728073540 2023-10-18T12:32:58.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72819e3d0 con 0x7fa728073540 2023-10-18T12:32:58.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa724ff9700 1 -- 172.21.15.70:0/1174328775 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c018680 con 0x7fa728073540 2023-10-18T12:32:58.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.448+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7281a5c10 con 0x7fa728073540 2023-10-18T12:32:58.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.451+0000 7fa724ff9700 1 -- 172.21.15.70:0/1174328775 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa71c0105a0 con 0x7fa728073540 2023-10-18T12:32:58.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.453+0000 7fa741483b80 1 Processor -- start 2023-10-18T12:32:58.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.453+0000 7fa741483b80 1 -- start start 2023-10-18T12:32:58.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.453+0000 7fa741483b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a2be0 0x5625817a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.453+0000 7fa741483b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x56258179d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa724ff9700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7100a1670 0x7fa7100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa741483b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x56258179da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa741483b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625817106e0 con 0x5625817a2be0 2023-10-18T12:32:58.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa741483b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625816f6c60 con 0x56258179d5d0 2023-10-18T12:32:58.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa741483b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625816f4580 con 0x5625817a3500 2023-10-18T12:32:58.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa727fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a2be0 0x5625817a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa724ff9700 1 -- 172.21.15.70:0/1174328775 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa71c014070 con 0x7fa728073540 2023-10-18T12:32:58.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa727fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a2be0 0x5625817a2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60044/0 (socket says 172.21.15.70:60044) 2023-10-18T12:32:58.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa727fff700 1 -- 172.21.15.70:0/1108673945 learned_addr learned my addr 172.21.15.70:0/1108673945 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa726ffd700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7100a1670 0x7fa7100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa7277fe700 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x56258179d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa727fff700 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x56258179da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa7277fe700 1 -- 172.21.15.70:0/1108673945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 msgr2=0x56258179da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa7277fe700 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x56258179da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa7277fe700 1 -- 172.21.15.70:0/1108673945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a2be0 msgr2=0x5625817a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa7277fe700 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a2be0 0x5625817a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa7277fe700 1 -- 172.21.15.70:0/1108673945 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625816ad3f0 con 0x5625817a3500 2023-10-18T12:32:58.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.454+0000 7fa727fff700 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x56258179da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:58.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.455+0000 7fa726ffd700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7100a1670 0x7fa7100a3b30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c510 tx=0x7fa71c010b80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.455+0000 7fa7277fe700 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x56258179d090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098e0 tx=0x7fa718005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aed3f308feb68f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.455+0000 7fa70dffb700 1 -- 172.21.15.70:0/1108673945 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71801d070 con 0x5625817a3500 2023-10-18T12:32:58.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.455+0000 7fa70dffb700 1 -- 172.21.15.70:0/1108673945 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71801e040 con 0x5625817a3500 2023-10-18T12:32:58.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.455+0000 7fa70dffb700 1 -- 172.21.15.70:0/1108673945 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718004bd0 con 0x5625817a3500 2023-10-18T12:32:58.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 -- 172.21.15.70:0/1108673945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 msgr2=0x56258179d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x56258179d090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098e0 tx=0x7fa718005f00 comp rx=0 tx=0).stop 2023-10-18T12:32:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 -- 172.21.15.70:0/1108673945 shutdown_connections 2023-10-18T12:32:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x56258179da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a2be0 0x5625817a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 --2- 172.21.15.70:0/1108673945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x56258179d090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 -- 172.21.15.70:0/1108673945 >> 172.21.15.70:0/1108673945 conn(0x562581711720 msgr2=0x562581797b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 -- 172.21.15.70:0/1108673945 shutdown_connections 2023-10-18T12:32:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 -- 172.21.15.70:0/1108673945 wait complete. 2023-10-18T12:32:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.456+0000 7fa741483b80 1 -- start start 2023-10-18T12:32:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 Processor -- start 2023-10-18T12:32:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x5625817a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a3500 0x5625817a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625816efe70 con 0x5625817a3500 2023-10-18T12:32:58.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258168c710 con 0x56258179d5d0 2023-10-18T12:32:58.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258168c3b0 con 0x5625817a2be0 2023-10-18T12:32:58.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa741483b80 10 client.0 _ll_register_callbacks cb 0x5625815d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:58.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa727fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x5625817a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35670/0 (socket says 172.21.15.70:35670) 2023-10-18T12:32:58.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.457+0000 7fa7277fe700 1 -- 172.21.15.70:0/2478513068 learned_addr learned my addr 172.21.15.70:0/2478513068 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa727fff700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a3500 0x5625817a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa7277fe700 1 -- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 msgr2=0x5625817a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa7277fe700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x5625817a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa7277fe700 1 -- 172.21.15.70:0/2478513068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a3500 msgr2=0x5625817a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa7277fe700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a3500 0x5625817a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa7277fe700 1 -- 172.21.15.70:0/2478513068 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728110090 con 0x5625817a2be0 2023-10-18T12:32:58.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa727fff700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a3500 0x5625817a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:32:58.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa7277fe700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817a6830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa71800fb80 tx=0x7fa71800fbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c1497474774512 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa706ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:58.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa706ffd700 1 -- 172.21.15.70:0/2478513068 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71801d040 con 0x5625817a2be0 2023-10-18T12:32:58.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625816f4ef0 con 0x5625817a2be0 2023-10-18T12:32:58.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.458+0000 7fa706ffd700 1 -- 172.21.15.70:0/2478513068 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa718005250 con 0x5625817a2be0 2023-10-18T12:32:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa706ffd700 1 -- 172.21.15.70:0/2478513068 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa718005bc0 con 0x5625817a2be0 2023-10-18T12:32:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 10 client.26578 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56258151ade0 con 0x5625817a2be0 2023-10-18T12:32:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa706ffd700 1 -- 172.21.15.70:0/2478513068 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625816f4ef0 con 0x5625817a2be0 2023-10-18T12:32:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa706ffd700 1 -- 172.21.15.70:0/2478513068 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa718022400 con 0x5625817a2be0 2023-10-18T12:32:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 10 client.26578 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 10 client.26578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 20 client.26578 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5625817b6b40 0x5625817b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 4 client.26578 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:32:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56258176c8b0 con 0x5625817b6b40 2023-10-18T12:32:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.459+0000 7fa727fff700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5625817b6b40 0x5625817b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.460+0000 7fa727fff700 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5625817b6b40 0x5625817b8f60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d61a6dfd6a032b9 server_cookie=e94b62a9034ac14a in_seq=0 out_seq=0 2023-10-18T12:32:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.460+0000 7fa706ffd700 10 client.26578 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa706ffd700 1 -- 172.21.15.70:0/2478513068 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56258176c8b0 con 0x5625817b6b40 2023-10-18T12:32:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa706ffd700 10 client.26578 handle_command_reply: tid=0 2023-10-18T12:32:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 client.26578 shutdown 2023-10-18T12:32:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 msgr2=0x5625817a6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817a6830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa71800fb80 tx=0x7fa71800fbb0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 shutdown_connections 2023-10-18T12:32:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5625817b6b40 0x5625817b8f60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5625817a3500 0x5625817a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa706ffd700 0 client.26578 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56258179d5d0 0x5625817a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 --2- 172.21.15.70:0/2478513068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817a6830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 >> 172.21.15.70:0/2478513068 conn(0x562581711720 msgr2=0x56258179c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 shutdown_connections 2023-10-18T12:32:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 1 -- 172.21.15.70:0/2478513068 wait complete. 2023-10-18T12:32:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.461+0000 7fa741483b80 20 client.26578 trim_cache size 0 max 16384 2023-10-18T12:32:58.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.479+0000 7fa741483b80 1 Processor -- start 2023-10-18T12:32:58.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 -- start start 2023-10-18T12:32:58.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a2be0 0x5625817ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x5625817a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562581570920 con 0x56258179d5d0 2023-10-18T12:32:58.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625817a7dc0 con 0x5625817a2be0 2023-10-18T12:32:58.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa741483b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258178ffd0 con 0x5625817a3500 2023-10-18T12:32:58.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa727fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa727fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60060/0 (socket says 172.21.15.70:60060) 2023-10-18T12:32:58.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa727fff700 1 -- 172.21.15.70:0/522861657 learned_addr learned my addr 172.21.15.70:0/522861657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa7277fe700 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a2be0 0x5625817ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.480+0000 7fa727fff700 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x5625817a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa727fff700 1 -- 172.21.15.70:0/522861657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 msgr2=0x5625817a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa727fff700 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x5625817a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa727fff700 1 -- 172.21.15.70:0/522861657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a2be0 msgr2=0x5625817ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa727fff700 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a2be0 0x5625817ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa727fff700 1 -- 172.21.15.70:0/522861657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562581503c60 con 0x56258179d5d0 2023-10-18T12:32:58.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa7277fe700 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a2be0 0x5625817ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:58.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa727fff700 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817a0dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5625817b9800 tx=0x7fa720009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242a511a1bbc67e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa70e7fc700 1 -- 172.21.15.70:0/522861657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000e040 con 0x56258179d5d0 2023-10-18T12:32:58.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa70e7fc700 1 -- 172.21.15.70:0/522861657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72002b020 con 0x56258179d5d0 2023-10-18T12:32:58.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.481+0000 7fa70e7fc700 1 -- 172.21.15.70:0/522861657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720007500 con 0x56258179d5d0 2023-10-18T12:32:58.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 -- 172.21.15.70:0/522861657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 msgr2=0x5625817a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817a0dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5625817b9800 tx=0x7fa720009920 comp rx=0 tx=0).stop 2023-10-18T12:32:58.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 -- 172.21.15.70:0/522861657 shutdown_connections 2023-10-18T12:32:58.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a3500 0x5625817a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817a0dc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 --2- 172.21.15.70:0/522861657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a2be0 0x5625817ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 -- 172.21.15.70:0/522861657 >> 172.21.15.70:0/522861657 conn(0x562581711720 msgr2=0x5625817a06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 -- 172.21.15.70:0/522861657 shutdown_connections 2023-10-18T12:32:58.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 -- 172.21.15.70:0/522861657 wait complete. 2023-10-18T12:32:58.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 -- start start 2023-10-18T12:32:58.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.482+0000 7fa741483b80 1 Processor -- start 2023-10-18T12:32:58.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a3500 0x5625817acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258180a940 con 0x56258179d5d0 2023-10-18T12:32:58.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258180aa80 con 0x5625817a3500 2023-10-18T12:32:58.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258180abc0 con 0x5625817a2be0 2023-10-18T12:32:58.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa741483b80 10 client.0 _ll_register_callbacks cb 0x5625815d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:32:58.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa727fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35686/0 (socket says 172.21.15.70:35686) 2023-10-18T12:32:58.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa7277fe700 1 -- 172.21.15.70:0/2244783188 learned_addr learned my addr 172.21.15.70:0/2244783188 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:32:58.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.483+0000 7fa727fff700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a3500 0x5625817acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa7277fe700 1 -- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a3500 msgr2=0x5625817acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa7277fe700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a3500 0x5625817acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa7277fe700 1 -- 172.21.15.70:0/2244783188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 msgr2=0x5625817ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:32:58.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa7277fe700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa7277fe700 1 -- 172.21.15.70:0/2244783188 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56258151ade0 con 0x5625817a2be0 2023-10-18T12:32:58.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa727fff700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:32:58.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa7277fe700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817aca80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa71800fb50 tx=0x7fa718022630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a3f844eb727eb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:32:58.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa706ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:32:58.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa706ffd700 1 -- 172.21.15.70:0/2244783188 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71801d070 con 0x5625817a2be0 2023-10-18T12:32:58.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa71801e040 con 0x5625817a2be0 2023-10-18T12:32:58.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa706ffd700 1 -- 172.21.15.70:0/2244783188 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa718000e40 con 0x5625817a2be0 2023-10-18T12:32:58.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa706ffd700 1 -- 172.21.15.70:0/2244783188 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa718005bc0 con 0x5625817a2be0 2023-10-18T12:32:58.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa741483b80 10 client.26584 fetch_fsmap learned FSMap version 20 2023-10-18T12:32:58.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.484+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa720023040 con 0x5625817a2be0 2023-10-18T12:32:58.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa706ffd700 1 -- 172.21.15.70:0/2244783188 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71801e040 con 0x5625817a2be0 2023-10-18T12:32:58.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa706ffd700 1 -- 172.21.15.70:0/2244783188 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa718005550 con 0x5625817a2be0 2023-10-18T12:32:58.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa741483b80 10 client.26584 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:32:58.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa741483b80 10 client.26584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:32:58.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa741483b80 20 client.26584 populate_metadata read hostname 'smithi070' 2023-10-18T12:32:58.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa741483b80 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56258180bec0 0x56258180e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:32:58.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa741483b80 4 client.26584 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:32:58.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625816e8e50 con 0x56258180bec0 2023-10-18T12:32:58.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa727fff700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56258180bec0 0x56258180e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:32:58.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa727fff700 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56258180bec0 0x56258180e2b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acad8c9d3921bab2 server_cookie=158403dc46234af6 in_seq=0 out_seq=0 2023-10-18T12:32:58.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.485+0000 7fa706ffd700 10 client.26584 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa706ffd700 1 -- 172.21.15.70:0/2244783188 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5625816e8e50 con 0x56258180bec0 2023-10-18T12:32:58.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa706ffd700 10 client.26584 handle_command_reply: tid=0 2023-10-18T12:32:58.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 client.26584 shutdown 2023-10-18T12:32:58.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 msgr2=0x5625817aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817aca80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa71800fb50 tx=0x7fa718022630 comp rx=0 tx=0).stop 2023-10-18T12:32:58.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 shutdown_connections 2023-10-18T12:32:58.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56258180bec0 0x56258180e2b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5625817a3500 0x5625817acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa706ffd700 0 client.26584 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:32:58.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258179d5d0 0x5625817ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 --2- 172.21.15.70:0/2244783188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625817a2be0 0x5625817aca80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 >> 172.21.15.70:0/2244783188 conn(0x562581711720 msgr2=0x5625817914b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 shutdown_connections 2023-10-18T12:32:58.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 1 -- 172.21.15.70:0/2244783188 wait complete. 2023-10-18T12:32:58.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.486+0000 7fa741483b80 20 client.26584 trim_cache size 0 max 16384 2023-10-18T12:32:58.507 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:32:58.508 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:32:58.508 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:32:58.508 INFO:teuthology.orchestra.run.smithi070.stdout: "5d8d4585-e9c8-4180-822e-9e5b4e8c42e2": { 2023-10-18T12:32:58.508 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:32:58.508 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "5d8d4585-e9c8-4180-822e-9e5b4e8c42e2", 2023-10-18T12:32:58.509 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:32:58.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:32:58.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:32:58.509 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:32:58.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7100a1670 msgr2=0x7fa7100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7100a1670 0x7fa7100a3b30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c510 tx=0x7fa71c010b80 comp rx=0 tx=0).stop 2023-10-18T12:32:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 msgr2=0x7fa7281993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:32:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 0x7fa7281993e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00e9a0 tx=0x7fa71c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 shutdown_connections 2023-10-18T12:32:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72810cf60 0x7fa72819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa728072c20 0x7fa728198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa7100a1670 0x7fa7100a3b30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 --2- 172.21.15.70:0/1174328775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa728073540 0x7fa7281993e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:32:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 >> 172.21.15.70:0/1174328775 conn(0x7fa7280fc1b0 msgr2=0x7fa7280fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:32:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 shutdown_connections 2023-10-18T12:32:58.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:32:58.487+0000 7fa72dbff700 1 -- 172.21.15.70:0/1174328775 wait complete. 2023-10-18T12:32:58.513 INFO:tasks.cephfs.filesystem:scrub status for tag:5d8d4585-e9c8-4180-822e-9e5b4e8c42e2 - {'path': '/', 'tag': '5d8d4585-e9c8-4180-822e-9e5b4e8c42e2', 'options': 'recursive,force'} 2023-10-18T12:32:58.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:58 smithi070 ceph-mon[98639]: pgmap v523: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-10-18T12:32:58.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:58 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:32:58.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:58 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:58.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:58 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:32:58.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:32:58 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:58.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:58 smithi152 ceph-mon[134767]: pgmap v523: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-10-18T12:32:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:58 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:32:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:58 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:58 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:32:58.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:32:58 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:58.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:58 smithi186 ceph-mon[135253]: pgmap v523: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 230 op/s 2023-10-18T12:32:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:58 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:32:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:58 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:58 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:32:58.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:32:58 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:32:58.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 767553 12.02 MB/sec execute 207 sec latency 16.592 ms 2023-10-18T12:32:59.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 769157 11.99 MB/sec execute 208 sec latency 75.330 ms 2023-10-18T12:33:00.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 770215 11.97 MB/sec execute 209 sec latency 144.349 ms 2023-10-18T12:33:01.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:01 smithi070 ceph-mon[98639]: pgmap v524: 129 pgs: 129 active+clean; 420 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 222 op/s 2023-10-18T12:33:01.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:01 smithi152 ceph-mon[134767]: pgmap v524: 129 pgs: 129 active+clean; 420 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 222 op/s 2023-10-18T12:33:01.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:01 smithi186 ceph-mon[135253]: pgmap v524: 129 pgs: 129 active+clean; 420 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 222 op/s 2023-10-18T12:33:01.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 771412 11.94 MB/sec execute 210 sec latency 68.598 ms 2023-10-18T12:33:02.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:02 smithi070 ceph-mon[98639]: pgmap v525: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 263 op/s 2023-10-18T12:33:02.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:02 smithi152 ceph-mon[134767]: pgmap v525: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 263 op/s 2023-10-18T12:33:02.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:02 smithi186 ceph-mon[135253]: pgmap v525: 129 pgs: 129 active+clean; 424 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 263 op/s 2023-10-18T12:33:02.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 774203 11.96 MB/sec execute 211 sec latency 12.654 ms 2023-10-18T12:33:03.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:03 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:33:03.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:03 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:33:03.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:03 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:33:03.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 775752 11.94 MB/sec execute 212 sec latency 38.214 ms 2023-10-18T12:33:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:04 smithi070 ceph-mon[98639]: osdmap e94: 12 total, 12 up, 12 in 2023-10-18T12:33:04.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:04 smithi070 ceph-mon[98639]: pgmap v527: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 311 op/s 2023-10-18T12:33:04.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:04 smithi152 ceph-mon[134767]: osdmap e94: 12 total, 12 up, 12 in 2023-10-18T12:33:04.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:04 smithi152 ceph-mon[134767]: pgmap v527: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 311 op/s 2023-10-18T12:33:04.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:04 smithi186 ceph-mon[135253]: osdmap e94: 12 total, 12 up, 12 in 2023-10-18T12:33:04.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:04 smithi186 ceph-mon[135253]: pgmap v527: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 311 op/s 2023-10-18T12:33:04.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 778001 11.95 MB/sec execute 213 sec latency 71.671 ms 2023-10-18T12:33:05.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 779937 11.92 MB/sec execute 214 sec latency 26.830 ms 2023-10-18T12:33:06.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 782062 11.93 MB/sec execute 215 sec latency 12.662 ms 2023-10-18T12:33:07.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:06 smithi070 ceph-mon[98639]: pgmap v528: 129 pgs: 129 active+clean; 432 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 262 op/s 2023-10-18T12:33:07.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:06 smithi152 ceph-mon[134767]: pgmap v528: 129 pgs: 129 active+clean; 432 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 262 op/s 2023-10-18T12:33:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:06 smithi186 ceph-mon[135253]: pgmap v528: 129 pgs: 129 active+clean; 432 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 262 op/s 2023-10-18T12:33:07.856 INFO:tasks.workunit.client.0.smithi070.stdout: 1 784897 11.96 MB/sec execute 216 sec latency 39.192 ms 2023-10-18T12:33:08.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 786543 11.93 MB/sec execute 217 sec latency 18.947 ms 2023-10-18T12:33:09.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:08 smithi070 ceph-mon[98639]: pgmap v529: 129 pgs: 129 active+clean; 433 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2023-10-18T12:33:09.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:08 smithi152 ceph-mon[134767]: pgmap v529: 129 pgs: 129 active+clean; 433 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2023-10-18T12:33:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:08 smithi186 ceph-mon[135253]: pgmap v529: 129 pgs: 129 active+clean; 433 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 246 op/s 2023-10-18T12:33:09.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 788688 11.94 MB/sec execute 218 sec latency 48.683 ms 2023-10-18T12:33:10.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 790433 11.91 MB/sec execute 219 sec latency 83.349 ms 2023-10-18T12:33:11.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:11 smithi070 ceph-mon[98639]: pgmap v530: 129 pgs: 129 active+clean; 437 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2023-10-18T12:33:11.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:11 smithi152 ceph-mon[134767]: pgmap v530: 129 pgs: 129 active+clean; 437 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2023-10-18T12:33:11.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:11 smithi186 ceph-mon[135253]: pgmap v530: 129 pgs: 129 active+clean; 437 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2023-10-18T12:33:11.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 792514 11.92 MB/sec execute 220 sec latency 15.207 ms 2023-10-18T12:33:12.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:12 smithi070 ceph-mon[98639]: pgmap v531: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 226 op/s 2023-10-18T12:33:12.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:12 smithi152 ceph-mon[134767]: pgmap v531: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 226 op/s 2023-10-18T12:33:12.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:12 smithi186 ceph-mon[135253]: pgmap v531: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 226 op/s 2023-10-18T12:33:12.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 795128 11.93 MB/sec execute 221 sec latency 12.731 ms 2023-10-18T12:33:13.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 796731 11.92 MB/sec execute 222 sec latency 35.370 ms 2023-10-18T12:33:14.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 799119 11.93 MB/sec execute 223 sec latency 20.524 ms 2023-10-18T12:33:15.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:15 smithi070 ceph-mon[98639]: pgmap v532: 129 pgs: 129 active+clean; 444 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 221 op/s 2023-10-18T12:33:15.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:15 smithi070 ceph-mon[98639]: osdmap e95: 12 total, 12 up, 12 in 2023-10-18T12:33:15.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:15 smithi152 ceph-mon[134767]: pgmap v532: 129 pgs: 129 active+clean; 444 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 221 op/s 2023-10-18T12:33:15.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:15 smithi152 ceph-mon[134767]: osdmap e95: 12 total, 12 up, 12 in 2023-10-18T12:33:15.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:15 smithi186 ceph-mon[135253]: pgmap v532: 129 pgs: 129 active+clean; 444 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 221 op/s 2023-10-18T12:33:15.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:15 smithi186 ceph-mon[135253]: osdmap e95: 12 total, 12 up, 12 in 2023-10-18T12:33:15.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 800381 11.90 MB/sec execute 224 sec latency 50.655 ms 2023-10-18T12:33:16.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:16 smithi070 ceph-mon[98639]: pgmap v534: 129 pgs: 129 active+clean; 451 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2023-10-18T12:33:16.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:16 smithi152 ceph-mon[134767]: pgmap v534: 129 pgs: 129 active+clean; 451 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2023-10-18T12:33:16.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:16 smithi186 ceph-mon[135253]: pgmap v534: 129 pgs: 129 active+clean; 451 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2023-10-18T12:33:16.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 802842 11.91 MB/sec execute 225 sec latency 15.790 ms 2023-10-18T12:33:17.857 INFO:tasks.workunit.client.0.smithi070.stdout: 1 804981 11.89 MB/sec execute 226 sec latency 37.871 ms 2023-10-18T12:33:18.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 806934 11.89 MB/sec execute 227 sec latency 28.860 ms 2023-10-18T12:33:19.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:18 smithi070 ceph-mon[98639]: pgmap v535: 129 pgs: 129 active+clean; 452 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 215 op/s 2023-10-18T12:33:19.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:18 smithi152 ceph-mon[134767]: pgmap v535: 129 pgs: 129 active+clean; 452 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 215 op/s 2023-10-18T12:33:19.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:18 smithi186 ceph-mon[135253]: pgmap v535: 129 pgs: 129 active+clean; 452 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 215 op/s 2023-10-18T12:33:19.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 809582 11.91 MB/sec execute 228 sec latency 15.850 ms 2023-10-18T12:33:20.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:20 smithi070 ceph-mon[98639]: pgmap v536: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 233 op/s 2023-10-18T12:33:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:20 smithi152 ceph-mon[134767]: pgmap v536: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 233 op/s 2023-10-18T12:33:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:20 smithi186 ceph-mon[135253]: pgmap v536: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 233 op/s 2023-10-18T12:33:20.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 810481 11.87 MB/sec execute 229 sec latency 68.420 ms 2023-10-18T12:33:21.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 813166 11.89 MB/sec execute 230 sec latency 15.458 ms 2023-10-18T12:33:22.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 814736 11.87 MB/sec execute 231 sec latency 18.462 ms 2023-10-18T12:33:23.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:22 smithi070 ceph-mon[98639]: pgmap v537: 129 pgs: 129 active+clean; 437 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2023-10-18T12:33:23.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:22 smithi152 ceph-mon[134767]: pgmap v537: 129 pgs: 129 active+clean; 437 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2023-10-18T12:33:23.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:22 smithi186 ceph-mon[135253]: pgmap v537: 129 pgs: 129 active+clean; 437 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2023-10-18T12:33:23.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 817067 11.88 MB/sec execute 232 sec latency 12.822 ms 2023-10-18T12:33:24.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 819084 11.86 MB/sec execute 233 sec latency 40.682 ms 2023-10-18T12:33:25.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:24 smithi070 ceph-mon[98639]: pgmap v538: 129 pgs: 129 active+clean; 429 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2023-10-18T12:33:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:24 smithi152 ceph-mon[134767]: pgmap v538: 129 pgs: 129 active+clean; 429 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2023-10-18T12:33:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:24 smithi186 ceph-mon[135253]: pgmap v538: 129 pgs: 129 active+clean; 429 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2023-10-18T12:33:25.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 820736 11.86 MB/sec execute 234 sec latency 80.120 ms 2023-10-18T12:33:26.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 823115 11.86 MB/sec execute 235 sec latency 28.872 ms 2023-10-18T12:33:27.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:26 smithi070 ceph-mon[98639]: pgmap v539: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2023-10-18T12:33:27.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:26 smithi152 ceph-mon[134767]: pgmap v539: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2023-10-18T12:33:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:26 smithi186 ceph-mon[135253]: pgmap v539: 129 pgs: 129 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2023-10-18T12:33:27.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 825137 11.86 MB/sec execute 236 sec latency 13.033 ms 2023-10-18T12:33:28.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:33:28.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 -- 172.21.15.70:0/266551866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c069230 msgr2=0x7f0f8c069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 --2- 172.21.15.70:0/266551866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c069230 0x7f0f8c069610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009990 tx=0x7f0f7c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 -- 172.21.15.70:0/266551866 shutdown_connections 2023-10-18T12:33:28.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 --2- 172.21.15.70:0/266551866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c105b80 0x7f0f8c10d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 --2- 172.21.15.70:0/266551866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c069230 0x7f0f8c069610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 --2- 172.21.15.70:0/266551866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f8c1051c0 0x7f0f8c105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 -- 172.21.15.70:0/266551866 >> 172.21.15.70:0/266551866 conn(0x7f0f8c0767a0 msgr2=0x7f0f8c076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:28.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 -- 172.21.15.70:0/266551866 shutdown_connections 2023-10-18T12:33:28.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.725+0000 7f0f919ff700 1 -- 172.21.15.70:0/266551866 wait complete. 2023-10-18T12:33:28.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.726+0000 7f0f919ff700 1 Processor -- start 2023-10-18T12:33:28.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.726+0000 7f0f919ff700 1 -- start start 2023-10-18T12:33:28.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f919ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 0x7f0f8c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f919ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c1051c0 0x7f0f8c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f919ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f8c105b80 0x7f0f8c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f919ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1970b0 con 0x7f0f8c105b80 2023-10-18T12:33:28.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f919ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c197220 con 0x7f0f8c1051c0 2023-10-18T12:33:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f919ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1973e0 con 0x7f0f8c069230 2023-10-18T12:33:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 0x7f0f8c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 0x7f0f8c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42320/0 (socket says 172.21.15.70:42320) 2023-10-18T12:33:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 -- 172.21.15.70:0/2173560147 learned_addr learned my addr 172.21.15.70:0/2173560147 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c1051c0 0x7f0f8c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f8c105b80 0x7f0f8c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 -- 172.21.15.70:0/2173560147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c1051c0 msgr2=0x7f0f8c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c1051c0 0x7f0f8c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 -- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f8c105b80 msgr2=0x7f0f8c196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f8c105b80 0x7f0f8c196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.727+0000 7f0f8affd700 1 -- 172.21.15.70:0/2173560147 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80009520 con 0x7f0f8c069230 2023-10-18T12:33:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.728+0000 7f0f8affd700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 0x7f0f8c19c920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005910 tx=0x7f0f7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.728+0000 7f0f909fd700 1 -- 172.21.15.70:0/2173560147 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c01d070 con 0x7f0f8c069230 2023-10-18T12:33:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.728+0000 7f0f909fd700 1 -- 172.21.15.70:0/2173560147 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7c022470 con 0x7f0f8c069230 2023-10-18T12:33:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.728+0000 7f0f909fd700 1 -- 172.21.15.70:0/2173560147 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c017650 con 0x7f0f8c069230 2023-10-18T12:33:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.728+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c0095f0 con 0x7f0f8c069230 2023-10-18T12:33:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.728+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1a3730 con 0x7f0f8c069230 2023-10-18T12:33:28.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.732+0000 7f0f909fd700 1 -- 172.21.15.70:0/2173560147 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0f7c0225e0 con 0x7f0f8c069230 2023-10-18T12:33:28.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 Processor -- start 2023-10-18T12:33:28.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 -- start start 2023-10-18T12:33:28.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0f909fd700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0f740a1670 0x7f0f740a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc84910 0x560c4bc7e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc7e9e0 0x560c4bc7ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bbed6e0 con 0x560c4bc7e9e0 2023-10-18T12:33:28.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bbd3c60 con 0x560c4bc84910 2023-10-18T12:33:28.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bbd1580 con 0x560c4bc83ff0 2023-10-18T12:33:28.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.733+0000 7f0f909fd700 1 -- 172.21.15.70:0/2173560147 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0f7c0ce630 con 0x7f0f8c069230 2023-10-18T12:33:28.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc84910 0x560c4bc7e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc84910 0x560c4bc7e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43326/0 (socket says 172.21.15.70:43326) 2023-10-18T12:33:28.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8a7fc700 1 -- 172.21.15.70:0/2926329527 learned_addr learned my addr 172.21.15.70:0/2926329527 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:28.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0f740a1670 0x7f0f740a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc7e9e0 0x560c4bc7ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 -- 172.21.15.70:0/2926329527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc84910 msgr2=0x560c4bc7e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc84910 0x560c4bc7e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 -- 172.21.15.70:0/2926329527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc7e9e0 msgr2=0x560c4bc7ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc7e9e0 0x560c4bc7ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 -- 172.21.15.70:0/2926329527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4bb8a3f0 con 0x560c4bc83ff0 2023-10-18T12:33:28.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc7e9e0 0x560c4bc7ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:33:28.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8affd700 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc843d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0cba20 tx=0x7f0f7c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e3fc4171fcb92e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:28.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0f740a1670 0x7f0f740a3b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009550 tx=0x7f0f84009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:28.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f71ffb700 1 -- 172.21.15.70:0/2926329527 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c09afa0 con 0x560c4bc83ff0 2023-10-18T12:33:28.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f71ffb700 1 -- 172.21.15.70:0/2926329527 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7c0b3e90 con 0x560c4bc83ff0 2023-10-18T12:33:28.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.734+0000 7f0f71ffb700 1 -- 172.21.15.70:0/2926329527 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c09f640 con 0x560c4bc83ff0 2023-10-18T12:33:28.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 -- 172.21.15.70:0/2926329527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 msgr2=0x560c4bc843d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc843d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0cba20 tx=0x7f0f7c009310 comp rx=0 tx=0).stop 2023-10-18T12:33:28.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 -- 172.21.15.70:0/2926329527 shutdown_connections 2023-10-18T12:33:28.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc7e9e0 0x560c4bc7ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc843d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/2926329527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc84910 0x560c4bc7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 -- 172.21.15.70:0/2926329527 >> 172.21.15.70:0/2926329527 conn(0x560c4bbee720 msgr2=0x560c4bc74b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:28.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 -- 172.21.15.70:0/2926329527 shutdown_connections 2023-10-18T12:33:28.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.735+0000 7f0fa5263b80 1 -- 172.21.15.70:0/2926329527 wait complete. 2023-10-18T12:33:28.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 -- start start 2023-10-18T12:33:28.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 Processor -- start 2023-10-18T12:33:28.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc87b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bc88150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bbcce70 con 0x560c4bc83ff0 2023-10-18T12:33:28.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bb69710 con 0x560c4bc7e9e0 2023-10-18T12:33:28.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.736+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bb693b0 con 0x560c4bc84910 2023-10-18T12:33:28.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0fa5263b80 10 client.0 _ll_register_callbacks cb 0x560c4bab3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:28.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43340/0 (socket says 172.21.15.70:43340) 2023-10-18T12:33:28.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 -- 172.21.15.70:0/1784254452 learned_addr learned my addr 172.21.15.70:0/1784254452 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:28.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bc88150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc87b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 -- 172.21.15.70:0/1784254452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 msgr2=0x560c4bc88150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bc88150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 -- 172.21.15.70:0/1784254452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 msgr2=0x560c4bc87b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc87b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 -- 172.21.15.70:0/1784254452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1a3730 con 0x560c4bc7e9e0 2023-10-18T12:33:28.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f8affd700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8a800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0cbad0 tx=0x7f0f7c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=723392e76716aace server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:28.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f6affd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:33:28.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.737+0000 7f0f6affd700 1 -- 172.21.15.70:0/1784254452 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c09b8c0 con 0x560c4bc7e9e0 2023-10-18T12:33:28.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0f6affd700 1 -- 172.21.15.70:0/1784254452 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0f7c09bb60 con 0x560c4bc7e9e0 2023-10-18T12:33:28.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c4bbd1ef0 con 0x560c4bc7e9e0 2023-10-18T12:33:28.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0f6affd700 1 -- 172.21.15.70:0/1784254452 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f7c0a3760 con 0x560c4bc7e9e0 2023-10-18T12:33:28.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 10 client.26591 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:28.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560c4b9f7de0 con 0x560c4bc7e9e0 2023-10-18T12:33:28.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0f6affd700 1 -- 172.21.15.70:0/1784254452 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c4bbd1ef0 con 0x560c4bc7e9e0 2023-10-18T12:33:28.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0f6affd700 1 -- 172.21.15.70:0/1784254452 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f7c081020 con 0x560c4bc7e9e0 2023-10-18T12:33:28.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 10 client.26591 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:28.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:28.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 20 client.26591 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:28.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bc97ae0 0x560c4bc99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 4 client.26591 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:33:28.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c4bc498b0 con 0x560c4bc97ae0 2023-10-18T12:33:28.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.738+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bc97ae0 0x560c4bc99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.739+0000 7f0f6affd700 10 client.26591 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:28.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.739+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bc97ae0 0x560c4bc99f00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c372597982da6297 server_cookie=48906ea6e260c24e in_seq=0 out_seq=0 2023-10-18T12:33:28.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0f6affd700 1 -- 172.21.15.70:0/1784254452 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c4bc498b0 con 0x560c4bc97ae0 2023-10-18T12:33:28.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0f6affd700 10 client.26591 handle_command_reply: tid=0 2023-10-18T12:33:28.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 client.26591 shutdown 2023-10-18T12:33:28.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 msgr2=0x560c4bc8a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8a800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0cbad0 tx=0x7f0f7c00bfd0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 shutdown_connections 2023-10-18T12:33:28.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bc97ae0 0x560c4bc99f00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bc88150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8a800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/1784254452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc87b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0f6affd700 0 client.26591 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:28.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 >> 172.21.15.70:0/1784254452 conn(0x560c4bbee720 msgr2=0x560c4bc7d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:28.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 shutdown_connections 2023-10-18T12:33:28.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 1 -- 172.21.15.70:0/1784254452 wait complete. 2023-10-18T12:33:28.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.741+0000 7f0fa5263b80 20 client.26591 trim_cache size 0 max 16384 2023-10-18T12:33:28.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 Processor -- start 2023-10-18T12:33:28.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 -- start start 2023-10-18T12:33:28.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc81ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc84910 0x560c4bc82000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ba4d920 con 0x560c4bc84910 2023-10-18T12:33:28.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bc76410 con 0x560c4bc7e9e0 2023-10-18T12:33:28.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bc8ae60 con 0x560c4bc83ff0 2023-10-18T12:33:28.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0f8b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc84910 0x560c4bc82000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc81ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc81ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42332/0 (socket says 172.21.15.70:42332) 2023-10-18T12:33:28.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43346/0 (socket says 172.21.15.70:43346) 2023-10-18T12:33:28.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.759+0000 7f0f8a7fc700 1 -- 172.21.15.70:0/3227470233 learned_addr learned my addr 172.21.15.70:0/3227470233 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:28.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8b7fe700 1 -- 172.21.15.70:0/3227470233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 msgr2=0x560c4bc81ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc81ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8b7fe700 1 -- 172.21.15.70:0/3227470233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 msgr2=0x560c4bc8f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8b7fe700 1 -- 172.21.15.70:0/3227470233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4b9e0c60 con 0x560c4bc84910 2023-10-18T12:33:28.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc81ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:28.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc84910 0x560c4bc82000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009fd0 tx=0x7f0f8400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc7fb98e5906441 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:28.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f727fc700 1 -- 172.21.15.70:0/3227470233 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400e070 con 0x560c4bc84910 2023-10-18T12:33:28.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.760+0000 7f0f727fc700 1 -- 172.21.15.70:0/3227470233 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f8401f050 con 0x560c4bc84910 2023-10-18T12:33:28.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0f727fc700 1 -- 172.21.15.70:0/3227470233 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840257a0 con 0x560c4bc84910 2023-10-18T12:33:28.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 -- 172.21.15.70:0/3227470233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc84910 msgr2=0x560c4bc82000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc84910 0x560c4bc82000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009fd0 tx=0x7f0f8400e000 comp rx=0 tx=0).stop 2023-10-18T12:33:28.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 -- 172.21.15.70:0/3227470233 shutdown_connections 2023-10-18T12:33:28.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc84910 0x560c4bc82000 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/3227470233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc83ff0 0x560c4bc81ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 -- 172.21.15.70:0/3227470233 >> 172.21.15.70:0/3227470233 conn(0x560c4bbee720 msgr2=0x560c4bc7edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:28.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 -- 172.21.15.70:0/3227470233 shutdown_connections 2023-10-18T12:33:28.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.761+0000 7f0fa5263b80 1 -- 172.21.15.70:0/3227470233 wait complete. 2023-10-18T12:33:28.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.762+0000 7f0fa5263b80 1 -- start start 2023-10-18T12:33:28.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.762+0000 7f0fa5263b80 1 Processor -- start 2023-10-18T12:33:28.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.762+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc8e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0fa5263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bceb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bcebe20 con 0x560c4bc83ff0 2023-10-18T12:33:28.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bcebf60 con 0x560c4bc7e9e0 2023-10-18T12:33:28.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0fa5263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4bcec0a0 con 0x560c4bc84910 2023-10-18T12:33:28.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0fa5263b80 10 client.0 _ll_register_callbacks cb 0x560c4bab3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:28.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43348/0 (socket says 172.21.15.70:43348) 2023-10-18T12:33:28.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 -- 172.21.15.70:0/4180644142 learned_addr learned my addr 172.21.15.70:0/4180644142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:28.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc8e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8b7fe700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bceb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 -- 172.21.15.70:0/4180644142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 msgr2=0x560c4bceb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bceb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 -- 172.21.15.70:0/4180644142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 msgr2=0x560c4bc8e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:28.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc8e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 -- 172.21.15.70:0/4180644142 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f80009e10 con 0x560c4bc7e9e0 2023-10-18T12:33:28.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f8affd700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8dd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c006010 tx=0x7f0f7c0a3500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=745a7ce3701243db server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:28.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.763+0000 7f0f6affd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:33:28.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0f6affd700 1 -- 172.21.15.70:0/4180644142 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c09ac20 con 0x560c4bc7e9e0 2023-10-18T12:33:28.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0f6affd700 1 -- 172.21.15.70:0/4180644142 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0f7c09bb60 con 0x560c4bc7e9e0 2023-10-18T12:33:28.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f7c0b3e90 con 0x560c4bc7e9e0 2023-10-18T12:33:28.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0f6affd700 1 -- 172.21.15.70:0/4180644142 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f7c0bc020 con 0x560c4bc7e9e0 2023-10-18T12:33:28.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 10 client.26597 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:28.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f0f84018440 con 0x560c4bc7e9e0 2023-10-18T12:33:28.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0f6affd700 1 -- 172.21.15.70:0/4180644142 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7c0b3e90 con 0x560c4bc7e9e0 2023-10-18T12:33:28.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0f6affd700 1 -- 172.21.15.70:0/4180644142 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f7c0aa030 con 0x560c4bc7e9e0 2023-10-18T12:33:28.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 10 client.26597 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:28.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:28.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 20 client.26597 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:28.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bced4e0 0x560c4bcef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:28.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 4 client.26597 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:33:28.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c4bbc5e50 con 0x560c4bced4e0 2023-10-18T12:33:28.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.764+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bced4e0 0x560c4bcef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:28.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.765+0000 7f0f6affd700 10 client.26597 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:28.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.765+0000 7f0f8a7fc700 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bced4e0 0x560c4bcef900 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733125af727f71a2 server_cookie=be6accb5b1a31f1f in_seq=0 out_seq=0 2023-10-18T12:33:28.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0f6affd700 1 -- 172.21.15.70:0/4180644142 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c4bbc5e50 con 0x560c4bced4e0 2023-10-18T12:33:28.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0f6affd700 10 client.26597 handle_command_reply: tid=0 2023-10-18T12:33:28.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 client.26597 shutdown 2023-10-18T12:33:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 msgr2=0x560c4bc8dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8dd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c006010 tx=0x7f0f7c0a3500 comp rx=0 tx=0).stop 2023-10-18T12:33:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 shutdown_connections 2023-10-18T12:33:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560c4bced4e0 0x560c4bcef900 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c4bc84910 0x560c4bceb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4bc7e9e0 0x560c4bc8dd00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 --2- 172.21.15.70:0/4180644142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c4bc83ff0 0x560c4bc8e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 >> 172.21.15.70:0/4180644142 conn(0x560c4bbee720 msgr2=0x560c4bc81ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:28.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0f6affd700 0 client.26597 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:28.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 shutdown_connections 2023-10-18T12:33:28.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.767+0000 7f0fa5263b80 1 -- 172.21.15.70:0/4180644142 wait complete. 2023-10-18T12:33:28.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0fa5263b80 20 client.26597 trim_cache size 0 max 16384 2023-10-18T12:33:28.790 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:33:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:33:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:33:28.791 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:33:28.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0f740a1670 msgr2=0x7f0f740a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0f740a1670 0x7f0f740a3b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009550 tx=0x7f0f84009920 comp rx=0 tx=0).stop 2023-10-18T12:33:28.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 msgr2=0x7f0f8c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:28.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 0x7f0f8c19c920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005910 tx=0x7f0f7c004730 comp rx=0 tx=0).stop 2023-10-18T12:33:28.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 shutdown_connections 2023-10-18T12:33:28.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f8c105b80 0x7f0f8c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c069230 0x7f0f8c19c920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0f740a1670 0x7f0f740a3b30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 --2- 172.21.15.70:0/2173560147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f8c1051c0 0x7f0f8c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:28.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 >> 172.21.15.70:0/2173560147 conn(0x7f0f8c0767a0 msgr2=0x7f0f8c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:28.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 shutdown_connections 2023-10-18T12:33:28.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:28.768+0000 7f0f919ff700 1 -- 172.21.15.70:0/2173560147 wait complete. 2023-10-18T12:33:28.795 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:33:28.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:33:28.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:28 smithi070 ceph-mon[98639]: pgmap v540: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 225 op/s 2023-10-18T12:33:28.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 827622 11.87 MB/sec execute 237 sec latency 13.964 ms 2023-10-18T12:33:29.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.010+0000 7f7554bff700 1 -- 172.21.15.70:0/1933979799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550100290 msgr2=0x7f755010c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.010+0000 7f7554bff700 1 --2- 172.21.15.70:0/1933979799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550100290 0x7f755010c420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7548009f90 tx=0x7f754800b330 comp rx=0 tx=0).stop 2023-10-18T12:33:29.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 -- 172.21.15.70:0/1933979799 shutdown_connections 2023-10-18T12:33:29.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 --2- 172.21.15.70:0/1933979799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550100290 0x7f755010c420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 --2- 172.21.15.70:0/1933979799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550105710 0x7f7550105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 --2- 172.21.15.70:0/1933979799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75500ff8d0 0x7f75500ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 -- 172.21.15.70:0/1933979799 >> 172.21.15.70:0/1933979799 conn(0x7f7550076400 msgr2=0x7f7550076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 -- 172.21.15.70:0/1933979799 shutdown_connections 2023-10-18T12:33:29.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.011+0000 7f7554bff700 1 -- 172.21.15.70:0/1933979799 wait complete. 2023-10-18T12:33:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 Processor -- start 2023-10-18T12:33:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 -- start start 2023-10-18T12:33:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75500ff8d0 0x7f7550198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 0x7f7550199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550105710 0x7f755019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755019dd50 con 0x7f7550105710 2023-10-18T12:33:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755019dec0 con 0x7f7550100290 2023-10-18T12:33:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f7554bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755019e080 con 0x7f75500ff8d0 2023-10-18T12:33:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f754dd9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 0x7f7550199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f754ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550105710 0x7f755019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f754dd9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 0x7f7550199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43354/0 (socket says 172.21.15.70:43354) 2023-10-18T12:33:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f754dd9b700 1 -- 172.21.15.70:0/3405967861 learned_addr learned my addr 172.21.15.70:0/3405967861 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.012+0000 7f754e59c700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75500ff8d0 0x7f7550198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754ed9d700 1 -- 172.21.15.70:0/3405967861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75500ff8d0 msgr2=0x7f7550198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754ed9d700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75500ff8d0 0x7f7550198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754ed9d700 1 -- 172.21.15.70:0/3405967861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 msgr2=0x7f7550199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754ed9d700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 0x7f7550199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754ed9d700 1 -- 172.21.15.70:0/3405967861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548009c40 con 0x7f7550105710 2023-10-18T12:33:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754dd9b700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 0x7f7550199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f754ed9d700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550105710 0x7f755019d710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f754800b6c0 tx=0x7f75480094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800e040 con 0x7f7550105710 2023-10-18T12:33:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7548007e10 con 0x7f7550105710 2023-10-18T12:33:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755019e300 con 0x7f7550105710 2023-10-18T12:33:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548005040 con 0x7f7550105710 2023-10-18T12:33:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.013+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75501a5ba0 con 0x7f7550105710 2023-10-18T12:33:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.014+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7550100cc0 con 0x7f7550105710 2023-10-18T12:33:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.016+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f754800bcf0 con 0x7f7550105710 2023-10-18T12:33:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.018+0000 7f753f7fe700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f75380a15f0 0x7f75380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.018+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f75480ce6d0 con 0x7f7550105710 2023-10-18T12:33:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.019+0000 7f754e59c700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f75380a15f0 0x7f75380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.019+0000 7f754e59c700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f75380a15f0 0x7f75380a3ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7540005fd0 tx=0x7f7540009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.023+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f75480976c0 con 0x7f7550105710 2023-10-18T12:33:29.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:28 smithi152 ceph-mon[134767]: pgmap v540: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 225 op/s 2023-10-18T12:33:29.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:28 smithi186 ceph-mon[135253]: pgmap v540: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 225 op/s 2023-10-18T12:33:29.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.305+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7550067530 con 0x7f7550105710 2023-10-18T12:33:29.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.306+0000 7f753f7fe700 1 -- 172.21.15.70:0/3405967861 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f754801bed0 con 0x7f7550105710 2023-10-18T12:33:29.308 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:33:29.308 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:33:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f75380a15f0 msgr2=0x7f75380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f75380a15f0 0x7f75380a3ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7540005fd0 tx=0x7f7540009310 comp rx=0 tx=0).stop 2023-10-18T12:33:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550105710 msgr2=0x7f755019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550105710 0x7f755019d710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f754800b6c0 tx=0x7f75480094b0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 shutdown_connections 2023-10-18T12:33:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7550105710 0x7f755019d710 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75500ff8d0 0x7f7550198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f75380a15f0 0x7f75380a3ab0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 --2- 172.21.15.70:0/3405967861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550100290 0x7f7550199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 >> 172.21.15.70:0/3405967861 conn(0x7f7550076400 msgr2=0x7f7550067850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 shutdown_connections 2023-10-18T12:33:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.309+0000 7f7554bff700 1 -- 172.21.15.70:0/3405967861 wait complete. 2023-10-18T12:33:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:33:29.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:33:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 -- 172.21.15.70:0/3846669137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 msgr2=0x7f137410f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 --2- 172.21.15.70:0/3846669137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137410f290 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f136c009f90 tx=0x7f136c00b330 comp rx=0 tx=0).stop 2023-10-18T12:33:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 -- 172.21.15.70:0/3846669137 shutdown_connections 2023-10-18T12:33:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 --2- 172.21.15.70:0/3846669137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137410f290 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 --2- 172.21.15.70:0/3846669137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1374072bc0 0x7f1374072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 --2- 172.21.15.70:0/3846669137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13740734e0 0x7f137410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 -- 172.21.15.70:0/3846669137 >> 172.21.15.70:0/3846669137 conn(0x7f13740fc0f0 msgr2=0x7f13740fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 -- 172.21.15.70:0/3846669137 shutdown_connections 2023-10-18T12:33:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.580+0000 7f1379f9d700 1 -- 172.21.15.70:0/3846669137 wait complete. 2023-10-18T12:33:29.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.581+0000 7f1379f9d700 1 Processor -- start 2023-10-18T12:33:29.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 -- start start 2023-10-18T12:33:29.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 0x7f1374198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13740734e0 0x7f1374199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137419dca0 con 0x7f13740734e0 2023-10-18T12:33:29.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137419de10 con 0x7f137410ce60 2023-10-18T12:33:29.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1379f9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137419dfd0 con 0x7f1374072bc0 2023-10-18T12:33:29.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 0x7f1374198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1373fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13740734e0 0x7f1374199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 0x7f1374198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42364/0 (socket says 172.21.15.70:42364) 2023-10-18T12:33:29.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f1378f9b700 1 -- 172.21.15.70:0/2289421775 learned_addr learned my addr 172.21.15.70:0/2289421775 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:29.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.582+0000 7f137979c700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1378f9b700 1 -- 172.21.15.70:0/2289421775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 msgr2=0x7f137419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1378f9b700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1378f9b700 1 -- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13740734e0 msgr2=0x7f1374199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1378f9b700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13740734e0 0x7f1374199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1378f9b700 1 -- 172.21.15.70:0/2289421775 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c009c40 con 0x7f1374072bc0 2023-10-18T12:33:29.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f137979c700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:29.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1378f9b700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 0x7f1374198dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f136800e9c0 tx=0x7f136800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1371ffb700 1 -- 172.21.15.70:0/2289421775 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368004a30 con 0x7f1374072bc0 2023-10-18T12:33:29.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137419e2b0 con 0x7f1374072bc0 2023-10-18T12:33:29.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1371ffb700 1 -- 172.21.15.70:0/2289421775 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136800cc80 con 0x7f1374072bc0 2023-10-18T12:33:29.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1371ffb700 1 -- 172.21.15.70:0/2289421775 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680054e0 con 0x7f1374072bc0 2023-10-18T12:33:29.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.583+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13741a5af0 con 0x7f1374072bc0 2023-10-18T12:33:29.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.587+0000 7f1371ffb700 1 -- 172.21.15.70:0/2289421775 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1368004b90 con 0x7f1374072bc0 2023-10-18T12:33:29.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 Processor -- start 2023-10-18T12:33:29.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 -- start start 2023-10-18T12:33:29.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f1371ffb700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f13640a1670 0x7f13640a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x55834235b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x5583423552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342355c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583422c86e0 con 0x55834235b680 2023-10-18T12:33:29.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583422aec60 con 0x558342355820 2023-10-18T12:33:29.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f138eb74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583422ac580 con 0x55834235ad60 2023-10-18T12:33:29.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f1371ffb700 1 -- 172.21.15.70:0/2289421775 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f1368014070 con 0x7f1374072bc0 2023-10-18T12:33:29.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x55834235b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x55834235b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42366/0 (socket says 172.21.15.70:42366) 2023-10-18T12:33:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f1378f9b700 1 -- 172.21.15.70:0/2961332808 learned_addr learned my addr 172.21.15.70:0/2961332808 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f137979c700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f13640a1670 0x7f13640a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.588+0000 7f137979c700 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342355c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f1378f9b700 1 -- 172.21.15.70:0/2961332808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 msgr2=0x558342355c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f1378f9b700 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342355c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f1378f9b700 1 -- 172.21.15.70:0/2961332808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 msgr2=0x5583423552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f1378f9b700 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x5583423552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f1378f9b700 1 -- 172.21.15.70:0/2961332808 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583422653f0 con 0x55834235ad60 2023-10-18T12:33:29.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f1378f9b700 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x55834235b140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1368096640 tx=0x7f1368099b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c8aefc3391353e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f135effd700 1 -- 172.21.15.70:0/2961332808 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136809a140 con 0x55834235ad60 2023-10-18T12:33:29.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f135effd700 1 -- 172.21.15.70:0/2961332808 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136809e290 con 0x55834235ad60 2023-10-18T12:33:29.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.589+0000 7f135effd700 1 -- 172.21.15.70:0/2961332808 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680cab30 con 0x55834235ad60 2023-10-18T12:33:29.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 -- 172.21.15.70:0/2961332808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 msgr2=0x55834235b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f137979c700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f13640a1670 0x7f13640a3b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f136c0060f0 tx=0x7f136c007420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x55834235b140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1368096640 tx=0x7f1368099b20 comp rx=0 tx=0).stop 2023-10-18T12:33:29.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 -- 172.21.15.70:0/2961332808 shutdown_connections 2023-10-18T12:33:29.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342355c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x55834235b140 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 --2- 172.21.15.70:0/2961332808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x5583423552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 -- 172.21.15.70:0/2961332808 >> 172.21.15.70:0/2961332808 conn(0x5583422c9720 msgr2=0x55834234fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 -- 172.21.15.70:0/2961332808 shutdown_connections 2023-10-18T12:33:29.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 -- 172.21.15.70:0/2961332808 wait complete. 2023-10-18T12:33:29.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.590+0000 7f138eb74b80 1 -- start start 2023-10-18T12:33:29.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 Processor -- start 2023-10-18T12:33:29.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 0x5583423615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235ad60 0x55834235e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x55834235ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583422a7e70 con 0x55834235b680 2023-10-18T12:33:29.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558342244710 con 0x55834235ad60 2023-10-18T12:33:29.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583422443b0 con 0x558342355820 2023-10-18T12:33:29.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f138eb74b80 10 client.0 _ll_register_callbacks cb 0x55834218ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:29.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 0x5583423615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f1373fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235ad60 0x55834235e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 0x5583423615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42378/0 (socket says 172.21.15.70:42378) 2023-10-18T12:33:29.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f1378f9b700 1 -- 172.21.15.70:0/389731011 learned_addr learned my addr 172.21.15.70:0/389731011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:29.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.591+0000 7f137979c700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x55834235ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1378f9b700 1 -- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235ad60 msgr2=0x55834235e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1378f9b700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235ad60 0x55834235e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1378f9b700 1 -- 172.21.15.70:0/389731011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 msgr2=0x55834235ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1378f9b700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x55834235ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1378f9b700 1 -- 172.21.15.70:0/389731011 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13741a5af0 con 0x558342355820 2023-10-18T12:33:29.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1373fff700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235ad60 0x55834235e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:29.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1378f9b700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 0x5583423615b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f136809a080 tx=0x7f1368005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94704b8a5f201957 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1353fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:33:29.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1353fff700 1 -- 172.21.15.70:0/389731011 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136809a1c0 con 0x558342355820 2023-10-18T12:33:29.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583422acef0 con 0x558342355820 2023-10-18T12:33:29.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1353fff700 1 -- 172.21.15.70:0/389731011 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f13680cab30 con 0x558342355820 2023-10-18T12:33:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.592+0000 7f1353fff700 1 -- 172.21.15.70:0/389731011 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13680cacf0 con 0x558342355820 2023-10-18T12:33:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 10 client.26620 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5583420d2de0 con 0x558342355820 2023-10-18T12:33:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f1353fff700 1 -- 172.21.15.70:0/389731011 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583422acef0 con 0x558342355820 2023-10-18T12:33:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f1353fff700 1 -- 172.21.15.70:0/389731011 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1368099630 con 0x558342355820 2023-10-18T12:33:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:29.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 10 client.26620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:29.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 20 client.26620 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:29.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55834236ecc0 0x5583423710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 4 client.26620 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:33:29.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583423248b0 con 0x55834236ecc0 2023-10-18T12:33:29.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.593+0000 7f1373fff700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55834236ecc0 0x5583423710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.594+0000 7f1353fff700 10 client.26620 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:29.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.594+0000 7f1373fff700 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55834236ecc0 0x5583423710e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e49ac4cb477f873 server_cookie=52ebd907283cacad in_seq=0 out_seq=0 2023-10-18T12:33:29.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.594+0000 7f1353fff700 1 -- 172.21.15.70:0/389731011 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583423248b0 con 0x55834236ecc0 2023-10-18T12:33:29.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.594+0000 7f1353fff700 10 client.26620 handle_command_reply: tid=0 2023-10-18T12:33:29.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.594+0000 7f138eb74b80 1 client.26620 shutdown 2023-10-18T12:33:29.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 msgr2=0x5583423615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 0x5583423615b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f136809a080 tx=0x7f1368005870 comp rx=0 tx=0).stop 2023-10-18T12:33:29.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 shutdown_connections 2023-10-18T12:33:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55834236ecc0 0x5583423710e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235b680 0x55834235ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558342355820 0x5583423615b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 --2- 172.21.15.70:0/389731011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235ad60 0x55834235e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f1353fff700 0 client.26620 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 >> 172.21.15.70:0/389731011 conn(0x5583422c9720 msgr2=0x5583423542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 shutdown_connections 2023-10-18T12:33:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 1 -- 172.21.15.70:0/389731011 wait complete. 2023-10-18T12:33:29.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.595+0000 7f138eb74b80 20 client.26620 trim_cache size 0 max 16384 2023-10-18T12:33:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.611+0000 7f138eb74b80 1 Processor -- start 2023-10-18T12:33:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.611+0000 7f138eb74b80 1 -- start start 2023-10-18T12:33:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.611+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558342355820 0x55834234a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x558342366090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235b680 0x558342354a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f138eb74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558342128920 con 0x558342355820 2023-10-18T12:33:29.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f138eb74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583423482b0 con 0x55834235b680 2023-10-18T12:33:29.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f138eb74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583423665d0 con 0x55834235ad60 2023-10-18T12:33:29.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f1373fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x558342366090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f1373fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x558342366090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42390/0 (socket says 172.21.15.70:42390) 2023-10-18T12:33:29.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f1373fff700 1 -- 172.21.15.70:0/3869703707 learned_addr learned my addr 172.21.15.70:0/3869703707 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:29.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f1378f9b700 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558342355820 0x55834234a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.612+0000 7f137979c700 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235b680 0x558342354a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1373fff700 1 -- 172.21.15.70:0/3869703707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235b680 msgr2=0x558342354a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1373fff700 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235b680 0x558342354a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1373fff700 1 -- 172.21.15.70:0/3869703707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558342355820 msgr2=0x55834234a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1373fff700 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558342355820 0x55834234a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1373fff700 1 -- 172.21.15.70:0/3869703707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583420bbc60 con 0x55834235ad60 2023-10-18T12:33:29.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1378f9b700 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558342355820 0x55834234a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:29.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f1373fff700 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x558342366090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1360005950 tx=0x7f136001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c07c87a95faa68 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f135f7fe700 1 -- 172.21.15.70:0/3869703707 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360017040 con 0x55834235ad60 2023-10-18T12:33:29.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f135f7fe700 1 -- 172.21.15.70:0/3869703707 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1360019040 con 0x55834235ad60 2023-10-18T12:33:29.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.613+0000 7f135f7fe700 1 -- 172.21.15.70:0/3869703707 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360010690 con 0x55834235ad60 2023-10-18T12:33:29.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 -- 172.21.15.70:0/3869703707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 msgr2=0x558342366090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x558342366090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1360005950 tx=0x7f136001ed30 comp rx=0 tx=0).stop 2023-10-18T12:33:29.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 -- 172.21.15.70:0/3869703707 shutdown_connections 2023-10-18T12:33:29.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55834235b680 0x558342354a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558342355820 0x55834234a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3869703707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235ad60 0x558342366090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 -- 172.21.15.70:0/3869703707 >> 172.21.15.70:0/3869703707 conn(0x5583422c9720 msgr2=0x558342355c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 -- 172.21.15.70:0/3869703707 shutdown_connections 2023-10-18T12:33:29.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.614+0000 7f138eb74b80 1 -- 172.21.15.70:0/3869703707 wait complete. 2023-10-18T12:33:29.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.615+0000 7f138eb74b80 1 -- start start 2023-10-18T12:33:29.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.615+0000 7f138eb74b80 1 Processor -- start 2023-10-18T12:33:29.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342364280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235ad60 0x5583423647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235b680 0x5583423c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583423c2030 con 0x55834235ad60 2023-10-18T12:33:29.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583423c21a0 con 0x558342355820 2023-10-18T12:33:29.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583423c2360 con 0x55834235b680 2023-10-18T12:33:29.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f138eb74b80 10 client.0 _ll_register_callbacks cb 0x55834218ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:29.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342364280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342364280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43390/0 (socket says 172.21.15.70:43390) 2023-10-18T12:33:29.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 -- 172.21.15.70:0/3803185467 learned_addr learned my addr 172.21.15.70:0/3803185467 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:29.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1373fff700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235ad60 0x5583423647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f137979c700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235b680 0x5583423c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 -- 172.21.15.70:0/3803185467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235b680 msgr2=0x5583423c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235b680 0x5583423c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 -- 172.21.15.70:0/3803185467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235ad60 msgr2=0x5583423647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:29.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.616+0000 7f1378f9b700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235ad60 0x5583423647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1378f9b700 1 -- 172.21.15.70:0/3803185467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583420d2de0 con 0x558342355820 2023-10-18T12:33:29.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1378f9b700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342364280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1368099e40 tx=0x7f1368099aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e24ee615cdde01 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:29.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1353fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:33:29.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1353fff700 1 -- 172.21.15.70:0/3803185467 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680b8460 con 0x558342355820 2023-10-18T12:33:29.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1353fff700 1 -- 172.21.15.70:0/3803185467 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f136800cdf0 con 0x558342355820 2023-10-18T12:33:29.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f136809e290 con 0x558342355820 2023-10-18T12:33:29.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1353fff700 1 -- 172.21.15.70:0/3803185467 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13680cad30 con 0x558342355820 2023-10-18T12:33:29.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f138eb74b80 10 client.26621 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:29.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f1360009d60 con 0x558342355820 2023-10-18T12:33:29.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.617+0000 7f1353fff700 1 -- 172.21.15.70:0/3803185467 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136809e290 con 0x558342355820 2023-10-18T12:33:29.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f1353fff700 1 -- 172.21.15.70:0/3803185467 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13680991c0 con 0x558342355820 2023-10-18T12:33:29.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f138eb74b80 10 client.26621 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:29.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f138eb74b80 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:29.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f138eb74b80 20 client.26621 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:29.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5583423c36a0 0x5583423c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:29.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f138eb74b80 4 client.26621 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:33:29.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5583422a0e50 con 0x5583423c36a0 2023-10-18T12:33:29.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f1373fff700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5583423c36a0 0x5583423c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:29.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f1353fff700 10 client.26621 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:29.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.618+0000 7f1373fff700 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5583423c36a0 0x5583423c5a90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64faef76191449ef server_cookie=3d3e4918936cc23 in_seq=0 out_seq=0 2023-10-18T12:33:29.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.619+0000 7f1353fff700 1 -- 172.21.15.70:0/3803185467 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5583422a0e50 con 0x5583423c36a0 2023-10-18T12:33:29.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.619+0000 7f1353fff700 10 client.26621 handle_command_reply: tid=0 2023-10-18T12:33:29.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.619+0000 7f138eb74b80 1 client.26621 shutdown 2023-10-18T12:33:29.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 msgr2=0x558342364280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342364280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1368099e40 tx=0x7f1368099aa0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 shutdown_connections 2023-10-18T12:33:29.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5583423c36a0 0x5583423c5a90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55834235b680 0x5583423c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558342355820 0x558342364280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 --2- 172.21.15.70:0/3803185467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55834235ad60 0x5583423647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 >> 172.21.15.70:0/3803185467 conn(0x5583422c9720 msgr2=0x558342354950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f1353fff700 0 client.26621 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 shutdown_connections 2023-10-18T12:33:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 1 -- 172.21.15.70:0/3803185467 wait complete. 2023-10-18T12:33:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.620+0000 7f138eb74b80 20 client.26621 trim_cache size 0 max 16384 2023-10-18T12:33:29.643 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:33:29.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f13640a1670 msgr2=0x7f13640a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f13640a1670 0x7f13640a3b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f136c0060f0 tx=0x7f136c007420 comp rx=0 tx=0).stop 2023-10-18T12:33:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 msgr2=0x7f1374198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 0x7f1374198dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f136800e9c0 tx=0x7f136800ecd0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 shutdown_connections 2023-10-18T12:33:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f13640a1670 0x7f13640a3b30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f137410ce60 0x7f137419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374072bc0 0x7f1374198dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 --2- 172.21.15.70:0/2289421775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13740734e0 0x7f1374199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:29.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 >> 172.21.15.70:0/2289421775 conn(0x7f13740fc0f0 msgr2=0x7f13740fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:29.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 shutdown_connections 2023-10-18T12:33:29.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:29.621+0000 7f1379f9d700 1 -- 172.21.15.70:0/2289421775 wait complete. 2023-10-18T12:33:29.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 829618 11.85 MB/sec execute 238 sec latency 34.856 ms 2023-10-18T12:33:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:30 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3405967861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:33:30.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:30 smithi070 ceph-mon[98639]: pgmap v541: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 229 op/s 2023-10-18T12:33:30.647 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:33:30.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:33:30.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:30 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3405967861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:33:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:30 smithi152 ceph-mon[134767]: pgmap v541: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 229 op/s 2023-10-18T12:33:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:30 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3405967861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:33:30.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:30 smithi186 ceph-mon[135253]: pgmap v541: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 229 op/s 2023-10-18T12:33:30.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 831377 11.85 MB/sec execute 239 sec latency 39.174 ms 2023-10-18T12:33:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 -- 172.21.15.70:0/782438557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66381010d0 msgr2=0x7f663810d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 --2- 172.21.15.70:0/782438557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66381010d0 0x7f663810d5e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6630009f90 tx=0x7f663000b330 comp rx=0 tx=0).stop 2023-10-18T12:33:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 -- 172.21.15.70:0/782438557 shutdown_connections 2023-10-18T12:33:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 --2- 172.21.15.70:0/782438557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66381010d0 0x7f663810d5e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 --2- 172.21.15.70:0/782438557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6638106720 0x7f6638106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 --2- 172.21.15.70:0/782438557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638100710 0x7f6638100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.857+0000 7f663e7ff700 1 -- 172.21.15.70:0/782438557 >> 172.21.15.70:0/782438557 conn(0x7f66380fc200 msgr2=0x7f66380fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.858+0000 7f663e7ff700 1 -- 172.21.15.70:0/782438557 shutdown_connections 2023-10-18T12:33:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.858+0000 7f663e7ff700 1 -- 172.21.15.70:0/782438557 wait complete. 2023-10-18T12:33:30.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 Processor -- start 2023-10-18T12:33:30.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 -- start start 2023-10-18T12:33:30.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 0x7f6638198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66381010d0 0x7f66381992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638106720 0x7f663819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663819cb60 con 0x7f6638100710 2023-10-18T12:33:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663819ccd0 con 0x7f6638106720 2023-10-18T12:33:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.859+0000 7f663e7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663819ce90 con 0x7f66381010d0 2023-10-18T12:33:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 0x7f6638198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 0x7f6638198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59982/0 (socket says 172.21.15.70:59982) 2023-10-18T12:33:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f663cd9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638106720 0x7f663819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 -- 172.21.15.70:0/438508750 learned_addr learned my addr 172.21.15.70:0/438508750 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f66377fe700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66381010d0 0x7f66381992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 -- 172.21.15.70:0/438508750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66381010d0 msgr2=0x7f66381992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66381010d0 0x7f66381992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 -- 172.21.15.70:0/438508750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638106720 msgr2=0x7f663819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638106720 0x7f663819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 -- 172.21.15.70:0/438508750 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009c40 con 0x7f6638100710 2023-10-18T12:33:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f663cd9c700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638106720 0x7f663819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f6637fff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 0x7f6638198d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f662800e9a0 tx=0x7f662800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.860+0000 7f66357fa700 1 -- 172.21.15.70:0/438508750 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628009790 con 0x7f6638100710 2023-10-18T12:33:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.861+0000 7f66357fa700 1 -- 172.21.15.70:0/438508750 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662800ce80 con 0x7f6638100710 2023-10-18T12:33:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.861+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663819d170 con 0x7f6638100710 2023-10-18T12:33:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.861+0000 7f66357fa700 1 -- 172.21.15.70:0/438508750 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628018680 con 0x7f6638100710 2023-10-18T12:33:30.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.861+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66381a5ad0 con 0x7f6638100710 2023-10-18T12:33:30.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.864+0000 7f66357fa700 1 -- 172.21.15.70:0/438508750 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6628010430 con 0x7f6638100710 2023-10-18T12:33:30.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.865+0000 7f66357fa700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f66200a1670 0x7f66200a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.865+0000 7f66357fa700 1 -- 172.21.15.70:0/438508750 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f6628014070 con 0x7f6638100710 2023-10-18T12:33:30.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.865+0000 7f6652112b80 1 Processor -- start 2023-10-18T12:33:30.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.865+0000 7f66377fe700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f66200a1670 0x7f66200a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.865+0000 7f6652112b80 1 -- start start 2023-10-18T12:33:30.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6652112b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 0x558e07b2d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6652112b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6652112b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b27650 0x558e07b27ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6652112b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07a9a760 con 0x558e07b2d580 2023-10-18T12:33:30.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6652112b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07a80ce0 con 0x558e07b2cc60 2023-10-18T12:33:30.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6652112b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07a7e600 con 0x558e07b27650 2023-10-18T12:33:30.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 0x558e07b2d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 0x558e07b2d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43428/0 (socket says 172.21.15.70:43428) 2023-10-18T12:33:30.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 -- 172.21.15.70:0/573667498 learned_addr learned my addr 172.21.15.70:0/573667498 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:30.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f66377fe700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f66200a1670 0x7f66200a3b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f663819a240 tx=0x7f662c009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:30.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f66377fe700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f663cd9c700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b27650 0x558e07b27ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 -- 172.21.15.70:0/573667498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b27650 msgr2=0x558e07b27ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b27650 0x558e07b27ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 -- 172.21.15.70:0/573667498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 msgr2=0x558e07b27110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b27110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 -- 172.21.15.70:0/573667498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e07a37470 con 0x558e07b2cc60 2023-10-18T12:33:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f663cd9c700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b27650 0x558e07b27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:33:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f66377fe700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:33:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.866+0000 7f6637fff700 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 0x558e07b2d040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66280a2000 tx=0x7f662809a8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb1da65782e7552 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f661e7fc700 1 -- 172.21.15.70:0/573667498 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662809e120 con 0x558e07b2cc60 2023-10-18T12:33:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f661e7fc700 1 -- 172.21.15.70:0/573667498 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66280a2530 con 0x558e07b2cc60 2023-10-18T12:33:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f661e7fc700 1 -- 172.21.15.70:0/573667498 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628099690 con 0x558e07b2cc60 2023-10-18T12:33:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 -- 172.21.15.70:0/573667498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 msgr2=0x558e07b2d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 0x558e07b2d040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66280a2000 tx=0x7f662809a8a0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 -- 172.21.15.70:0/573667498 shutdown_connections 2023-10-18T12:33:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b27650 0x558e07b27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2cc60 0x558e07b2d040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 --2- 172.21.15.70:0/573667498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 -- 172.21.15.70:0/573667498 >> 172.21.15.70:0/573667498 conn(0x558e07a9b7a0 msgr2=0x558e07b21be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:30.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.867+0000 7f6652112b80 1 -- 172.21.15.70:0/573667498 shutdown_connections 2023-10-18T12:33:30.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.868+0000 7f6652112b80 1 -- 172.21.15.70:0/573667498 wait complete. 2023-10-18T12:33:30.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.868+0000 7f6652112b80 1 -- start start 2023-10-18T12:33:30.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.868+0000 7f6652112b80 1 Processor -- start 2023-10-18T12:33:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 0x558e07b30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b30dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 0x558e07b31300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07a79ef0 con 0x558e07b27650 2023-10-18T12:33:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07a16790 con 0x558e07b2d580 2023-10-18T12:33:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07a16430 con 0x558e07b2cc60 2023-10-18T12:33:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6652112b80 10 client.0 _ll_register_callbacks cb 0x558e07960b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6637fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 0x558e07b30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 0x558e07b31300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6637fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 0x558e07b30880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60014/0 (socket says 172.21.15.70:60014) 2023-10-18T12:33:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 0x558e07b31300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43444/0 (socket says 172.21.15.70:43444) 2023-10-18T12:33:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6637fff700 1 -- 172.21.15.70:0/1940258484 learned_addr learned my addr 172.21.15.70:0/1940258484 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f66377fe700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b30dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 -- 172.21.15.70:0/1940258484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 msgr2=0x558e07b30dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b30dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 -- 172.21.15.70:0/1940258484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 msgr2=0x558e07b30880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 0x558e07b30880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f663cd9c700 1 -- 172.21.15.70:0/1940258484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66381a5ad0 con 0x558e07b2d580 2023-10-18T12:33:30.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.869+0000 7f6637fff700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 0x558e07b30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:30.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f663cd9c700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 0x558e07b31300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f663000b6c0 tx=0x7f6630007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4726b162ddf5fd8d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:30.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f66177fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:33:30.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f66177fe700 1 -- 172.21.15.70:0/1940258484 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000e040 con 0x558e07b2d580 2023-10-18T12:33:30.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e07a7ef70 con 0x558e07b2d580 2023-10-18T12:33:30.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f66177fe700 1 -- 172.21.15.70:0/1940258484 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f66300078b0 con 0x558e07b2d580 2023-10-18T12:33:30.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f66177fe700 1 -- 172.21.15.70:0/1940258484 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f663000bbf0 con 0x558e07b2d580 2023-10-18T12:33:30.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f6652112b80 10 client.26633 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:30.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558e078a4e60 con 0x558e07b2d580 2023-10-18T12:33:30.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.870+0000 7f66177fe700 1 -- 172.21.15.70:0/1940258484 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e07a7ef70 con 0x558e07b2d580 2023-10-18T12:33:30.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f66177fe700 1 -- 172.21.15.70:0/1940258484 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6630022020 con 0x558e07b2d580 2023-10-18T12:33:30.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6652112b80 10 client.26633 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:30.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6652112b80 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:30.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6652112b80 20 client.26633 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:30.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6652112b80 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b40ef0 0x558e07b43310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6652112b80 4 client.26633 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:33:30.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e07af6930 con 0x558e07b40ef0 2023-10-18T12:33:30.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6637fff700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b40ef0 0x558e07b43310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f6637fff700 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b40ef0 0x558e07b43310 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987a3fe1220943a2 server_cookie=43714148f32685e1 in_seq=0 out_seq=0 2023-10-18T12:33:30.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.871+0000 7f66177fe700 10 client.26633 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:30.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.872+0000 7f66177fe700 1 -- 172.21.15.70:0/1940258484 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e07af6930 con 0x558e07b40ef0 2023-10-18T12:33:30.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.872+0000 7f66177fe700 10 client.26633 handle_command_reply: tid=0 2023-10-18T12:33:30.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.872+0000 7f6652112b80 1 client.26633 shutdown 2023-10-18T12:33:30.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 msgr2=0x558e07b31300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 0x558e07b31300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f663000b6c0 tx=0x7f6630007f90 comp rx=0 tx=0).stop 2023-10-18T12:33:30.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 shutdown_connections 2023-10-18T12:33:30.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b40ef0 0x558e07b43310 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b2d580 0x558e07b31300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b27650 0x558e07b30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 --2- 172.21.15.70:0/1940258484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b30dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 >> 172.21.15.70:0/1940258484 conn(0x558e07a9b7a0 msgr2=0x558e07b26100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:30.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f66177fe700 0 client.26633 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:30.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 shutdown_connections 2023-10-18T12:33:30.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 1 -- 172.21.15.70:0/1940258484 wait complete. 2023-10-18T12:33:30.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.873+0000 7f6652112b80 20 client.26633 trim_cache size 0 max 16384 2023-10-18T12:33:30.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.892+0000 7f6652112b80 1 Processor -- start 2023-10-18T12:33:30.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.892+0000 7f6652112b80 1 -- start start 2023-10-18T12:33:30.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6652112b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b2afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6652112b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 0x558e07b37f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6652112b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2d580 0x558e07b2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6652112b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e078fa9a0 con 0x558e07b2cc60 2023-10-18T12:33:30.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6652112b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07b31ef0 con 0x558e07b27650 2023-10-18T12:33:30.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6652112b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07b1e2e0 con 0x558e07b2d580 2023-10-18T12:33:30.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f66377fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 0x558e07b37f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6637fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b2afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6637fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b2afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43452/0 (socket says 172.21.15.70:43452) 2023-10-18T12:33:30.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f6637fff700 1 -- 172.21.15.70:0/785962295 learned_addr learned my addr 172.21.15.70:0/785962295 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:30.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f663cd9c700 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2d580 0x558e07b2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f66377fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 0x558e07b37f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60016/0 (socket says 172.21.15.70:60016) 2023-10-18T12:33:30.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f663cd9c700 1 -- 172.21.15.70:0/785962295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 msgr2=0x558e07b2afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f663cd9c700 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b2afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f663cd9c700 1 -- 172.21.15.70:0/785962295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 msgr2=0x558e07b37f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f663cd9c700 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 0x558e07b37f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f663cd9c700 1 -- 172.21.15.70:0/785962295 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0788dce0 con 0x558e07b2d580 2023-10-18T12:33:30.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.893+0000 7f66377fe700 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 0x558e07b37f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:30.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.894+0000 7f663cd9c700 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2d580 0x558e07b2c0e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6630009790 tx=0x7f6630009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3971bbabcfdf17 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:30.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.894+0000 7f661effd700 1 -- 172.21.15.70:0/785962295 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000e030 con 0x558e07b2d580 2023-10-18T12:33:30.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.894+0000 7f661effd700 1 -- 172.21.15.70:0/785962295 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f663001c650 con 0x558e07b2d580 2023-10-18T12:33:30.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.894+0000 7f661effd700 1 -- 172.21.15.70:0/785962295 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663001ce40 con 0x558e07b2d580 2023-10-18T12:33:30.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 -- 172.21.15.70:0/785962295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2d580 msgr2=0x558e07b2c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2d580 0x558e07b2c0e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6630009790 tx=0x7f6630009f10 comp rx=0 tx=0).stop 2023-10-18T12:33:30.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 -- 172.21.15.70:0/785962295 shutdown_connections 2023-10-18T12:33:30.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2d580 0x558e07b2c0e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b2afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 --2- 172.21.15.70:0/785962295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2cc60 0x558e07b37f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 -- 172.21.15.70:0/785962295 >> 172.21.15.70:0/785962295 conn(0x558e07a9b7a0 msgr2=0x558e07b2b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:30.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 -- 172.21.15.70:0/785962295 shutdown_connections 2023-10-18T12:33:30.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 -- 172.21.15.70:0/785962295 wait complete. 2023-10-18T12:33:30.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.895+0000 7f6652112b80 1 -- start start 2023-10-18T12:33:30.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 Processor -- start 2023-10-18T12:33:30.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b36b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b37070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07b38510 con 0x558e07b2d580 2023-10-18T12:33:30.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07b94cf0 con 0x558e07b27650 2023-10-18T12:33:30.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e07b94e80 con 0x558e07b2cc60 2023-10-18T12:33:30.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6652112b80 10 client.0 _ll_register_callbacks cb 0x558e07960b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:30.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.896+0000 7f6637fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b365f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43462/0 (socket says 172.21.15.70:43462) 2023-10-18T12:33:30.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 -- 172.21.15.70:0/1906767319 learned_addr learned my addr 172.21.15.70:0/1906767319 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:30.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f66377fe700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b36b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f663cd9c700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b37070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 -- 172.21.15.70:0/1906767319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 msgr2=0x558e07b36b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b36b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 -- 172.21.15.70:0/1906767319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 msgr2=0x558e07b37070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:30.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b37070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 -- 172.21.15.70:0/1906767319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c017070 con 0x558e07b27650 2023-10-18T12:33:30.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f663cd9c700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b37070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:33:30.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f6637fff700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b365f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f66280966a0 tx=0x7f662800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ac973d15494d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:30.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.897+0000 7f66177fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:33:30.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.898+0000 7f66177fe700 1 -- 172.21.15.70:0/1906767319 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280985c0 con 0x558e07b27650 2023-10-18T12:33:30.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.898+0000 7f66177fe700 1 -- 172.21.15.70:0/1906767319 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f66280b8300 con 0x558e07b27650 2023-10-18T12:33:30.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.898+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66280a2530 con 0x558e07b27650 2023-10-18T12:33:30.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f66177fe700 1 -- 172.21.15.70:0/1906767319 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66280b84c0 con 0x558e07b27650 2023-10-18T12:33:30.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 10 client.26639 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:30.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558e078a4e60 con 0x558e07b27650 2023-10-18T12:33:30.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f66177fe700 1 -- 172.21.15.70:0/1906767319 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66280a2530 con 0x558e07b27650 2023-10-18T12:33:30.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f66177fe700 1 -- 172.21.15.70:0/1906767319 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6628080050 con 0x558e07b27650 2023-10-18T12:33:30.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 10 client.26639 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:30.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:30.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 20 client.26639 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:30.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b96560 0x558e07b98980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:30.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 4 client.26639 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:33:30.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.899+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558e07a72ed0 con 0x558e07b96560 2023-10-18T12:33:30.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.900+0000 7f66377fe700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b96560 0x558e07b98980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:30.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.900+0000 7f66377fe700 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b96560 0x558e07b98980 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4936332de7ce25 server_cookie=c47921cb4751ffb3 in_seq=0 out_seq=0 2023-10-18T12:33:30.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.900+0000 7f66177fe700 10 client.26639 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:30.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.901+0000 7f66177fe700 1 -- 172.21.15.70:0/1906767319 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558e07a72ed0 con 0x558e07b96560 2023-10-18T12:33:30.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f66177fe700 10 client.26639 handle_command_reply: tid=0 2023-10-18T12:33:30.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 client.26639 shutdown 2023-10-18T12:33:30.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 msgr2=0x558e07b365f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b365f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f66280966a0 tx=0x7f662800c6e0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 shutdown_connections 2023-10-18T12:33:30.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558e07b96560 0x558e07b98980 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e07b2d580 0x558e07b37070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558e07b27650 0x558e07b365f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 --2- 172.21.15.70:0/1906767319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e07b2cc60 0x558e07b36b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 >> 172.21.15.70:0/1906767319 conn(0x558e07a9b7a0 msgr2=0x558e07b2ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:30.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f66177fe700 0 client.26639 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:30.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 shutdown_connections 2023-10-18T12:33:30.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.902+0000 7f6652112b80 1 -- 172.21.15.70:0/1906767319 wait complete. 2023-10-18T12:33:30.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.903+0000 7f6652112b80 20 client.26639 trim_cache size 0 max 16384 2023-10-18T12:33:30.922 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:33:30.922 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:33:30.923 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "3e6fc337-5a94-4e77-9021-d4382d20b631", 2023-10-18T12:33:30.923 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:33:30.923 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:33:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.903+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f66200a1670 msgr2=0x7f66200a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.903+0000 7f663e7ff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f66200a1670 0x7f66200a3b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f663819a240 tx=0x7f662c009310 comp rx=0 tx=0).stop 2023-10-18T12:33:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 msgr2=0x7f6638198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 0x7f6638198d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f662800e9a0 tx=0x7f662800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 shutdown_connections 2023-10-18T12:33:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f66200a1670 0x7f66200a3b30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6638106720 0x7f663819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6638100710 0x7f6638198d80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 --2- 172.21.15.70:0/438508750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66381010d0 0x7f66381992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 >> 172.21.15.70:0/438508750 conn(0x7f66380fc200 msgr2=0x7f66380fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 shutdown_connections 2023-10-18T12:33:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:30.904+0000 7f663e7ff700 1 -- 172.21.15.70:0/438508750 wait complete. 2023-10-18T12:33:30.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:33:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 -- 172.21.15.70:0/3515625986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8106730 msgr2=0x7fedb8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 --2- 172.21.15.70:0/3515625986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8106730 0x7fedb8106b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009990 tx=0x7fedb4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 -- 172.21.15.70:0/3515625986 shutdown_connections 2023-10-18T12:33:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 --2- 172.21.15.70:0/3515625986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb81010e0 0x7fedb810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 --2- 172.21.15.70:0/3515625986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8106730 0x7fedb8106b10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 --2- 172.21.15.70:0/3515625986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb8100720 0x7fedb8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.149+0000 7fedbfbff700 1 -- 172.21.15.70:0/3515625986 >> 172.21.15.70:0/3515625986 conn(0x7fedb80fc1b0 msgr2=0x7fedb80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.150+0000 7fedbfbff700 1 -- 172.21.15.70:0/3515625986 shutdown_connections 2023-10-18T12:33:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.150+0000 7fedbfbff700 1 -- 172.21.15.70:0/3515625986 wait complete. 2023-10-18T12:33:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.151+0000 7fedbfbff700 1 Processor -- start 2023-10-18T12:33:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.151+0000 7fedbfbff700 1 -- start start 2023-10-18T12:33:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbfbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 0x7fedb81a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbfbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb81010e0 0x7fedb81a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbfbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb8106730 0x7fedb810b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbfbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb810bc50 con 0x7fedb81010e0 2023-10-18T12:33:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbfbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb810bdc0 con 0x7fedb8100720 2023-10-18T12:33:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbfbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb810bf80 con 0x7fedb8106730 2023-10-18T12:33:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb81010e0 0x7fedb81a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 0x7fedb81a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 0x7fedb81a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43488/0 (socket says 172.21.15.70:43488) 2023-10-18T12:33:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb81010e0 0x7fedb81a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60042/0 (socket says 172.21.15.70:60042) 2023-10-18T12:33:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 -- 172.21.15.70:0/2867168643 learned_addr learned my addr 172.21.15.70:0/2867168643 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbe19c700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb8106730 0x7fedb810b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 -- 172.21.15.70:0/2867168643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb8106730 msgr2=0x7fedb810b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb8106730 0x7fedb810b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 -- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb81010e0 msgr2=0x7fedb81a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb81010e0 0x7fedb81a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbd99b700 1 -- 172.21.15.70:0/2867168643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40095f0 con 0x7fedb8100720 2023-10-18T12:33:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.152+0000 7fedbe19c700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb8106730 0x7fedb810b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.153+0000 7fedbd99b700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 0x7fedb81a1db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fedb4000c00 tx=0x7fedb4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.153+0000 7fedaeffd700 1 -- 172.21.15.70:0/2867168643 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb401d070 con 0x7fedb8100720 2023-10-18T12:33:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.153+0000 7fedaeffd700 1 -- 172.21.15.70:0/2867168643 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedb4022470 con 0x7fedb8100720 2023-10-18T12:33:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.153+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb810c1a0 con 0x7fedb8100720 2023-10-18T12:33:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.153+0000 7fedaeffd700 1 -- 172.21.15.70:0/2867168643 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4017650 con 0x7fedb8100720 2023-10-18T12:33:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.153+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb81a9eb0 con 0x7fedb8100720 2023-10-18T12:33:31.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.156+0000 7fedaeffd700 1 -- 172.21.15.70:0/2867168643 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fedb40225e0 con 0x7fedb8100720 2023-10-18T12:33:31.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.157+0000 7fedd34ddb80 1 Processor -- start 2023-10-18T12:33:31.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.157+0000 7fedd34ddb80 1 -- start start 2023-10-18T12:33:31.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedaeffd700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda40a1670 0x7feda40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 0x560e70f59090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70ecc6e0 con 0x560e70f5ebe0 2023-10-18T12:33:31.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70eb2c60 con 0x560e70f5f500 2023-10-18T12:33:31.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70eb0580 con 0x560e70f595d0 2023-10-18T12:33:31.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedaeffd700 1 -- 172.21.15.70:0/2867168643 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fedb40ce670 con 0x7fedb8100720 2023-10-18T12:33:31.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbe19c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 0x560e70f59090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbd99b700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda40a1670 0x7feda40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbe19c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f5efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60056/0 (socket says 172.21.15.70:60056) 2023-10-18T12:33:31.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 0x560e70f59090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43498/0 (socket says 172.21.15.70:43498) 2023-10-18T12:33:31.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbe19c700 1 -- 172.21.15.70:0/393237268 learned_addr learned my addr 172.21.15.70:0/393237268 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:31.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.158+0000 7fedbe19c700 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd19a700 1 -- 172.21.15.70:0/393237268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 msgr2=0x560e70f59a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd19a700 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f59a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd19a700 1 -- 172.21.15.70:0/393237268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 msgr2=0x560e70f5efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd19a700 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f5efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd19a700 1 -- 172.21.15.70:0/393237268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e70e693f0 con 0x560e70f5f500 2023-10-18T12:33:31.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbe19c700 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:33:31.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd19a700 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 0x560e70f59090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feda8009c80 tx=0x7feda8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69348ed44e4a3e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:31.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7fedbd99b700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda40a1670 0x7feda40a3b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedb4097eb0 tx=0x7fedb4022f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:31.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7feda3fff700 1 -- 172.21.15.70:0/393237268 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8011cb0 con 0x560e70f5f500 2023-10-18T12:33:31.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7feda3fff700 1 -- 172.21.15.70:0/393237268 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda8011e10 con 0x560e70f5f500 2023-10-18T12:33:31.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.159+0000 7feda3fff700 1 -- 172.21.15.70:0/393237268 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8005280 con 0x560e70f5f500 2023-10-18T12:33:31.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 -- 172.21.15.70:0/393237268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 msgr2=0x560e70f59090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 0x560e70f59090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feda8009c80 tx=0x7feda8009f90 comp rx=0 tx=0).stop 2023-10-18T12:33:31.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 -- 172.21.15.70:0/393237268 shutdown_connections 2023-10-18T12:33:31.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/393237268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5f500 0x560e70f59090 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 -- 172.21.15.70:0/393237268 >> 172.21.15.70:0/393237268 conn(0x560e70ecd720 msgr2=0x560e70f53b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 -- 172.21.15.70:0/393237268 shutdown_connections 2023-10-18T12:33:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.160+0000 7fedd34ddb80 1 -- 172.21.15.70:0/393237268 wait complete. 2023-10-18T12:33:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 -- start start 2023-10-18T12:33:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 Processor -- start 2023-10-18T12:33:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 0x560e70f65500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70f62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70eabe70 con 0x560e70f595d0 2023-10-18T12:33:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70e48710 con 0x560e70f5ebe0 2023-10-18T12:33:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.161+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70e483b0 con 0x560e70f5f500 2023-10-18T12:33:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedd34ddb80 10 client.0 _ll_register_callbacks cb 0x560e70d92a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 0x560e70f65500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f62780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43506/0 (socket says 172.21.15.70:43506) 2023-10-18T12:33:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbd19a700 1 -- 172.21.15.70:0/1687464154 learned_addr learned my addr 172.21.15.70:0/1687464154 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:31.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 0x560e70f65500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60062/0 (socket says 172.21.15.70:60062) 2023-10-18T12:33:31.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 -- 172.21.15.70:0/1687464154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 msgr2=0x560e70f62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70f62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 -- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 msgr2=0x560e70f62780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f62780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 -- 172.21.15.70:0/1687464154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda80098e0 con 0x560e70f595d0 2023-10-18T12:33:31.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbd19a700 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f62780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:31.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 0x560e70f65500 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009fd0 tx=0x7fedb0004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e834893602a430 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:31.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.162+0000 7feda17fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:33:31.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7feda17fa700 1 -- 172.21.15.70:0/1687464154 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00090d0 con 0x560e70f595d0 2023-10-18T12:33:31.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7feda17fa700 1 -- 172.21.15.70:0/1687464154 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fedb0009230 con 0x560e70f595d0 2023-10-18T12:33:31.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e70eb0ef0 con 0x560e70f595d0 2023-10-18T12:33:31.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7feda17fa700 1 -- 172.21.15.70:0/1687464154 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb00093f0 con 0x560e70f595d0 2023-10-18T12:33:31.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 10 client.16617 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:31.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560e70cd6de0 con 0x560e70f595d0 2023-10-18T12:33:31.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7feda17fa700 1 -- 172.21.15.70:0/1687464154 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e70eb0ef0 con 0x560e70f595d0 2023-10-18T12:33:31.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7feda17fa700 1 -- 172.21.15.70:0/1687464154 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fedb001a590 con 0x560e70f595d0 2023-10-18T12:33:31.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 10 client.16617 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:31.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:31.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 20 client.16617 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:31.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70f72ff0 0x560e70f75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 4 client.16617 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:33:31.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.163+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e70f288b0 con 0x560e70f72ff0 2023-10-18T12:33:31.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.164+0000 7fedbd19a700 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70f72ff0 0x560e70f75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.164+0000 7fedbd19a700 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70f72ff0 0x560e70f75410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02cca9e78a4c148 server_cookie=2d8a3129e11b6627 in_seq=0 out_seq=0 2023-10-18T12:33:31.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.164+0000 7feda17fa700 10 client.16617 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:31.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7feda17fa700 1 -- 172.21.15.70:0/1687464154 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e70f288b0 con 0x560e70f72ff0 2023-10-18T12:33:31.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7feda17fa700 10 client.16617 handle_command_reply: tid=0 2023-10-18T12:33:31.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7fedd34ddb80 1 client.16617 shutdown 2023-10-18T12:33:31.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 msgr2=0x560e70f65500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 0x560e70f65500 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009fd0 tx=0x7fedb0004980 comp rx=0 tx=0).stop 2023-10-18T12:33:31.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 shutdown_connections 2023-10-18T12:33:31.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.165+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70f72ff0 0x560e70f75410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70f62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f595d0 0x560e70f65500 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7feda17fa700 0 client.16617 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:31.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1687464154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f62780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 >> 172.21.15.70:0/1687464154 conn(0x560e70ecd720 msgr2=0x560e70f58080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:31.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 shutdown_connections 2023-10-18T12:33:31.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1687464154 wait complete. 2023-10-18T12:33:31.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.166+0000 7fedd34ddb80 20 client.16617 trim_cache size 0 max 16384 2023-10-18T12:33:31.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.183+0000 7fedd34ddb80 1 Processor -- start 2023-10-18T12:33:31.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 -- start start 2023-10-18T12:33:31.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f5cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f69f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5f500 0x560e70f65030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70f4eaf0 con 0x560e70f5f500 2023-10-18T12:33:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70d2c920 con 0x560e70f5ebe0 2023-10-18T12:33:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70f553b0 con 0x560e70f595d0 2023-10-18T12:33:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f69f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f69f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43514/0 (socket says 172.21.15.70:43514) 2023-10-18T12:33:31.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedbd19a700 1 -- 172.21.15.70:0/568070687 learned_addr learned my addr 172.21.15.70:0/568070687 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:31.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.184+0000 7fedbe19c700 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5f500 0x560e70f65030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbe19c700 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f5cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbd19a700 1 -- 172.21.15.70:0/568070687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 msgr2=0x560e70f5cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbd19a700 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f5cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbd19a700 1 -- 172.21.15.70:0/568070687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5f500 msgr2=0x560e70f65030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbd19a700 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5f500 0x560e70f65030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbd19a700 1 -- 172.21.15.70:0/568070687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e70cbfc60 con 0x560e70f5ebe0 2023-10-18T12:33:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbe19c700 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f5cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:33:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedbd19a700 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f69f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feda8004d10 tx=0x7feda8004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d5c001250b5392 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedacbff700 1 -- 172.21.15.70:0/568070687 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8004050 con 0x560e70f5ebe0 2023-10-18T12:33:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedacbff700 1 -- 172.21.15.70:0/568070687 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda8004670 con 0x560e70f5ebe0 2023-10-18T12:33:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.185+0000 7fedacbff700 1 -- 172.21.15.70:0/568070687 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8004d40 con 0x560e70f5ebe0 2023-10-18T12:33:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 -- 172.21.15.70:0/568070687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 msgr2=0x560e70f69f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f69f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feda8004d10 tx=0x7feda8004a60 comp rx=0 tx=0).stop 2023-10-18T12:33:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 -- 172.21.15.70:0/568070687 shutdown_connections 2023-10-18T12:33:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5f500 0x560e70f65030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f595d0 0x560e70f5cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/568070687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f5ebe0 0x560e70f69f40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 -- 172.21.15.70:0/568070687 >> 172.21.15.70:0/568070687 conn(0x560e70ecd720 msgr2=0x560e70f5c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 -- 172.21.15.70:0/568070687 shutdown_connections 2023-10-18T12:33:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.186+0000 7fedd34ddb80 1 -- 172.21.15.70:0/568070687 wait complete. 2023-10-18T12:33:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 -- start start 2023-10-18T12:33:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 Processor -- start 2023-10-18T12:33:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 0x560e70f68110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f68650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70fc5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70fc6360 con 0x560e70f5ebe0 2023-10-18T12:33:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70fc64a0 con 0x560e70f595d0 2023-10-18T12:33:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e70fc6610 con 0x560e70f5f500 2023-10-18T12:33:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.187+0000 7fedd34ddb80 10 client.0 _ll_register_callbacks cb 0x560e70d92a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:33:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 0x560e70f68110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 0x560e70f68110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:43528/0 (socket says 172.21.15.70:43528) 2023-10-18T12:33:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 -- 172.21.15.70:0/1459732627 learned_addr learned my addr 172.21.15.70:0/1459732627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:33:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbd19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f68650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70fc5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 -- 172.21.15.70:0/1459732627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 msgr2=0x560e70fc5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70fc5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 -- 172.21.15.70:0/1459732627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 msgr2=0x560e70f68650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:33:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f68650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 -- 172.21.15.70:0/1459732627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda801d040 con 0x560e70f595d0 2023-10-18T12:33:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbd19a700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f68650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:33:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7fedbe19c700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 0x560e70f68110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fedb00099e0 tx=0x7fedb00021f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270ae7015d5cf4a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:33:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7feda17fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:33:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.188+0000 7feda17fa700 1 -- 172.21.15.70:0/1459732627 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb001acf0 con 0x560e70f595d0 2023-10-18T12:33:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feda8011e10 con 0x560e70f595d0 2023-10-18T12:33:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7feda17fa700 1 -- 172.21.15.70:0/1459732627 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fedb001a470 con 0x560e70f595d0 2023-10-18T12:33:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7feda17fa700 1 -- 172.21.15.70:0/1459732627 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb001a630 con 0x560e70f595d0 2023-10-18T12:33:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 10 client.26657 fetch_fsmap learned FSMap version 20 2023-10-18T12:33:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560e70cfd250 con 0x560e70f595d0 2023-10-18T12:33:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7feda17fa700 1 -- 172.21.15.70:0/1459732627 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda8011e10 con 0x560e70f595d0 2023-10-18T12:33:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7feda17fa700 1 -- 172.21.15.70:0/1459732627 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fedb0007620 con 0x560e70f595d0 2023-10-18T12:33:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 10 client.26657 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:33:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:33:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 20 client.26657 populate_metadata read hostname 'smithi070' 2023-10-18T12:33:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70fc7d50 0x560e70fca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:33:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 4 client.26657 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:33:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e70ea4e50 con 0x560e70fc7d50 2023-10-18T12:33:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.189+0000 7fedbd19a700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70fc7d50 0x560e70fca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:33:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.190+0000 7fedbd19a700 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70fc7d50 0x560e70fca170 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f47c4ff848eb636 server_cookie=16d0cf24f25c28c0 in_seq=0 out_seq=0 2023-10-18T12:33:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.190+0000 7feda17fa700 10 client.26657 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.191+0000 7feda17fa700 1 -- 172.21.15.70:0/1459732627 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560e70ea4e50 con 0x560e70fc7d50 2023-10-18T12:33:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.191+0000 7feda17fa700 10 client.26657 handle_command_reply: tid=0 2023-10-18T12:33:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.191+0000 7fedd34ddb80 1 client.26657 shutdown 2023-10-18T12:33:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 msgr2=0x560e70f68110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 0x560e70f68110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fedb00099e0 tx=0x7fedb00021f0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 shutdown_connections 2023-10-18T12:33:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x560e70fc7d50 0x560e70fca170 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e70f5f500 0x560e70fc5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e70f595d0 0x560e70f68110 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7feda17fa700 0 client.26657 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:33:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 --2- 172.21.15.70:0/1459732627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560e70f5ebe0 0x560e70f68650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 >> 172.21.15.70:0/1459732627 conn(0x560e70ecd720 msgr2=0x560e70f4d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 shutdown_connections 2023-10-18T12:33:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 1 -- 172.21.15.70:0/1459732627 wait complete. 2023-10-18T12:33:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.192+0000 7fedd34ddb80 20 client.26657 trim_cache size 0 max 16384 2023-10-18T12:33:31.212 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:33:31.212 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:33:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:33:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "3e6fc337-5a94-4e77-9021-d4382d20b631": { 2023-10-18T12:33:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:33:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "3e6fc337-5a94-4e77-9021-d4382d20b631", 2023-10-18T12:33:31.213 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:33:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:33:31.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:33:31.214 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:33:31.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda40a1670 msgr2=0x7feda40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda40a1670 0x7feda40a3b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedb4097eb0 tx=0x7fedb4022f90 comp rx=0 tx=0).stop 2023-10-18T12:33:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 msgr2=0x7fedb81a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:33:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 0x7fedb81a1db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fedb4000c00 tx=0x7fedb4005740 comp rx=0 tx=0).stop 2023-10-18T12:33:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 shutdown_connections 2023-10-18T12:33:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb8106730 0x7fedb810b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feda40a1670 0x7feda40a3b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedb8100720 0x7fedb81a1db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 --2- 172.21.15.70:0/2867168643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedb81010e0 0x7fedb81a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:33:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 >> 172.21.15.70:0/2867168643 conn(0x7fedb80fc1b0 msgr2=0x7fedb8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:33:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 shutdown_connections 2023-10-18T12:33:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:33:31.193+0000 7fedbfbff700 1 -- 172.21.15.70:0/2867168643 wait complete. 2023-10-18T12:33:31.218 INFO:tasks.cephfs.filesystem:scrub status for tag:3e6fc337-5a94-4e77-9021-d4382d20b631 - {'path': '/', 'tag': '3e6fc337-5a94-4e77-9021-d4382d20b631', 'options': 'recursive,force'} 2023-10-18T12:33:31.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:31 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:33:31.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:31 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:33:31.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:31 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:33:31.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:31 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:33:31.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:31 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:33:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:31 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:33:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:31 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:33:31.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:31 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:33:31.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:31 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:33:31.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:31 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:33:31.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:31 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:33:31.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:31 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:33:31.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 833748 11.85 MB/sec execute 240 sec latency 34.941 ms 2023-10-18T12:33:32.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:32 smithi070 ceph-mon[98639]: pgmap v542: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 228 op/s 2023-10-18T12:33:32.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:32 smithi152 ceph-mon[134767]: pgmap v542: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 228 op/s 2023-10-18T12:33:32.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:32 smithi186 ceph-mon[135253]: pgmap v542: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 228 op/s 2023-10-18T12:33:32.858 INFO:tasks.workunit.client.0.smithi070.stdout: 1 835602 11.84 MB/sec execute 241 sec latency 19.445 ms 2023-10-18T12:33:33.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 837867 11.85 MB/sec execute 242 sec latency 209.987 ms 2023-10-18T12:33:34.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 839734 11.83 MB/sec execute 243 sec latency 13.006 ms 2023-10-18T12:33:35.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:34 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:33:35.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:34 smithi070 ceph-mon[98639]: pgmap v543: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2023-10-18T12:33:35.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:34 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:33:35.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:34 smithi152 ceph-mon[134767]: pgmap v543: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2023-10-18T12:33:35.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:34 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:33:35.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:34 smithi186 ceph-mon[135253]: pgmap v543: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2023-10-18T12:33:35.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 841866 11.84 MB/sec execute 244 sec latency 57.790 ms 2023-10-18T12:33:36.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 844031 11.82 MB/sec execute 245 sec latency 36.132 ms 2023-10-18T12:33:37.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:36 smithi070 ceph-mon[98639]: pgmap v544: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 218 op/s 2023-10-18T12:33:37.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:36 smithi152 ceph-mon[134767]: pgmap v544: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 218 op/s 2023-10-18T12:33:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:36 smithi186 ceph-mon[135253]: pgmap v544: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 218 op/s 2023-10-18T12:33:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 846258 11.83 MB/sec execute 246 sec latency 12.081 ms 2023-10-18T12:33:38.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 848818 11.85 MB/sec execute 247 sec latency 32.260 ms 2023-10-18T12:33:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: pgmap v545: 129 pgs: 129 active+clean; 453 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 202 op/s 2023-10-18T12:33:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:33:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:38 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: pgmap v545: 129 pgs: 129 active+clean; 453 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 202 op/s 2023-10-18T12:33:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:33:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:38 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: pgmap v545: 129 pgs: 129 active+clean; 453 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 202 op/s 2023-10-18T12:33:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:33:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:38 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:39.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 850923 11.83 MB/sec execute 248 sec latency 12.472 ms 2023-10-18T12:33:40.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi152 to 196.2M 2023-10-18T12:33:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-18T12:33:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: Unable to set osd_memory_target on smithi152 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-18T12:33:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-18T12:33:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: Adjusting osd_memory_target on smithi070 to 221.9M 2023-10-18T12:33:40.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: Unable to set osd_memory_target on smithi070 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-10-18T12:33:40.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:33:40.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:33:40.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:33:40.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi152 to 196.2M 2023-10-18T12:33:40.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-18T12:33:40.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: Unable to set osd_memory_target on smithi152 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-18T12:33:40.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-18T12:33:40.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: Adjusting osd_memory_target on smithi070 to 221.9M 2023-10-18T12:33:40.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: Unable to set osd_memory_target on smithi070 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-10-18T12:33:40.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:33:40.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:33:40.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:33:40.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi152 to 196.2M 2023-10-18T12:33:40.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-18T12:33:40.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: Unable to set osd_memory_target on smithi152 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-18T12:33:40.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-18T12:33:40.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T12:33:40.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: Adjusting osd_memory_target on smithi070 to 221.9M 2023-10-18T12:33:40.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: Unable to set osd_memory_target on smithi070 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-10-18T12:33:40.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:33:40.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:33:40.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:33:40.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 852677 11.83 MB/sec execute 249 sec latency 61.926 ms 2023-10-18T12:33:41.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:40 smithi070 ceph-mon[98639]: pgmap v546: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 199 op/s 2023-10-18T12:33:41.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:40 smithi152 ceph-mon[134767]: pgmap v546: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 199 op/s 2023-10-18T12:33:41.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:40 smithi186 ceph-mon[135253]: pgmap v546: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 199 op/s 2023-10-18T12:33:41.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 854719 11.82 MB/sec execute 250 sec latency 49.771 ms 2023-10-18T12:33:42.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 856850 11.82 MB/sec execute 251 sec latency 19.650 ms 2023-10-18T12:33:43.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:42 smithi070 ceph-mon[98639]: pgmap v547: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2023-10-18T12:33:43.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:42 smithi152 ceph-mon[134767]: pgmap v547: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2023-10-18T12:33:43.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:42 smithi186 ceph-mon[135253]: pgmap v547: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 248 op/s 2023-10-18T12:33:43.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 859454 11.84 MB/sec execute 252 sec latency 33.790 ms 2023-10-18T12:33:44.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 861029 11.82 MB/sec execute 253 sec latency 83.937 ms 2023-10-18T12:33:45.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:44 smithi070 ceph-mon[98639]: pgmap v548: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 217 op/s 2023-10-18T12:33:45.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:44 smithi152 ceph-mon[134767]: pgmap v548: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 217 op/s 2023-10-18T12:33:45.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:44 smithi186 ceph-mon[135253]: pgmap v548: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 217 op/s 2023-10-18T12:33:45.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 863190 11.83 MB/sec execute 254 sec latency 84.837 ms 2023-10-18T12:33:46.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 865303 11.81 MB/sec execute 255 sec latency 34.583 ms 2023-10-18T12:33:47.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:46 smithi070 ceph-mon[98639]: pgmap v549: 129 pgs: 129 active+clean; 465 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 191 op/s 2023-10-18T12:33:47.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:46 smithi152 ceph-mon[134767]: pgmap v549: 129 pgs: 129 active+clean; 465 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 191 op/s 2023-10-18T12:33:47.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:46 smithi186 ceph-mon[135253]: pgmap v549: 129 pgs: 129 active+clean; 465 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 191 op/s 2023-10-18T12:33:47.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 867476 11.82 MB/sec execute 256 sec latency 11.332 ms 2023-10-18T12:33:48.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 870075 11.84 MB/sec execute 257 sec latency 22.786 ms 2023-10-18T12:33:49.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:48 smithi070 ceph-mon[98639]: pgmap v550: 129 pgs: 129 active+clean; 471 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2023-10-18T12:33:49.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:48 smithi152 ceph-mon[134767]: pgmap v550: 129 pgs: 129 active+clean; 471 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2023-10-18T12:33:49.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:48 smithi186 ceph-mon[135253]: pgmap v550: 129 pgs: 129 active+clean; 471 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2023-10-18T12:33:49.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 871815 11.81 MB/sec execute 258 sec latency 14.073 ms 2023-10-18T12:33:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:50 smithi070 ceph-mon[98639]: pgmap v551: 129 pgs: 129 active+clean; 475 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2023-10-18T12:33:50.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:50 smithi152 ceph-mon[134767]: pgmap v551: 129 pgs: 129 active+clean; 475 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2023-10-18T12:33:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:50 smithi186 ceph-mon[135253]: pgmap v551: 129 pgs: 129 active+clean; 475 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2023-10-18T12:33:50.859 INFO:tasks.workunit.client.0.smithi070.stdout: 1 873915 11.82 MB/sec execute 259 sec latency 45.418 ms 2023-10-18T12:33:51.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 875792 11.80 MB/sec execute 260 sec latency 46.159 ms 2023-10-18T12:33:52.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 877906 11.81 MB/sec execute 261 sec latency 12.150 ms 2023-10-18T12:33:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:52 smithi070 ceph-mon[98639]: pgmap v552: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2023-10-18T12:33:53.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:52 smithi152 ceph-mon[134767]: pgmap v552: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2023-10-18T12:33:53.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:52 smithi186 ceph-mon[135253]: pgmap v552: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2023-10-18T12:33:53.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 880683 11.83 MB/sec execute 262 sec latency 25.708 ms 2023-10-18T12:33:54.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 882125 11.80 MB/sec execute 263 sec latency 24.251 ms 2023-10-18T12:33:55.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:54 smithi070 ceph-mon[98639]: pgmap v553: 129 pgs: 129 active+clean; 485 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-10-18T12:33:55.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:54 smithi152 ceph-mon[134767]: pgmap v553: 129 pgs: 129 active+clean; 485 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-10-18T12:33:55.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:54 smithi186 ceph-mon[135253]: pgmap v553: 129 pgs: 129 active+clean; 485 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-10-18T12:33:55.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 884451 11.81 MB/sec execute 264 sec latency 38.438 ms 2023-10-18T12:33:56.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 885932 11.79 MB/sec execute 265 sec latency 50.063 ms 2023-10-18T12:33:57.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:56 smithi070 ceph-mon[98639]: pgmap v554: 129 pgs: 129 active+clean; 486 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 213 op/s 2023-10-18T12:33:57.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:56 smithi152 ceph-mon[134767]: pgmap v554: 129 pgs: 129 active+clean; 486 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 213 op/s 2023-10-18T12:33:57.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:56 smithi186 ceph-mon[135253]: pgmap v554: 129 pgs: 129 active+clean; 486 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 213 op/s 2023-10-18T12:33:57.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 888288 11.80 MB/sec execute 266 sec latency 12.749 ms 2023-10-18T12:33:58.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 890860 11.81 MB/sec execute 267 sec latency 14.761 ms 2023-10-18T12:33:59.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:33:58 smithi070 ceph-mon[98639]: pgmap v555: 129 pgs: 129 active+clean; 486 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-10-18T12:33:59.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:33:58 smithi152 ceph-mon[134767]: pgmap v555: 129 pgs: 129 active+clean; 486 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-10-18T12:33:59.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:33:58 smithi186 ceph-mon[135253]: pgmap v555: 129 pgs: 129 active+clean; 486 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-10-18T12:33:59.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 892603 11.80 MB/sec execute 268 sec latency 28.785 ms 2023-10-18T12:34:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:00 smithi070 ceph-mon[98639]: pgmap v556: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 242 op/s 2023-10-18T12:34:00.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:00 smithi152 ceph-mon[134767]: pgmap v556: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 242 op/s 2023-10-18T12:34:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:00 smithi186 ceph-mon[135253]: pgmap v556: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 242 op/s 2023-10-18T12:34:00.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 894361 11.79 MB/sec execute 269 sec latency 49.638 ms 2023-10-18T12:34:01.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:34:01.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 -- 172.21.15.70:0/1937212809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c0100710 msgr2=0x7f81c0100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1937212809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c0100710 0x7f81c0100b90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009990 tx=0x7f81b4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 -- 172.21.15.70:0/1937212809 shutdown_connections 2023-10-18T12:34:01.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1937212809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c01010d0 0x7f81c010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1937212809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 0x7f81c0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1937212809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c0100710 0x7f81c0100b90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 -- 172.21.15.70:0/1937212809 >> 172.21.15.70:0/1937212809 conn(0x7f81c00fc200 msgr2=0x7f81c00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 -- 172.21.15.70:0/1937212809 shutdown_connections 2023-10-18T12:34:01.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.461+0000 7f81c53ff700 1 -- 172.21.15.70:0/1937212809 wait complete. 2023-10-18T12:34:01.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.462+0000 7f81c53ff700 1 Processor -- start 2023-10-18T12:34:01.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.462+0000 7f81c53ff700 1 -- start start 2023-10-18T12:34:01.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81c53ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 0x7f81c0198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81c53ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c01010d0 0x7f81c01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81c53ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 0x7f81c019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81c53ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019dcb0 con 0x7f81c01010d0 2023-10-18T12:34:01.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81c53ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019de20 con 0x7f81c0100710 2023-10-18T12:34:01.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81c53ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019dfe0 con 0x7f81c0106720 2023-10-18T12:34:01.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81beffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 0x7f81c0198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81be7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c01010d0 0x7f81c01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81beffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 0x7f81c0198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59206/0 (socket says 172.21.15.70:59206) 2023-10-18T12:34:01.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81beffd700 1 -- 172.21.15.70:0/1438725180 learned_addr learned my addr 172.21.15.70:0/1438725180 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:01.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.463+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 0x7f81c019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81beffd700 1 -- 172.21.15.70:0/1438725180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 msgr2=0x7f81c019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81beffd700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 0x7f81c019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81beffd700 1 -- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c01010d0 msgr2=0x7f81c01993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81beffd700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c01010d0 0x7f81c01993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81beffd700 1 -- 172.21.15.70:0/1438725180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b40095f0 con 0x7f81c0100710 2023-10-18T12:34:01.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81be7fc700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c01010d0 0x7f81c01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:01.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 0x7f81c019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:01.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81beffd700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 0x7f81c0198e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e9a0 tx=0x7f81b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81a7fff700 1 -- 172.21.15.70:0/1438725180 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0009790 con 0x7f81c0100710 2023-10-18T12:34:01.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c019e2c0 con 0x7f81c0100710 2023-10-18T12:34:01.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81a7fff700 1 -- 172.21.15.70:0/1438725180 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81b0010430 con 0x7f81c0100710 2023-10-18T12:34:01.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81a7fff700 1 -- 172.21.15.70:0/1438725180 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0018680 con 0x7f81c0100710 2023-10-18T12:34:01.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.464+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c01a5b00 con 0x7f81c0100710 2023-10-18T12:34:01.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.467+0000 7f81a7fff700 1 -- 172.21.15.70:0/1438725180 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f81b00105a0 con 0x7f81c0100710 2023-10-18T12:34:01.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81a7fff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f81a80a15f0 0x7f81a80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 Processor -- start 2023-10-18T12:34:01.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81a7fff700 1 -- 172.21.15.70:0/1438725180 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f81b0014070 con 0x7f81c0100710 2023-10-18T12:34:01.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 -- start start 2023-10-18T12:34:01.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81be7fc700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f81a80a15f0 0x7f81a80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b2500 0x562bf23ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf231f6e0 con 0x562bf23b1be0 2023-10-18T12:34:01.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf2305c60 con 0x562bf23ac5d0 2023-10-18T12:34:01.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf2303580 con 0x562bf23b2500 2023-10-18T12:34:01.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81beffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81beffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33876/0 (socket says 172.21.15.70:33876) 2023-10-18T12:34:01.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81beffd700 1 -- 172.21.15.70:0/2344933409 learned_addr learned my addr 172.21.15.70:0/2344933409 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:01.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.469+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81be7fc700 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b2500 0x562bf23ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81beffd700 1 -- 172.21.15.70:0/2344933409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b2500 msgr2=0x562bf23ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81beffd700 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b2500 0x562bf23ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81beffd700 1 -- 172.21.15.70:0/2344933409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 msgr2=0x562bf23aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81beffd700 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81beffd700 1 -- 172.21.15.70:0/2344933409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bf22bc3f0 con 0x562bf23b1be0 2023-10-18T12:34:01.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:01.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81be7fc700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f81a80a15f0 0x7f81a80a3ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81c019a380 tx=0x7f81b400bb60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81beffd700 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b1fc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f81b0096cb0 tx=0x7f81b0099c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac77cc54a8330c06 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81a57fa700 1 -- 172.21.15.70:0/2344933409 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b009e390 con 0x562bf23b1be0 2023-10-18T12:34:01.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81a57fa700 1 -- 172.21.15.70:0/2344933409 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81b00a25d0 con 0x562bf23b1be0 2023-10-18T12:34:01.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.470+0000 7f81a57fa700 1 -- 172.21.15.70:0/2344933409 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b009a4a0 con 0x562bf23b1be0 2023-10-18T12:34:01.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2344933409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 msgr2=0x562bf23b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b1fc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f81b0096cb0 tx=0x7f81b0099c90 comp rx=0 tx=0).stop 2023-10-18T12:34:01.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2344933409 shutdown_connections 2023-10-18T12:34:01.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b1fc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2344933409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b2500 0x562bf23ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2344933409 >> 172.21.15.70:0/2344933409 conn(0x562bf2320720 msgr2=0x562bf23a6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2344933409 shutdown_connections 2023-10-18T12:34:01.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.471+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2344933409 wait complete. 2023-10-18T12:34:01.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.472+0000 7f81d8d5eb80 1 -- start start 2023-10-18T12:34:01.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.472+0000 7f81d8d5eb80 1 Processor -- start 2023-10-18T12:34:01.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b2500 0x562bf23b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf22fee70 con 0x562bf23b2500 2023-10-18T12:34:01.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf229b710 con 0x562bf23ac5d0 2023-10-18T12:34:01.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf229b3b0 con 0x562bf23b1be0 2023-10-18T12:34:01.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81d8d5eb80 10 client.0 _ll_register_callbacks cb 0x562bf21e5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:01.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59226/0 (socket says 172.21.15.70:59226) 2023-10-18T12:34:01.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 -- 172.21.15.70:0/3172454848 learned_addr learned my addr 172.21.15.70:0/3172454848 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:01.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81be7fc700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b2500 0x562bf23b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 -- 172.21.15.70:0/3172454848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 msgr2=0x562bf23b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 -- 172.21.15.70:0/3172454848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b2500 msgr2=0x562bf23b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b2500 0x562bf23b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81beffd700 1 -- 172.21.15.70:0/3172454848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b8003a60 con 0x562bf23ac5d0 2023-10-18T12:34:01.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.473+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b2500 0x562bf23b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:34:01.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.474+0000 7f81beffd700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23b5840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e970 tx=0x7f81b009e410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df717337a4c46be0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.474+0000 7f81a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:01.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.474+0000 7f81a2ff5700 1 -- 172.21.15.70:0/3172454848 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00b6070 con 0x562bf23ac5d0 2023-10-18T12:34:01.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.474+0000 7f81a2ff5700 1 -- 172.21.15.70:0/3172454848 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f81b00b82b0 con 0x562bf23ac5d0 2023-10-18T12:34:01.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.474+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bf2303ef0 con 0x562bf23ac5d0 2023-10-18T12:34:01.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.475+0000 7f81a2ff5700 1 -- 172.21.15.70:0/3172454848 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81b00b8470 con 0x562bf23ac5d0 2023-10-18T12:34:01.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.475+0000 7f81d8d5eb80 10 client.26669 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:01.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.475+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f81c01a5b00 con 0x562bf23ac5d0 2023-10-18T12:34:01.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.475+0000 7f81a2ff5700 1 -- 172.21.15.70:0/3172454848 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bf2303ef0 con 0x562bf23ac5d0 2023-10-18T12:34:01.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81a2ff5700 1 -- 172.21.15.70:0/3172454848 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f81b00a2140 con 0x562bf23ac5d0 2023-10-18T12:34:01.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81d8d5eb80 10 client.26669 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:01.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81d8d5eb80 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:01.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81d8d5eb80 20 client.26669 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:01.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf23ce2e0 0x562bf23d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81d8d5eb80 4 client.26669 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:01.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bf237b8b0 con 0x562bf23ce2e0 2023-10-18T12:34:01.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.476+0000 7f81be7fc700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf23ce2e0 0x562bf23d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.477+0000 7f81be7fc700 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf23ce2e0 0x562bf23d0700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea92e7bcfb107662 server_cookie=5738a17a3101bcfa in_seq=0 out_seq=0 2023-10-18T12:34:01.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.477+0000 7f81a2ff5700 10 client.26669 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:01.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.479+0000 7f81a2ff5700 1 -- 172.21.15.70:0/3172454848 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562bf237b8b0 con 0x562bf23ce2e0 2023-10-18T12:34:01.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.479+0000 7f81a2ff5700 10 client.26669 handle_command_reply: tid=0 2023-10-18T12:34:01.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.479+0000 7f81d8d5eb80 1 client.26669 shutdown 2023-10-18T12:34:01.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 msgr2=0x562bf23b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23b5840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e970 tx=0x7f81b009e410 comp rx=0 tx=0).stop 2023-10-18T12:34:01.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 shutdown_connections 2023-10-18T12:34:01.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf23ce2e0 0x562bf23d0700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b2500 0x562bf23b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23ac5d0 0x562bf23b5840 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/3172454848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81a2ff5700 0 client.26669 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:01.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 >> 172.21.15.70:0/3172454848 conn(0x562bf2320720 msgr2=0x562bf23ab080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 shutdown_connections 2023-10-18T12:34:01.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/3172454848 wait complete. 2023-10-18T12:34:01.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.480+0000 7f81d8d5eb80 20 client.26669 trim_cache size 0 max 16384 2023-10-18T12:34:01.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 Processor -- start 2023-10-18T12:34:01.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 -- start start 2023-10-18T12:34:01.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 0x562bf23bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf23a7130 con 0x562bf23b1be0 2023-10-18T12:34:01.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf217f920 con 0x562bf23b2500 2023-10-18T12:34:01.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf23ad030 con 0x562bf23ac5d0 2023-10-18T12:34:01.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81beffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 0x562bf23bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81beffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 0x562bf23bceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47750/0 (socket says 172.21.15.70:47750) 2023-10-18T12:34:01.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81beffd700 1 -- 172.21.15.70:0/2230383466 learned_addr learned my addr 172.21.15.70:0/2230383466 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:01.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.499+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81be7fc700 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81beffd700 1 -- 172.21.15.70:0/2230383466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 msgr2=0x562bf23b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81beffd700 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81beffd700 1 -- 172.21.15.70:0/2230383466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 msgr2=0x562bf23b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81beffd700 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81beffd700 1 -- 172.21.15.70:0/2230383466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bf2129de0 con 0x562bf23ac5d0 2023-10-18T12:34:01.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81beffd700 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 0x562bf23bceb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f81b0000c00 tx=0x7f81b0011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1ec217e83724ee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81a5ffb700 1 -- 172.21.15.70:0/2230383466 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00b6070 con 0x562bf23ac5d0 2023-10-18T12:34:01.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81a5ffb700 1 -- 172.21.15.70:0/2230383466 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81b00991f0 con 0x562bf23ac5d0 2023-10-18T12:34:01.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.500+0000 7f81a5ffb700 1 -- 172.21.15.70:0/2230383466 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00a2360 con 0x562bf23ac5d0 2023-10-18T12:34:01.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2230383466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 msgr2=0x562bf23bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 0x562bf23bceb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f81b0000c00 tx=0x7f81b0011f90 comp rx=0 tx=0).stop 2023-10-18T12:34:01.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2230383466 shutdown_connections 2023-10-18T12:34:01.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23ac5d0 0x562bf23bceb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/2230383466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23b1be0 0x562bf23b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.501+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2230383466 >> 172.21.15.70:0/2230383466 conn(0x562bf2320720 msgr2=0x562bf23b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.502+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2230383466 shutdown_connections 2023-10-18T12:34:01.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.503+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/2230383466 wait complete. 2023-10-18T12:34:01.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.504+0000 7f81d8d5eb80 1 -- start start 2023-10-18T12:34:01.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.504+0000 7f81d8d5eb80 1 Processor -- start 2023-10-18T12:34:01.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23ac5d0 0x562bf23bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf23bb520 con 0x562bf23ac5d0 2023-10-18T12:34:01.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf23bb690 con 0x562bf23b2500 2023-10-18T12:34:01.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf23bb850 con 0x562bf23b1be0 2023-10-18T12:34:01.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.506+0000 7f81d8d5eb80 10 client.0 _ll_register_callbacks cb 0x562bf21e5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:01.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.507+0000 7f81be7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.507+0000 7f81be7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23bcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47758/0 (socket says 172.21.15.70:47758) 2023-10-18T12:34:01.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.507+0000 7f81be7fc700 1 -- 172.21.15.70:0/4009202569 learned_addr learned my addr 172.21.15.70:0/4009202569 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:01.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.509+0000 7f81bf7fe700 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.509+0000 7f81be7fc700 1 -- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 msgr2=0x562bf23baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:01.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.509+0000 7f81be7fc700 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.509+0000 7f81be7fc700 1 -- 172.21.15.70:0/4009202569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23ac5d0 msgr2=0x562bf23bc890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T12:34:01.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.509+0000 7f81be7fc700 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23ac5d0 0x562bf23bc890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.509+0000 7f81be7fc700 1 -- 172.21.15.70:0/4009202569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c01a5b00 con 0x562bf23b1be0 2023-10-18T12:34:01.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.512+0000 7f81be7fc700 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23bcdd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009990 tx=0x7f81b4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f73a996a76e3b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.512+0000 7f81a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:34:01.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.513+0000 7f81a2ff5700 1 -- 172.21.15.70:0/4009202569 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40363a0 con 0x562bf23b1be0 2023-10-18T12:34:01.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.513+0000 7f81a2ff5700 1 -- 172.21.15.70:0/4009202569 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f81b402c040 con 0x562bf23b1be0 2023-10-18T12:34:01.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.513+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81b00a25d0 con 0x562bf23b1be0 2023-10-18T12:34:01.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.513+0000 7f81a2ff5700 1 -- 172.21.15.70:0/4009202569 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81b4036500 con 0x562bf23b1be0 2023-10-18T12:34:01.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.514+0000 7f81d8d5eb80 10 client.26671 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:01.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.514+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562bf2112c60 con 0x562bf23b1be0 2023-10-18T12:34:01.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.514+0000 7f81a2ff5700 1 -- 172.21.15.70:0/4009202569 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81b00a25d0 con 0x562bf23b1be0 2023-10-18T12:34:01.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81a2ff5700 1 -- 172.21.15.70:0/4009202569 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f81b400f740 con 0x562bf23b1be0 2023-10-18T12:34:01.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81d8d5eb80 10 client.26671 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:01.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81d8d5eb80 10 client.26671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:01.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81d8d5eb80 20 client.26671 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:01.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf2423460 0x562bf2425880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81d8d5eb80 4 client.26671 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:34:01.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.515+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562bf22f7e50 con 0x562bf2423460 2023-10-18T12:34:01.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.516+0000 7f81beffd700 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf2423460 0x562bf2425880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.518+0000 7f81a2ff5700 10 client.26671 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:01.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.518+0000 7f81beffd700 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf2423460 0x562bf2425880 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87891942559069a5 server_cookie=1d67aa640c290e5e in_seq=0 out_seq=0 2023-10-18T12:34:01.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.527+0000 7f81a2ff5700 1 -- 172.21.15.70:0/4009202569 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562bf22f7e50 con 0x562bf2423460 2023-10-18T12:34:01.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.528+0000 7f81a2ff5700 10 client.26671 handle_command_reply: tid=0 2023-10-18T12:34:01.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.528+0000 7f81d8d5eb80 1 client.26671 shutdown 2023-10-18T12:34:01.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.531+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 msgr2=0x562bf23bcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.531+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23bcdd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009990 tx=0x7f81b4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 shutdown_connections 2023-10-18T12:34:01.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bf2423460 0x562bf2425880 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bf23b2500 0x562bf23baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bf23ac5d0 0x562bf23bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 --2- 172.21.15.70:0/4009202569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bf23b1be0 0x562bf23bcdd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 >> 172.21.15.70:0/4009202569 conn(0x562bf2320720 msgr2=0x562bf23a04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81a2ff5700 0 client.26671 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:01.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.532+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 shutdown_connections 2023-10-18T12:34:01.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.533+0000 7f81d8d5eb80 1 -- 172.21.15.70:0/4009202569 wait complete. 2023-10-18T12:34:01.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.533+0000 7f81d8d5eb80 20 client.26671 trim_cache size 0 max 16384 2023-10-18T12:34:01.540 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:34:01.540 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:34:01.540 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:34:01.540 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:34:01.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f81a80a15f0 msgr2=0x7f81a80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f81a80a15f0 0x7f81a80a3ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81c019a380 tx=0x7f81b400bb60 comp rx=0 tx=0).stop 2023-10-18T12:34:01.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 msgr2=0x7f81c0198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 0x7f81c0198e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e9a0 tx=0x7f81b000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 shutdown_connections 2023-10-18T12:34:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c0106720 0x7f81c019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f81a80a15f0 0x7f81a80a3ab0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c0100710 0x7f81c0198e80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 --2- 172.21.15.70:0/1438725180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c01010d0 0x7f81c01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.534+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 >> 172.21.15.70:0/1438725180 conn(0x7f81c00fc200 msgr2=0x7f81c00fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.535+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 shutdown_connections 2023-10-18T12:34:01.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.535+0000 7f81c53ff700 1 -- 172.21.15.70:0/1438725180 wait complete. 2023-10-18T12:34:01.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:34:01.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:34:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.808+0000 7f1e037ff700 1 -- 172.21.15.70:0/2051860661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc101060 msgr2=0x7f1dfc10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.808+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2051860661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc101060 0x7f1dfc10d570 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1df400b0a0 tx=0x7f1df400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 -- 172.21.15.70:0/2051860661 shutdown_connections 2023-10-18T12:34:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2051860661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc101060 0x7f1dfc10d570 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2051860661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1066b0 0x7f1dfc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2051860661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1006a0 0x7f1dfc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 -- 172.21.15.70:0/2051860661 >> 172.21.15.70:0/2051860661 conn(0x7f1dfc0fc170 msgr2=0x7f1dfc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:01.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 -- 172.21.15.70:0/2051860661 shutdown_connections 2023-10-18T12:34:01.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.809+0000 7f1e037ff700 1 -- 172.21.15.70:0/2051860661 wait complete. 2023-10-18T12:34:01.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 Processor -- start 2023-10-18T12:34:01.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 -- start start 2023-10-18T12:34:01.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1006a0 0x7f1dfc19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc101060 0x7f1dfc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc1066b0 0x7f1dfc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc199270 con 0x7f1dfc101060 2023-10-18T12:34:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1993e0 con 0x7f1dfc1066b0 2023-10-18T12:34:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.810+0000 7f1e037ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1995a0 con 0x7f1dfc1006a0 2023-10-18T12:34:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e0159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1006a0 0x7f1dfc19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e0159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1006a0 0x7f1dfc19ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47782/0 (socket says 172.21.15.70:47782) 2023-10-18T12:34:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc101060 0x7f1dfc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e0159b700 1 -- 172.21.15.70:0/2884088172 learned_addr learned my addr 172.21.15.70:0/2884088172 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:01.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e01d9c700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc1066b0 0x7f1dfc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 -- 172.21.15.70:0/2884088172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1006a0 msgr2=0x7f1dfc19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1006a0 0x7f1dfc19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 -- 172.21.15.70:0/2884088172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc1066b0 msgr2=0x7f1dfc198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:01.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc1066b0 0x7f1dfc198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:01.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 -- 172.21.15.70:0/2884088172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009520 con 0x7f1dfc101060 2023-10-18T12:34:01.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.811+0000 7f1e00d9a700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc101060 0x7f1dfc19ef40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00e9f0 tx=0x7f1dec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.812+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00cb70 con 0x7f1dfc101060 2023-10-18T12:34:01.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.812+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dec00ee90 con 0x7f1dfc101060 2023-10-18T12:34:01.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.812+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec003cf0 con 0x7f1dfc101060 2023-10-18T12:34:01.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.812+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df4009d00 con 0x7f1dfc101060 2023-10-18T12:34:01.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.813+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc1a5980 con 0x7f1dfc101060 2023-10-18T12:34:01.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.814+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc101de0 con 0x7f1dfc101060 2023-10-18T12:34:01.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.815+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1dec00ccd0 con 0x7f1dfc101060 2023-10-18T12:34:01.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.817+0000 7f1df27fc700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1de80a1670 0x7f1de80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:01.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.817+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f1dec014070 con 0x7f1dfc101060 2023-10-18T12:34:01.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.817+0000 7f1e0159b700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1de80a1670 0x7f1de80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:01.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.818+0000 7f1e0159b700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1de80a1670 0x7f1de80a3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009ca0 tx=0x7f1df8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:01.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:01.821+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1dec096640 con 0x7f1dfc101060 2023-10-18T12:34:01.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 895088 11.76 MB/sec execute 270 sec latency 142.100 ms 2023-10-18T12:34:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.100+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dfc110620 con 0x7f1dfc101060 2023-10-18T12:34:02.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.101+0000 7f1df27fc700 1 -- 172.21.15.70:0/2884088172 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f1dec099c60 con 0x7f1dfc101060 2023-10-18T12:34:02.104 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:34:02.104 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:34:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1de80a1670 msgr2=0x7f1de80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1de80a1670 0x7f1de80a3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009ca0 tx=0x7f1df8009260 comp rx=0 tx=0).stop 2023-10-18T12:34:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc101060 msgr2=0x7f1dfc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc101060 0x7f1dfc19ef40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00e9f0 tx=0x7f1dec00ed00 comp rx=0 tx=0).stop 2023-10-18T12:34:02.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 shutdown_connections 2023-10-18T12:34:02.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1de80a1670 0x7f1de80a3b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc1066b0 0x7f1dfc198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dfc1006a0 0x7f1dfc19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 --2- 172.21.15.70:0/2884088172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc101060 0x7f1dfc19ef40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 >> 172.21.15.70:0/2884088172 conn(0x7f1dfc0fc170 msgr2=0x7f1dfc10bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 shutdown_connections 2023-10-18T12:34:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.104+0000 7f1e037ff700 1 -- 172.21.15.70:0/2884088172 wait complete. 2023-10-18T12:34:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:34:02.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:34:02.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 -- 172.21.15.70:0/3160991595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd8106720 msgr2=0x7fefd8106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 --2- 172.21.15.70:0/3160991595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd8106720 0x7fefd8106b00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009990 tx=0x7fefc8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 -- 172.21.15.70:0/3160991595 shutdown_connections 2023-10-18T12:34:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 --2- 172.21.15.70:0/3160991595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefd81010d0 0x7fefd810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 --2- 172.21.15.70:0/3160991595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd8106720 0x7fefd8106b00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 --2- 172.21.15.70:0/3160991595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 -- 172.21.15.70:0/3160991595 >> 172.21.15.70:0/3160991595 conn(0x7fefd80fc200 msgr2=0x7fefd80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 -- 172.21.15.70:0/3160991595 shutdown_connections 2023-10-18T12:34:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.334+0000 7fefdddff700 1 -- 172.21.15.70:0/3160991595 wait complete. 2023-10-18T12:34:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.335+0000 7fefdddff700 1 Processor -- start 2023-10-18T12:34:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 -- start start 2023-10-18T12:34:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd81010d0 0x7fefd81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefd8106720 0x7fefd819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd819dcb0 con 0x7fefd81010d0 2023-10-18T12:34:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd819de20 con 0x7fefd8100710 2023-10-18T12:34:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefdddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd819dfe0 con 0x7fefd8106720 2023-10-18T12:34:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd81010d0 0x7fefd81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.336+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59286/0 (socket says 172.21.15.70:59286) 2023-10-18T12:34:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 -- 172.21.15.70:0/658267588 learned_addr learned my addr 172.21.15.70:0/658267588 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd7fff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefd8106720 0x7fefd819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 -- 172.21.15.70:0/658267588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefd8106720 msgr2=0x7fefd819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefd8106720 0x7fefd819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 -- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd81010d0 msgr2=0x7fefd81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd81010d0 0x7fefd81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 -- 172.21.15.70:0/658267588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd0009c40 con 0x7fefd8100710 2023-10-18T12:34:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd81010d0 0x7fefd81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd77fe700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8198d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd4ff9700 1 -- 172.21.15.70:0/658267588 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc801d070 con 0x7fefd8100710 2023-10-18T12:34:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefd4ff9700 1 -- 172.21.15.70:0/658267588 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefc8022470 con 0x7fefd8100710 2023-10-18T12:34:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.337+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc80095f0 con 0x7fefd8100710 2023-10-18T12:34:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.338+0000 7fefd4ff9700 1 -- 172.21.15.70:0/658267588 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017650 con 0x7fefd8100710 2023-10-18T12:34:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.338+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd81a5920 con 0x7fefd8100710 2023-10-18T12:34:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.341+0000 7fefd4ff9700 1 -- 172.21.15.70:0/658267588 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fefc80225e0 con 0x7fefd8100710 2023-10-18T12:34:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.342+0000 7fefd4ff9700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fefc00a15f0 0x7fefc00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.342+0000 7fefd4ff9700 1 -- 172.21.15.70:0/658267588 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fefc80ce720 con 0x7fefd8100710 2023-10-18T12:34:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.342+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fefc00a15f0 0x7fefc00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 Processor -- start 2023-10-18T12:34:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 -- start start 2023-10-18T12:34:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fefc00a15f0 0x7fefc00a3ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fefd819a2e0 tx=0x7fefcc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0c500 0x55a6c2a06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a06a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c29796e0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c295fc60 con 0x55a6c2a065d0 2023-10-18T12:34:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7feff164cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c295d580 con 0x55a6c2a0c500 2023-10-18T12:34:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33968/0 (socket says 172.21.15.70:33968) 2023-10-18T12:34:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7fefd77fe700 1 -- 172.21.15.70:0/536115663 learned_addr learned my addr 172.21.15.70:0/536115663 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7fefd7fff700 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a06a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.343+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0c500 0x55a6c2a06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefd77fe700 1 -- 172.21.15.70:0/536115663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0c500 msgr2=0x55a6c2a06090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefd77fe700 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0c500 0x55a6c2a06090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefd77fe700 1 -- 172.21.15.70:0/536115663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 msgr2=0x55a6c2a06a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefd77fe700 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a06a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefd77fe700 1 -- 172.21.15.70:0/536115663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6c29163f0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefd77fe700 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0bfc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fefc8097e40 tx=0x7fefc8097710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f46cfd6c6d91f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefbdffb700 1 -- 172.21.15.70:0/536115663 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc809ad30 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefbdffb700 1 -- 172.21.15.70:0/536115663 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefc809bad0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.344+0000 7fefbdffb700 1 -- 172.21.15.70:0/536115663 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc809f320 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 -- 172.21.15.70:0/536115663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 msgr2=0x55a6c2a0bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0bfc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fefc8097e40 tx=0x7fefc8097710 comp rx=0 tx=0).stop 2023-10-18T12:34:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 -- 172.21.15.70:0/536115663 shutdown_connections 2023-10-18T12:34:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a06a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0bfc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 --2- 172.21.15.70:0/536115663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0c500 0x55a6c2a06090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 -- 172.21.15.70:0/536115663 >> 172.21.15.70:0/536115663 conn(0x55a6c297a720 msgr2=0x55a6c2a00b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 -- 172.21.15.70:0/536115663 shutdown_connections 2023-10-18T12:34:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.345+0000 7feff164cb80 1 -- 172.21.15.70:0/536115663 wait complete. 2023-10-18T12:34:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.346+0000 7feff164cb80 1 -- start start 2023-10-18T12:34:02.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.346+0000 7feff164cb80 1 Processor -- start 2023-10-18T12:34:02.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a065d0 0x55a6c2a12380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c2958e70 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c28f5710 con 0x55a6c2a0c500 2023-10-18T12:34:02.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c28f53b0 con 0x55a6c2a065d0 2023-10-18T12:34:02.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7feff164cb80 10 client.0 _ll_register_callbacks cb 0x55a6c283fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:02.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a065d0 0x55a6c2a12380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33974/0 (socket says 172.21.15.70:33974) 2023-10-18T12:34:02.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd6ffd700 1 -- 172.21.15.70:0/1067067979 learned_addr learned my addr 172.21.15.70:0/1067067979 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:02.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd7fff700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd77fe700 1 -- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a0c500 msgr2=0x55a6c2a0fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd77fe700 1 -- 172.21.15.70:0/1067067979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 msgr2=0x55a6c2a0f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd77fe700 1 -- 172.21.15.70:0/1067067979 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd81a5920 con 0x55a6c2a065d0 2023-10-18T12:34:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.347+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a065d0 0x55a6c2a12380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fefc809b0d0 tx=0x7fefc8022ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26881742473e29e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:34:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1067067979 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80ce510 con 0x55a6c2a065d0 2023-10-18T12:34:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6c295def0 con 0x55a6c2a065d0 2023-10-18T12:34:02.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1067067979 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fefc809b3d0 con 0x55a6c2a065d0 2023-10-18T12:34:02.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1067067979 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefc8099770 con 0x55a6c2a065d0 2023-10-18T12:34:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 10 client.26689 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a6c2783de0 con 0x55a6c2a065d0 2023-10-18T12:34:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1067067979 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6c295def0 con 0x55a6c2a065d0 2023-10-18T12:34:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1067067979 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fefc80a3240 con 0x55a6c2a065d0 2023-10-18T12:34:02.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 10 client.26689 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 10 client.26689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 20 client.26689 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a1f640 0x55a6c2a21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 4 client.26689 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:02.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.348+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6c29d58b0 con 0x55a6c2a1f640 2023-10-18T12:34:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.349+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a1f640 0x55a6c2a21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.349+0000 7fefb6ffd700 10 client.26689 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.349+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a1f640 0x55a6c2a21a60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6613a7ab90b44f2 server_cookie=d7c63d4cd0af8d14 in_seq=0 out_seq=0 2023-10-18T12:34:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.350+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1067067979 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a6c29d58b0 con 0x55a6c2a1f640 2023-10-18T12:34:02.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.350+0000 7fefb6ffd700 10 client.26689 handle_command_reply: tid=0 2023-10-18T12:34:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.350+0000 7feff164cb80 1 client.26689 shutdown 2023-10-18T12:34:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.350+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a065d0 msgr2=0x55a6c2a12380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.350+0000 7feff164cb80 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a065d0 0x55a6c2a12380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fefc809b0d0 tx=0x7fefc8022ad0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 shutdown_connections 2023-10-18T12:34:02.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a1f640 0x55a6c2a21a60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a065d0 0x55a6c2a12380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 --2- 172.21.15.70:0/1067067979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a0f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7fefb6ffd700 0 client.26689 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:02.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 >> 172.21.15.70:0/1067067979 conn(0x55a6c297a720 msgr2=0x55a6c2a05080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 shutdown_connections 2023-10-18T12:34:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 1 -- 172.21.15.70:0/1067067979 wait complete. 2023-10-18T12:34:02.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.351+0000 7feff164cb80 20 client.26689 trim_cache size 0 max 16384 2023-10-18T12:34:02.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 Processor -- start 2023-10-18T12:34:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 -- start start 2023-10-18T12:34:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a16c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a05890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c29fbaf0 con 0x55a6c2a0c500 2023-10-18T12:34:02.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c27d9920 con 0x55a6c2a065d0 2023-10-18T12:34:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7feff164cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c2a12990 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a05890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a05890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47832/0 (socket says 172.21.15.70:47832) 2023-10-18T12:34:02.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a16c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.367+0000 7fefd6ffd700 1 -- 172.21.15.70:0/2691717769 learned_addr learned my addr 172.21.15.70:0/2691717769 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefd6ffd700 1 -- 172.21.15.70:0/2691717769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 msgr2=0x55a6c2a16c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a16c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefd6ffd700 1 -- 172.21.15.70:0/2691717769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 msgr2=0x55a6c2a0df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefd6ffd700 1 -- 172.21.15.70:0/2691717769 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6c2a26650 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a05890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fefcc011bb0 tx=0x7fefcc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b773c72746a473 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefbe7fc700 1 -- 172.21.15.70:0/2691717769 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc024040 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefbe7fc700 1 -- 172.21.15.70:0/2691717769 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefcc029dc0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.368+0000 7fefbe7fc700 1 -- 172.21.15.70:0/2691717769 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc01e870 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 -- 172.21.15.70:0/2691717769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 msgr2=0x55a6c2a05890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a05890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fefcc011bb0 tx=0x7fefcc014000 comp rx=0 tx=0).stop 2023-10-18T12:34:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 -- 172.21.15.70:0/2691717769 shutdown_connections 2023-10-18T12:34:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a0df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a16c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 --2- 172.21.15.70:0/2691717769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a05890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 -- 172.21.15.70:0/2691717769 >> 172.21.15.70:0/2691717769 conn(0x55a6c297a720 msgr2=0x55a6c2a069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.369+0000 7feff164cb80 1 -- 172.21.15.70:0/2691717769 shutdown_connections 2023-10-18T12:34:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.370+0000 7feff164cb80 1 -- 172.21.15.70:0/2691717769 wait complete. 2023-10-18T12:34:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.370+0000 7feff164cb80 1 -- start start 2023-10-18T12:34:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.370+0000 7feff164cb80 1 Processor -- start 2023-10-18T12:34:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a0ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a15c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c2a166e0 con 0x55a6c2a0c500 2023-10-18T12:34:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c2a16850 con 0x55a6c2a065d0 2023-10-18T12:34:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6c2a16a10 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7feff164cb80 10 client.0 _ll_register_callbacks cb 0x55a6c283fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a15c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.371+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a15c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47848/0 (socket says 172.21.15.70:47848) 2023-10-18T12:34:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33996/0 (socket says 172.21.15.70:33996) 2023-10-18T12:34:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 -- 172.21.15.70:0/1326807738 learned_addr learned my addr 172.21.15.70:0/1326807738 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a0ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 -- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 msgr2=0x55a6c2a0ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a0ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 -- 172.21.15.70:0/1326807738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 msgr2=0x55a6c2a161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 -- 172.21.15.70:0/1326807738 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc017070 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a0ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefd6ffd700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a15c60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0057f0 tx=0x7fefcc003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd58a1e1967840be server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.372+0000 7fefb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:34:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1326807738 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc024040 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefc809bad0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1326807738 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fefcc029b00 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1326807738 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefcc0295c0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7feff164cb80 10 client.26695 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a6c276cc60 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1326807738 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefc809bad0 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.373+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1326807738 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fefcc00e440 con 0x55a6c2a0bbe0 2023-10-18T12:34:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7feff164cb80 10 client.26695 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7feff164cb80 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7feff164cb80 20 client.26695 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7feff164cb80 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a74800 0x55a6c2a76c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7feff164cb80 4 client.26695 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:34:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a6c2951e50 con 0x55a6c2a74800 2023-10-18T12:34:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a74800 0x55a6c2a76c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7fefd77fe700 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a74800 0x55a6c2a76c20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a907ba43d3c1ed server_cookie=10e5614bc31856f4 in_seq=0 out_seq=0 2023-10-18T12:34:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.374+0000 7fefb6ffd700 10 client.26695 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.375+0000 7fefb6ffd700 1 -- 172.21.15.70:0/1326807738 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6c2951e50 con 0x55a6c2a74800 2023-10-18T12:34:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.375+0000 7fefb6ffd700 10 client.26695 handle_command_reply: tid=0 2023-10-18T12:34:02.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.375+0000 7feff164cb80 1 client.26695 shutdown 2023-10-18T12:34:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.375+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 msgr2=0x55a6c2a15c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.375+0000 7feff164cb80 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a15c60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0057f0 tx=0x7fefcc003ee0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 shutdown_connections 2023-10-18T12:34:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a6c2a74800 0x55a6c2a76c20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a6c2a0c500 0x55a6c2a161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a6c2a065d0 0x55a6c2a0ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7fefb6ffd700 0 client.26695 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 --2- 172.21.15.70:0/1326807738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6c2a0bbe0 0x55a6c2a15c60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 >> 172.21.15.70:0/1326807738 conn(0x55a6c297a720 msgr2=0x55a6c2a05830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 shutdown_connections 2023-10-18T12:34:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 1 -- 172.21.15.70:0/1326807738 wait complete. 2023-10-18T12:34:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.376+0000 7feff164cb80 20 client.26695 trim_cache size 0 max 16384 2023-10-18T12:34:02.400 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:34:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fefc00a15f0 msgr2=0x7fefc00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fefc00a15f0 0x7fefc00a3ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fefd819a2e0 tx=0x7fefcc009260 comp rx=0 tx=0).stop 2023-10-18T12:34:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 msgr2=0x7fefd8198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8198d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8005740 comp rx=0 tx=0).stop 2023-10-18T12:34:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 shutdown_connections 2023-10-18T12:34:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefd8106720 0x7fefd819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fefc00a15f0 0x7fefc00a3ab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fefd8100710 0x7fefd8198d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 --2- 172.21.15.70:0/658267588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefd81010d0 0x7fefd81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 >> 172.21.15.70:0/658267588 conn(0x7fefd80fc200 msgr2=0x7fefd810be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 shutdown_connections 2023-10-18T12:34:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:02.377+0000 7fefdddff700 1 -- 172.21.15.70:0/658267588 wait complete. 2023-10-18T12:34:02.860 INFO:tasks.workunit.client.0.smithi070.stdout: 1 897253 11.75 MB/sec execute 271 sec latency 15.077 ms 2023-10-18T12:34:03.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:02 smithi070 ceph-mon[98639]: pgmap v557: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 237 op/s 2023-10-18T12:34:03.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:02 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2884088172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:34:03.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:02 smithi152 ceph-mon[134767]: pgmap v557: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 237 op/s 2023-10-18T12:34:03.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:02 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2884088172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:34:03.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:02 smithi186 ceph-mon[135253]: pgmap v557: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 237 op/s 2023-10-18T12:34:03.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:02 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2884088172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:34:03.404 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:34:03.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:34:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 -- 172.21.15.70:0/2299900519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0105170 msgr2=0x7f29d01055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 --2- 172.21.15.70:0/2299900519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0105170 0x7f29d01055f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009990 tx=0x7f29c0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 -- 172.21.15.70:0/2299900519 shutdown_connections 2023-10-18T12:34:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 --2- 172.21.15.70:0/2299900519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d0105b30 0x7f29d010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 --2- 172.21.15.70:0/2299900519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0067d40 0x7f29d0068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 --2- 172.21.15.70:0/2299900519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0105170 0x7f29d01055f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 -- 172.21.15.70:0/2299900519 >> 172.21.15.70:0/2299900519 conn(0x7f29d00767f0 msgr2=0x7f29d0076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 -- 172.21.15.70:0/2299900519 shutdown_connections 2023-10-18T12:34:03.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.651+0000 7f29d7fff700 1 -- 172.21.15.70:0/2299900519 wait complete. 2023-10-18T12:34:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.652+0000 7f29d7fff700 1 Processor -- start 2023-10-18T12:34:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.652+0000 7f29d7fff700 1 -- start start 2023-10-18T12:34:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 0x7f29d019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0105170 0x7f29d019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d0105b30 0x7f29d0196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d7fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0196fd0 con 0x7f29d0105b30 2023-10-18T12:34:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0197140 con 0x7f29d0067d40 2023-10-18T12:34:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d7fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0197300 con 0x7f29d0105170 2023-10-18T12:34:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 0x7f29d019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d0105b30 0x7f29d0196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 0x7f29d019c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59352/0 (socket says 172.21.15.70:59352) 2023-10-18T12:34:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 -- 172.21.15.70:0/733235332 learned_addr learned my addr 172.21.15.70:0/733235332 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d559a700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0105170 0x7f29d019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 -- 172.21.15.70:0/733235332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0105170 msgr2=0x7f29d019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0105170 0x7f29d019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 -- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d0105b30 msgr2=0x7f29d0196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d0105b30 0x7f29d0196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.653+0000 7f29d5d9b700 1 -- 172.21.15.70:0/733235332 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00095f0 con 0x7f29d0067d40 2023-10-18T12:34:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.654+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 0x7f29d019c840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00e9a0 tx=0x7f29cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.654+0000 7f29c6ffd700 1 -- 172.21.15.70:0/733235332 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc009790 con 0x7f29d0067d40 2023-10-18T12:34:03.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.654+0000 7f29c6ffd700 1 -- 172.21.15.70:0/733235332 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc010430 con 0x7f29d0067d40 2023-10-18T12:34:03.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.654+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d01975e0 con 0x7f29d0067d40 2023-10-18T12:34:03.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.654+0000 7f29c6ffd700 1 -- 172.21.15.70:0/733235332 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc018680 con 0x7f29d0067d40 2023-10-18T12:34:03.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.654+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d01a3830 con 0x7f29d0067d40 2023-10-18T12:34:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.658+0000 7f29c6ffd700 1 -- 172.21.15.70:0/733235332 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f29cc0105a0 con 0x7f29d0067d40 2023-10-18T12:34:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.658+0000 7f29eb9d2b80 1 Processor -- start 2023-10-18T12:34:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 -- start start 2023-10-18T12:34:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c98d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c987110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c987ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c8fa760 con 0x55e36c987650 2023-10-18T12:34:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c8e0ce0 con 0x55e36c98cc60 2023-10-18T12:34:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c8de600 con 0x55e36c98d580 2023-10-18T12:34:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29c6ffd700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f29bc0a1670 0x7f29bc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29d659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c98d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29d659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c98d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59360/0 (socket says 172.21.15.70:59360) 2023-10-18T12:34:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29d659c700 1 -- 172.21.15.70:0/3400805695 learned_addr learned my addr 172.21.15.70:0/3400805695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c987110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29d559a700 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c987ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.659+0000 7f29c6ffd700 1 -- 172.21.15.70:0/733235332 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f29cc014070 con 0x7f29d0067d40 2023-10-18T12:34:03.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d659c700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f29bc0a1670 0x7f29bc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d5d9b700 1 -- 172.21.15.70:0/3400805695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 msgr2=0x55e36c98d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c98d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d5d9b700 1 -- 172.21.15.70:0/3400805695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 msgr2=0x55e36c987ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c987ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d5d9b700 1 -- 172.21.15.70:0/3400805695 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e36c897470 con 0x55e36c98d580 2023-10-18T12:34:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d559a700 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c987ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c987110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0ca9b0 tx=0x7f29cc06d1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778eb840178bc561 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29bbfff700 1 -- 172.21.15.70:0/3400805695 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc09a940 con 0x55e36c98d580 2023-10-18T12:34:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29d659c700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f29bc0a1670 0x7f29bc0a3b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009fd0 tx=0x7f29c8004b40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29bbfff700 1 -- 172.21.15.70:0/3400805695 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc09a240 con 0x55e36c98d580 2023-10-18T12:34:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.660+0000 7f29bbfff700 1 -- 172.21.15.70:0/3400805695 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01d470 con 0x55e36c98d580 2023-10-18T12:34:03.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/3400805695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 msgr2=0x55e36c987110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c987110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0ca9b0 tx=0x7f29cc06d1b0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/3400805695 shutdown_connections 2023-10-18T12:34:03.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c987ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c98d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/3400805695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c987110 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/3400805695 >> 172.21.15.70:0/3400805695 conn(0x55e36c8fb7a0 msgr2=0x55e36c981be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/3400805695 shutdown_connections 2023-10-18T12:34:03.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.661+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/3400805695 wait complete. 2023-10-18T12:34:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 -- start start 2023-10-18T12:34:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 Processor -- start 2023-10-18T12:34:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 0x55e36c9935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c990910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c991050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c8d9ef0 con 0x55e36c98cc60 2023-10-18T12:34:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c876790 con 0x55e36c987650 2023-10-18T12:34:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c876430 con 0x55e36c98d580 2023-10-18T12:34:03.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.662+0000 7f29eb9d2b80 10 client.0 _ll_register_callbacks cb 0x55e36c7c0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:03.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c990910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 0x55e36c9935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c990910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34030/0 (socket says 172.21.15.70:34030) 2023-10-18T12:34:03.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 0x55e36c9935b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59376/0 (socket says 172.21.15.70:59376) 2023-10-18T12:34:03.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 -- 172.21.15.70:0/2663486467 learned_addr learned my addr 172.21.15.70:0/2663486467 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:03.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d659c700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c991050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 -- 172.21.15.70:0/2663486467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 msgr2=0x55e36c991050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c991050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 -- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 msgr2=0x55e36c9935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 0x55e36c9935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 -- 172.21.15.70:0/2663486467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d010de70 con 0x55e36c98cc60 2023-10-18T12:34:03.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 0x55e36c9935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:03.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29d559a700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c990910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009990 tx=0x7f29c0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c8ed8c73a53fbd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:03.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.663+0000 7f29b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:34:03.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29b97fa700 1 -- 172.21.15.70:0/2663486467 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c001d070 con 0x55e36c98cc60 2023-10-18T12:34:03.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29b97fa700 1 -- 172.21.15.70:0/2663486467 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f29c0004500 con 0x55e36c98cc60 2023-10-18T12:34:03.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e36c8def70 con 0x55e36c98cc60 2023-10-18T12:34:03.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29b97fa700 1 -- 172.21.15.70:0/2663486467 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c00046c0 con 0x55e36c98cc60 2023-10-18T12:34:03.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 10 client.16671 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:03.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f29d01a3830 con 0x55e36c98cc60 2023-10-18T12:34:03.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29b97fa700 1 -- 172.21.15.70:0/2663486467 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e36c8def70 con 0x55e36c98cc60 2023-10-18T12:34:03.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29b97fa700 1 -- 172.21.15.70:0/2663486467 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29c000f760 con 0x55e36c98cc60 2023-10-18T12:34:03.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:03.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:03.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 20 client.16671 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:03.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9967c0 0x55e36c996bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 4 client.16671 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:03.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.664+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e36c956930 con 0x55e36c9967c0 2023-10-18T12:34:03.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.665+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9967c0 0x55e36c996bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.665+0000 7f29b97fa700 10 client.16671 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:03.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.665+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9967c0 0x55e36c996bd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22534466884a856 server_cookie=3a42a792730526e8 in_seq=0 out_seq=0 2023-10-18T12:34:03.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29b97fa700 1 -- 172.21.15.70:0/2663486467 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e36c956930 con 0x55e36c9967c0 2023-10-18T12:34:03.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29b97fa700 10 client.16671 handle_command_reply: tid=0 2023-10-18T12:34:03.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 client.16671 shutdown 2023-10-18T12:34:03.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 msgr2=0x55e36c990910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c990910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009990 tx=0x7f29c0005800 comp rx=0 tx=0).stop 2023-10-18T12:34:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 shutdown_connections 2023-10-18T12:34:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9967c0 0x55e36c996bd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c991050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c987650 0x55e36c9935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/2663486467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c990910 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29b97fa700 0 client.16671 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:03.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 >> 172.21.15.70:0/2663486467 conn(0x55e36c8fb7a0 msgr2=0x55e36c986100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 shutdown_connections 2023-10-18T12:34:03.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/2663486467 wait complete. 2023-10-18T12:34:03.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.666+0000 7f29eb9d2b80 20 client.16671 trim_cache size 0 max 16384 2023-10-18T12:34:03.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.686+0000 7f29eb9d2b80 1 Processor -- start 2023-10-18T12:34:03.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 -- start start 2023-10-18T12:34:03.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c987650 0x55e36c98c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c9a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 0x55e36c97b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c75a9a0 con 0x55e36c98cc60 2023-10-18T12:34:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c98c650 con 0x55e36c98d580 2023-10-18T12:34:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c991a20 con 0x55e36c987650 2023-10-18T12:34:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29d659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 0x55e36c97b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c987650 0x55e36c98c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29d659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 0x55e36c97b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59386/0 (socket says 172.21.15.70:59386) 2023-10-18T12:34:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29d659c700 1 -- 172.21.15.70:0/1702588968 learned_addr learned my addr 172.21.15.70:0/1702588968 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.687+0000 7f29d559a700 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c9a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d5d9b700 1 -- 172.21.15.70:0/1702588968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 msgr2=0x55e36c97b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 0x55e36c97b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d5d9b700 1 -- 172.21.15.70:0/1702588968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 msgr2=0x55e36c9a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c9a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d5d9b700 1 -- 172.21.15.70:0/1702588968 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e36c704e60 con 0x55e36c987650 2023-10-18T12:34:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d659c700 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 0x55e36c97b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c987650 0x55e36c98c110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55e36c993bf0 tx=0x7f29cc0c48d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dcfb1c286009c03 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:03.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29c4ff9700 1 -- 172.21.15.70:0/1702588968 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc069070 con 0x55e36c987650 2023-10-18T12:34:03.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29c4ff9700 1 -- 172.21.15.70:0/1702588968 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc09e410 con 0x55e36c987650 2023-10-18T12:34:03.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.688+0000 7f29c4ff9700 1 -- 172.21.15.70:0/1702588968 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc099730 con 0x55e36c987650 2023-10-18T12:34:03.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/1702588968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c987650 msgr2=0x55e36c98c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c987650 0x55e36c98c110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55e36c993bf0 tx=0x7f29cc0c48d0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/1702588968 shutdown_connections 2023-10-18T12:34:03.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98d580 0x55e36c97b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c987650 0x55e36c98c110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/1702588968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c98cc60 0x55e36c9a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/1702588968 >> 172.21.15.70:0/1702588968 conn(0x55e36c8fb7a0 msgr2=0x55e36c98ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/1702588968 shutdown_connections 2023-10-18T12:34:03.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.689+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/1702588968 wait complete. 2023-10-18T12:34:03.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.690+0000 7f29eb9d2b80 1 -- start start 2023-10-18T12:34:03.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.690+0000 7f29eb9d2b80 1 Processor -- start 2023-10-18T12:34:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c9edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c9ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c9e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c9ea0f0 con 0x55e36c987650 2023-10-18T12:34:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c9ea260 con 0x55e36c98cc60 2023-10-18T12:34:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e36c9ea420 con 0x55e36c98d580 2023-10-18T12:34:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29eb9d2b80 10 client.0 _ll_register_callbacks cb 0x55e36c7c0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c9edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d559a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c9ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c9edf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34052/0 (socket says 172.21.15.70:34052) 2023-10-18T12:34:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 -- 172.21.15.70:0/4194728155 learned_addr learned my addr 172.21.15.70:0/4194728155 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d559a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c9ee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59388/0 (socket says 172.21.15.70:59388) 2023-10-18T12:34:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d659c700 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c9e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 -- 172.21.15.70:0/4194728155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 msgr2=0x55e36c9e9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c9e9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 -- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 msgr2=0x55e36c9ee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c9ee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.691+0000 7f29d5d9b700 1 -- 172.21.15.70:0/4194728155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c80090e0 con 0x55e36c987650 2023-10-18T12:34:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29d5d9b700 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c9edf50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0a20a0 tx=0x7f29cc09e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e403ea6dea2546d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:34:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29b97fa700 1 -- 172.21.15.70:0/4194728155 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc069030 con 0x55e36c987650 2023-10-18T12:34:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29b97fa700 1 -- 172.21.15.70:0/4194728155 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f29cc06a070 con 0x55e36c987650 2023-10-18T12:34:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e36c9880c0 con 0x55e36c987650 2023-10-18T12:34:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29b97fa700 1 -- 172.21.15.70:0/4194728155 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29cc099730 con 0x55e36c987650 2023-10-18T12:34:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29eb9d2b80 10 client.16677 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.692+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e36c897470 con 0x55e36c987650 2023-10-18T12:34:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29b97fa700 1 -- 172.21.15.70:0/4194728155 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc0c8020 con 0x55e36c987650 2023-10-18T12:34:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29b97fa700 1 -- 172.21.15.70:0/4194728155 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29cc0991a0 con 0x55e36c987650 2023-10-18T12:34:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29eb9d2b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29eb9d2b80 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29eb9d2b80 20 client.16677 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9f6ae0 0x55e36c9f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29eb9d2b80 4 client.16677 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:34:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f29cc09a240 con 0x55e36c9f6ae0 2023-10-18T12:34:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.693+0000 7f29d559a700 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9f6ae0 0x55e36c9f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.694+0000 7f29d559a700 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9f6ae0 0x55e36c9f8ed0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d548a57f53b3809f server_cookie=318d3dad19641122 in_seq=0 out_seq=0 2023-10-18T12:34:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.694+0000 7f29b97fa700 10 client.16677 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29b97fa700 1 -- 172.21.15.70:0/4194728155 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f29cc09a240 con 0x55e36c9f6ae0 2023-10-18T12:34:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29b97fa700 10 client.16677 handle_command_reply: tid=0 2023-10-18T12:34:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 client.16677 shutdown 2023-10-18T12:34:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 msgr2=0x55e36c9edf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c9edf50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0a20a0 tx=0x7f29cc09e1e0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 shutdown_connections 2023-10-18T12:34:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e36c9f6ae0 0x55e36c9f8ed0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e36c98d580 0x55e36c9e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e36c987650 0x55e36c9edf50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 --2- 172.21.15.70:0/4194728155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e36c98cc60 0x55e36c9ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 >> 172.21.15.70:0/4194728155 conn(0x55e36c8fb7a0 msgr2=0x55e36c97b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29b97fa700 0 client.16677 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 shutdown_connections 2023-10-18T12:34:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.695+0000 7f29eb9d2b80 1 -- 172.21.15.70:0/4194728155 wait complete. 2023-10-18T12:34:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29eb9d2b80 20 client.16677 trim_cache size 0 max 16384 2023-10-18T12:34:03.737 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:34:03.737 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:34:03.737 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "b5f74590-2034-435e-b634-3b7d7c19d814", 2023-10-18T12:34:03.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:34:03.738 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:34:03.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f29bc0a1670 msgr2=0x7f29bc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f29bc0a1670 0x7f29bc0a3b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009fd0 tx=0x7f29c8004b40 comp rx=0 tx=0).stop 2023-10-18T12:34:03.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 msgr2=0x7f29d019c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 0x7f29d019c840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00e9a0 tx=0x7f29cc00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 shutdown_connections 2023-10-18T12:34:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d0105b30 0x7f29d0196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f29bc0a1670 0x7f29bc0a3b30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29d0067d40 0x7f29d019c840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.696+0000 7f29d7fff700 1 --2- 172.21.15.70:0/733235332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0105170 0x7f29d019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.697+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 >> 172.21.15.70:0/733235332 conn(0x7f29d00767f0 msgr2=0x7f29d00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.697+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 shutdown_connections 2023-10-18T12:34:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.697+0000 7f29d7fff700 1 -- 172.21.15.70:0/733235332 wait complete. 2023-10-18T12:34:03.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:34:03.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 899083 11.75 MB/sec execute 272 sec latency 33.279 ms 2023-10-18T12:34:03.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 -- 172.21.15.70:0/2738159823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c100720 msgr2=0x7fca8c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 --2- 172.21.15.70:0/2738159823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c100720 0x7fca8c100ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0098f0 tx=0x7fca7c009c00 comp rx=0 tx=0).stop 2023-10-18T12:34:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 -- 172.21.15.70:0/2738159823 shutdown_connections 2023-10-18T12:34:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 --2- 172.21.15.70:0/2738159823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c1010e0 0x7fca8c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 --2- 172.21.15.70:0/2738159823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 0x7fca8c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 --2- 172.21.15.70:0/2738159823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c100720 0x7fca8c100ba0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 -- 172.21.15.70:0/2738159823 >> 172.21.15.70:0/2738159823 conn(0x7fca8c0fc1b0 msgr2=0x7fca8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 -- 172.21.15.70:0/2738159823 shutdown_connections 2023-10-18T12:34:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.993+0000 7fca933ff700 1 -- 172.21.15.70:0/2738159823 wait complete. 2023-10-18T12:34:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.994+0000 7fca933ff700 1 Processor -- start 2023-10-18T12:34:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.994+0000 7fca933ff700 1 -- start start 2023-10-18T12:34:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca933ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 0x7fca8c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca933ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 0x7fca8c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca933ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 0x7fca8c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca933ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19dd00 con 0x7fca8c106730 2023-10-18T12:34:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca933ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19de70 con 0x7fca8c1010e0 2023-10-18T12:34:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca933ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19e030 con 0x7fca8c100720 2023-10-18T12:34:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9119b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 0x7fca8c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 0x7fca8c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 0x7fca8c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59404/0 (socket says 172.21.15.70:59404) 2023-10-18T12:34:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9119b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 0x7fca8c198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47946/0 (socket says 172.21.15.70:47946) 2023-10-18T12:34:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 -- 172.21.15.70:0/1553397195 learned_addr learned my addr 172.21.15.70:0/1553397195 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9199c700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 0x7fca8c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 -- 172.21.15.70:0/1553397195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 msgr2=0x7fca8c198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 0x7fca8c198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 -- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 msgr2=0x7fca8c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 0x7fca8c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9099a700 1 -- 172.21.15.70:0/1553397195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880095f0 con 0x7fca8c1010e0 2023-10-18T12:34:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9199c700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 0x7fca8c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:34:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.995+0000 7fca9119b700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 0x7fca8c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.996+0000 7fca9099a700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 0x7fca8c199320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca7c005950 tx=0x7fca7c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.996+0000 7fca827fc700 1 -- 172.21.15.70:0/1553397195 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c01c070 con 0x7fca8c1010e0 2023-10-18T12:34:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.996+0000 7fca827fc700 1 -- 172.21.15.70:0/1553397195 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7c00bb20 con 0x7fca8c1010e0 2023-10-18T12:34:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.996+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c009520 con 0x7fca8c1010e0 2023-10-18T12:34:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.996+0000 7fca827fc700 1 -- 172.21.15.70:0/1553397195 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0038e0 con 0x7fca8c1010e0 2023-10-18T12:34:04.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.996+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c1a5970 con 0x7fca8c1010e0 2023-10-18T12:34:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:03.999+0000 7fca827fc700 1 -- 172.21.15.70:0/1553397195 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fca7c020030 con 0x7fca8c1010e0 2023-10-18T12:34:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.000+0000 7fcaa6c74b80 1 Processor -- start 2023-10-18T12:34:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.000+0000 7fca827fc700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fca780a1670 0x7fca780a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fca827fc700 1 -- 172.21.15.70:0/1553397195 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fca7c0ce2e0 con 0x7fca8c1010e0 2023-10-18T12:34:04.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 -- start start 2023-10-18T12:34:04.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fca9199c700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fca780a1670 0x7fca780a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab00858fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab00853090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab008535d0 0x55ab00853a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab007c66e0 con 0x55ab00858be0 2023-10-18T12:34:04.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab007acc60 con 0x55ab008535d0 2023-10-18T12:34:04.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab007aa580 con 0x55ab00859500 2023-10-18T12:34:04.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fca9119b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab00858fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.001+0000 7fca9099a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab00853090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab00853090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47962/0 (socket says 172.21.15.70:47962) 2023-10-18T12:34:04.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 -- 172.21.15.70:0/774807466 learned_addr learned my addr 172.21.15.70:0/774807466 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:04.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9199c700 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab008535d0 0x55ab00853a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 -- 172.21.15.70:0/774807466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab008535d0 msgr2=0x55ab00853a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab008535d0 0x55ab00853a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 -- 172.21.15.70:0/774807466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 msgr2=0x55ab00858fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9199c700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fca780a1670 0x7fca780a3b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fca84009410 tx=0x7fca84009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:04.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab00858fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 -- 172.21.15.70:0/774807466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab007633f0 con 0x55ab00859500 2023-10-18T12:34:04.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca9099a700 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab00853090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0979b0 tx=0x7fca7c005570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d19a00726890da server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:04.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca777fe700 1 -- 172.21.15.70:0/774807466 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c09f090 con 0x55ab00859500 2023-10-18T12:34:04.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca777fe700 1 -- 172.21.15.70:0/774807466 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7c0a3240 con 0x55ab00859500 2023-10-18T12:34:04.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.002+0000 7fca777fe700 1 -- 172.21.15.70:0/774807466 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0977a0 con 0x55ab00859500 2023-10-18T12:34:04.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/774807466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 msgr2=0x55ab00853090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab00853090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0979b0 tx=0x7fca7c005570 comp rx=0 tx=0).stop 2023-10-18T12:34:04.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/774807466 shutdown_connections 2023-10-18T12:34:04.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab008535d0 0x55ab00853a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab00858fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/774807466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab00853090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/774807466 >> 172.21.15.70:0/774807466 conn(0x55ab007c7720 msgr2=0x55ab0084db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:04.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/774807466 shutdown_connections 2023-10-18T12:34:04.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.003+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/774807466 wait complete. 2023-10-18T12:34:04.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.004+0000 7fcaa6c74b80 1 -- start start 2023-10-18T12:34:04.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.004+0000 7fcaa6c74b80 1 Processor -- start 2023-10-18T12:34:04.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.004+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab008535d0 0x55ab0085f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab0085c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab0085d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab007a5e70 con 0x55ab008535d0 2023-10-18T12:34:04.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab00742710 con 0x55ab00858be0 2023-10-18T12:34:04.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab007423b0 con 0x55ab00859500 2023-10-18T12:34:04.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fcaa6c74b80 10 client.0 _ll_register_callbacks cb 0x55ab0068ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:04.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab0085c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9119b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab008535d0 0x55ab0085f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab0085c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59412/0 (socket says 172.21.15.70:59412) 2023-10-18T12:34:04.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 -- 172.21.15.70:0/2497905877 learned_addr learned my addr 172.21.15.70:0/2497905877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:04.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9199c700 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab0085d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 -- 172.21.15.70:0/2497905877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 msgr2=0x55ab0085d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab0085d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 -- 172.21.15.70:0/2497905877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab008535d0 msgr2=0x55ab0085f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab008535d0 0x55ab0085f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.005+0000 7fca9099a700 1 -- 172.21.15.70:0/2497905877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c1a5970 con 0x55ab00858be0 2023-10-18T12:34:04.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca9099a700 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab0085c8e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca7c096cc0 tx=0x7fca7c0cb5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a96595fdf33bbbd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:04.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:04.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca74ff9700 1 -- 172.21.15.70:0/2497905877 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b9070 con 0x55ab00858be0 2023-10-18T12:34:04.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab007aaef0 con 0x55ab00858be0 2023-10-18T12:34:04.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca74ff9700 1 -- 172.21.15.70:0/2497905877 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fca7c0a1050 con 0x55ab00858be0 2023-10-18T12:34:04.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca74ff9700 1 -- 172.21.15.70:0/2497905877 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7c0b91d0 con 0x55ab00858be0 2023-10-18T12:34:04.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fcaa6c74b80 10 client.26729 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ab005d0de0 con 0x55ab00858be0 2023-10-18T12:34:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca74ff9700 1 -- 172.21.15.70:0/2497905877 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab007aaef0 con 0x55ab00858be0 2023-10-18T12:34:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fca74ff9700 1 -- 172.21.15.70:0/2497905877 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca7c0051d0 con 0x55ab00858be0 2023-10-18T12:34:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fcaa6c74b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.006+0000 7fcaa6c74b80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fcaa6c74b80 20 client.26729 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab0086c650 0x55ab0086ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fcaa6c74b80 4 client.26729 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab008228b0 con 0x55ab0086c650 2023-10-18T12:34:04.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fca9119b700 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab0086c650 0x55ab0086ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fca9119b700 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab0086c650 0x55ab0086ea40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49493491a54aab4 server_cookie=a2fd3993b1abd383 in_seq=0 out_seq=0 2023-10-18T12:34:04.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.007+0000 7fca74ff9700 10 client.26729 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:04.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fca74ff9700 1 -- 172.21.15.70:0/2497905877 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab008228b0 con 0x55ab0086c650 2023-10-18T12:34:04.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fca74ff9700 10 client.26729 handle_command_reply: tid=0 2023-10-18T12:34:04.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 client.26729 shutdown 2023-10-18T12:34:04.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 msgr2=0x55ab0085c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab0085c8e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca7c096cc0 tx=0x7fca7c0cb5c0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 shutdown_connections 2023-10-18T12:34:04.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab0086c650 0x55ab0086ea40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab00859500 0x55ab0085d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab008535d0 0x55ab0085f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fca74ff9700 0 client.26729 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:04.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/2497905877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab0085c8e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 >> 172.21.15.70:0/2497905877 conn(0x55ab007c7720 msgr2=0x55ab00852080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:04.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.008+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 shutdown_connections 2023-10-18T12:34:04.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.009+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/2497905877 wait complete. 2023-10-18T12:34:04.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.009+0000 7fcaa6c74b80 20 client.26729 trim_cache size 0 max 16384 2023-10-18T12:34:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 Processor -- start 2023-10-18T12:34:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 -- start start 2023-10-18T12:34:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00863c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab0085c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 0x55ab0085f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab00626920 con 0x55ab00858be0 2023-10-18T12:34:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0085a8f0 con 0x55ab00859500 2023-10-18T12:34:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.026+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0084a2e0 con 0x55ab008535d0 2023-10-18T12:34:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 0x55ab0085f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 0x55ab0085f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:59416/0 (socket says 172.21.15.70:59416) 2023-10-18T12:34:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 -- 172.21.15.70:0/1008930995 learned_addr learned my addr 172.21.15.70:0/1008930995 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9099a700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab0085c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9119b700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00863c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 -- 172.21.15.70:0/1008930995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 msgr2=0x55ab00863c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00863c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 -- 172.21.15.70:0/1008930995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 msgr2=0x55ab0085c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab0085c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 -- 172.21.15.70:0/1008930995 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab005b9c60 con 0x55ab00859500 2023-10-18T12:34:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9099a700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab0085c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9119b700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00863c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:04.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.027+0000 7fca9199c700 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 0x55ab0085f080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca84017810 tx=0x7fca8401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62f66d82b69a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:04.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.028+0000 7fca77fff700 1 -- 172.21.15.70:0/1008930995 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400e050 con 0x55ab00859500 2023-10-18T12:34:04.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.028+0000 7fca77fff700 1 -- 172.21.15.70:0/1008930995 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8400f040 con 0x55ab00859500 2023-10-18T12:34:04.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.028+0000 7fca77fff700 1 -- 172.21.15.70:0/1008930995 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84018a70 con 0x55ab00859500 2023-10-18T12:34:04.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.028+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/1008930995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 msgr2=0x55ab0085f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.028+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 0x55ab0085f080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca84017810 tx=0x7fca8401b000 comp rx=0 tx=0).stop 2023-10-18T12:34:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/1008930995 shutdown_connections 2023-10-18T12:34:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00859500 0x55ab0085f080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00863c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/1008930995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00858be0 0x55ab0085c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/1008930995 >> 172.21.15.70:0/1008930995 conn(0x55ab007c7720 msgr2=0x55ab00856d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/1008930995 shutdown_connections 2023-10-18T12:34:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/1008930995 wait complete. 2023-10-18T12:34:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 -- start start 2023-10-18T12:34:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.029+0000 7fcaa6c74b80 1 Processor -- start 2023-10-18T12:34:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00862350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab00862890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00859500 0x55ab00862dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab008641b0 con 0x55ab00859500 2023-10-18T12:34:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab008c0340 con 0x55ab00858be0 2023-10-18T12:34:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab008c0480 con 0x55ab008535d0 2023-10-18T12:34:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fcaa6c74b80 10 client.0 _ll_register_callbacks cb 0x55ab0068ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9119b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00862350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9099a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab00862890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9119b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00862350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:47994/0 (socket says 172.21.15.70:47994) 2023-10-18T12:34:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9119b700 1 -- 172.21.15.70:0/82291186 learned_addr learned my addr 172.21.15.70:0/82291186 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:04.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9199c700 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00859500 0x55ab00862dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9099a700 1 -- 172.21.15.70:0/82291186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 msgr2=0x55ab00862350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9099a700 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00862350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9099a700 1 -- 172.21.15.70:0/82291186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00859500 msgr2=0x55ab00862dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:04.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9099a700 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00859500 0x55ab00862dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.030+0000 7fca9099a700 1 -- 172.21.15.70:0/82291186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca88021a00 con 0x55ab00858be0 2023-10-18T12:34:04.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca9099a700 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab00862890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0979b0 tx=0x7fca7c005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3287e506257afcdd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:04.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:04.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca74ff9700 1 -- 172.21.15.70:0/82291186 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b9050 con 0x55ab00858be0 2023-10-18T12:34:04.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca7c0a3240 con 0x55ab00858be0 2023-10-18T12:34:04.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca74ff9700 1 -- 172.21.15.70:0/82291186 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fca7c021520 con 0x55ab00858be0 2023-10-18T12:34:04.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca74ff9700 1 -- 172.21.15.70:0/82291186 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7c0cbab0 con 0x55ab00858be0 2023-10-18T12:34:04.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 10 client.26735 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:04.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fca8c1a5970 con 0x55ab00858be0 2023-10-18T12:34:04.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca74ff9700 1 -- 172.21.15.70:0/82291186 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7c0a3240 con 0x55ab00858be0 2023-10-18T12:34:04.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fca74ff9700 1 -- 172.21.15.70:0/82291186 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca7c005260 con 0x55ab00858be0 2023-10-18T12:34:04.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 10 client.26735 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:04.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 10 client.26735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:04.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 20 client.26735 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:04.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab008c1830 0x55ab008c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:04.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 4 client.26735 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:34:04.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.031+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab0079ee50 con 0x55ab008c1830 2023-10-18T12:34:04.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.032+0000 7fca9119b700 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab008c1830 0x55ab008c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:04.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.032+0000 7fca9119b700 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab008c1830 0x55ab008c3c50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b79079e9fedd4e server_cookie=1cbe8bedd0b060f in_seq=0 out_seq=0 2023-10-18T12:34:04.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.032+0000 7fca74ff9700 10 client.26735 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:04.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fca74ff9700 1 -- 172.21.15.70:0/82291186 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ab0079ee50 con 0x55ab008c1830 2023-10-18T12:34:04.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fca74ff9700 10 client.26735 handle_command_reply: tid=0 2023-10-18T12:34:04.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 client.26735 shutdown 2023-10-18T12:34:04.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 msgr2=0x55ab00862890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab00862890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0979b0 tx=0x7fca7c005bb0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 shutdown_connections 2023-10-18T12:34:04.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ab008c1830 0x55ab008c3c50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab00859500 0x55ab00862dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ab008535d0 0x55ab00862350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fca74ff9700 0 client.26735 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:04.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 --2- 172.21.15.70:0/82291186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab00858be0 0x55ab00862890 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.033+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 >> 172.21.15.70:0/82291186 conn(0x55ab007c7720 msgr2=0x55ab008474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:04.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 shutdown_connections 2023-10-18T12:34:04.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fcaa6c74b80 1 -- 172.21.15.70:0/82291186 wait complete. 2023-10-18T12:34:04.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fcaa6c74b80 20 client.26735 trim_cache size 0 max 16384 2023-10-18T12:34:04.053 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:34:04.053 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:34:04.053 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:34:04.054 INFO:teuthology.orchestra.run.smithi070.stdout: "b5f74590-2034-435e-b634-3b7d7c19d814": { 2023-10-18T12:34:04.054 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:34:04.054 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "b5f74590-2034-435e-b634-3b7d7c19d814", 2023-10-18T12:34:04.054 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:34:04.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:34:04.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:34:04.055 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:34:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fca780a1670 msgr2=0x7fca780a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fca780a1670 0x7fca780a3b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fca84009410 tx=0x7fca84009920 comp rx=0 tx=0).stop 2023-10-18T12:34:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 msgr2=0x7fca8c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:04.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 0x7fca8c199320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca7c005950 tx=0x7fca7c00fa00 comp rx=0 tx=0).stop 2023-10-18T12:34:04.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 shutdown_connections 2023-10-18T12:34:04.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c106730 0x7fca8c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca8c100720 0x7fca8c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fca780a1670 0x7fca780a3b30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 --2- 172.21.15.70:0/1553397195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca8c1010e0 0x7fca8c199320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:04.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.034+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 >> 172.21.15.70:0/1553397195 conn(0x7fca8c0fc1b0 msgr2=0x7fca8c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:04.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.035+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 shutdown_connections 2023-10-18T12:34:04.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:04.035+0000 7fca933ff700 1 -- 172.21.15.70:0/1553397195 wait complete. 2023-10-18T12:34:04.059 INFO:tasks.cephfs.filesystem:scrub status for tag:b5f74590-2034-435e-b634-3b7d7c19d814 - {'path': '/', 'tag': 'b5f74590-2034-435e-b634-3b7d7c19d814', 'options': 'recursive,force'} 2023-10-18T12:34:04.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 901834 11.77 MB/sec execute 273 sec latency 44.255 ms 2023-10-18T12:34:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:04 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:34:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:04 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:04 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:34:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:04 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:05.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:04 smithi070 ceph-mon[98639]: pgmap v558: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2023-10-18T12:34:05.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:04 smithi070 ceph-mon[98639]: osdmap e96: 12 total, 12 up, 12 in 2023-10-18T12:34:05.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:04 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:34:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:04 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:04 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:34:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:04 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:05.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:04 smithi152 ceph-mon[134767]: pgmap v558: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2023-10-18T12:34:05.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:04 smithi152 ceph-mon[134767]: osdmap e96: 12 total, 12 up, 12 in 2023-10-18T12:34:05.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:04 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:34:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:04 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:04 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:34:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:04 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:04 smithi186 ceph-mon[135253]: pgmap v558: 129 pgs: 129 active+clean; 496 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2023-10-18T12:34:05.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:04 smithi186 ceph-mon[135253]: osdmap e96: 12 total, 12 up, 12 in 2023-10-18T12:34:05.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 903416 11.75 MB/sec execute 274 sec latency 26.570 ms 2023-10-18T12:34:06.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 905554 11.76 MB/sec execute 275 sec latency 100.199 ms 2023-10-18T12:34:07.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:06 smithi070 ceph-mon[98639]: pgmap v560: 129 pgs: 129 active+clean; 480 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2023-10-18T12:34:07.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:06 smithi152 ceph-mon[134767]: pgmap v560: 129 pgs: 129 active+clean; 480 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2023-10-18T12:34:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:06 smithi186 ceph-mon[135253]: pgmap v560: 129 pgs: 129 active+clean; 480 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2023-10-18T12:34:07.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 907490 11.74 MB/sec execute 276 sec latency 36.163 ms 2023-10-18T12:34:08.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 909578 11.74 MB/sec execute 277 sec latency 74.551 ms 2023-10-18T12:34:09.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:08 smithi070 ceph-mon[98639]: pgmap v561: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2023-10-18T12:34:09.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:08 smithi152 ceph-mon[134767]: pgmap v561: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2023-10-18T12:34:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:08 smithi186 ceph-mon[135253]: pgmap v561: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2023-10-18T12:34:09.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 912194 11.75 MB/sec execute 278 sec latency 11.979 ms 2023-10-18T12:34:10.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:09 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:34:10.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:09 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:34:10.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:09 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:34:10.861 INFO:tasks.workunit.client.0.smithi070.stdout: 1 913568 11.73 MB/sec execute 279 sec latency 88.805 ms 2023-10-18T12:34:11.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:10 smithi070 ceph-mon[98639]: pgmap v562: 129 pgs: 129 active+clean; 471 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 243 op/s 2023-10-18T12:34:11.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:10 smithi152 ceph-mon[134767]: pgmap v562: 129 pgs: 129 active+clean; 471 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 243 op/s 2023-10-18T12:34:11.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:10 smithi186 ceph-mon[135253]: pgmap v562: 129 pgs: 129 active+clean; 471 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 243 op/s 2023-10-18T12:34:11.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 915718 11.74 MB/sec execute 280 sec latency 60.443 ms 2023-10-18T12:34:12.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 917101 11.76 MB/sec execute 281 sec latency 71.069 ms 2023-10-18T12:34:13.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:12 smithi070 ceph-mon[98639]: pgmap v563: 129 pgs: 129 active+clean; 471 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 224 op/s 2023-10-18T12:34:13.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:12 smithi152 ceph-mon[134767]: pgmap v563: 129 pgs: 129 active+clean; 471 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 224 op/s 2023-10-18T12:34:13.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:12 smithi186 ceph-mon[135253]: pgmap v563: 129 pgs: 129 active+clean; 471 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 224 op/s 2023-10-18T12:34:13.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 918979 11.77 MB/sec execute 282 sec latency 59.472 ms 2023-10-18T12:34:14.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 921134 11.77 MB/sec execute 283 sec latency 13.668 ms 2023-10-18T12:34:15.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:14 smithi070 ceph-mon[98639]: pgmap v564: 129 pgs: 129 active+clean; 479 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-18T12:34:15.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:14 smithi070 ceph-mon[98639]: osdmap e97: 12 total, 12 up, 12 in 2023-10-18T12:34:15.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:14 smithi152 ceph-mon[134767]: pgmap v564: 129 pgs: 129 active+clean; 479 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-18T12:34:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:14 smithi152 ceph-mon[134767]: osdmap e97: 12 total, 12 up, 12 in 2023-10-18T12:34:15.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:14 smithi186 ceph-mon[135253]: pgmap v564: 129 pgs: 129 active+clean; 479 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 255 op/s 2023-10-18T12:34:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:14 smithi186 ceph-mon[135253]: osdmap e97: 12 total, 12 up, 12 in 2023-10-18T12:34:15.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 923948 11.79 MB/sec execute 284 sec latency 14.886 ms 2023-10-18T12:34:16.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 924716 11.76 MB/sec execute 285 sec latency 261.488 ms 2023-10-18T12:34:17.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:16 smithi070 ceph-mon[98639]: pgmap v566: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 249 op/s 2023-10-18T12:34:17.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:16 smithi152 ceph-mon[134767]: pgmap v566: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 249 op/s 2023-10-18T12:34:17.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:16 smithi186 ceph-mon[135253]: pgmap v566: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 249 op/s 2023-10-18T12:34:17.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 927092 11.76 MB/sec execute 286 sec latency 336.145 ms 2023-10-18T12:34:18.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 928946 11.76 MB/sec execute 287 sec latency 52.115 ms 2023-10-18T12:34:19.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:18 smithi070 ceph-mon[98639]: pgmap v567: 129 pgs: 129 active+clean; 491 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2023-10-18T12:34:19.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:18 smithi152 ceph-mon[134767]: pgmap v567: 129 pgs: 129 active+clean; 491 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2023-10-18T12:34:19.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:18 smithi186 ceph-mon[135253]: pgmap v567: 129 pgs: 129 active+clean; 491 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2023-10-18T12:34:19.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 931416 11.77 MB/sec execute 288 sec latency 12.701 ms 2023-10-18T12:34:20.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:20 smithi070 ceph-mon[98639]: pgmap v568: 129 pgs: 129 active+clean; 492 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 211 op/s 2023-10-18T12:34:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:20 smithi152 ceph-mon[134767]: pgmap v568: 129 pgs: 129 active+clean; 492 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 211 op/s 2023-10-18T12:34:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:20 smithi186 ceph-mon[135253]: pgmap v568: 129 pgs: 129 active+clean; 492 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 211 op/s 2023-10-18T12:34:20.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 933756 11.76 MB/sec execute 289 sec latency 20.350 ms 2023-10-18T12:34:21.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 935115 11.75 MB/sec execute 290 sec latency 54.803 ms 2023-10-18T12:34:22.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 937672 11.76 MB/sec execute 291 sec latency 36.161 ms 2023-10-18T12:34:23.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:22 smithi070 ceph-mon[98639]: pgmap v569: 129 pgs: 129 active+clean; 494 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 228 op/s 2023-10-18T12:34:23.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:22 smithi152 ceph-mon[134767]: pgmap v569: 129 pgs: 129 active+clean; 494 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 228 op/s 2023-10-18T12:34:23.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:22 smithi186 ceph-mon[135253]: pgmap v569: 129 pgs: 129 active+clean; 494 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 228 op/s 2023-10-18T12:34:23.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 939505 11.75 MB/sec execute 292 sec latency 20.019 ms 2023-10-18T12:34:24.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 941979 11.76 MB/sec execute 293 sec latency 31.099 ms 2023-10-18T12:34:25.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:24 smithi070 ceph-mon[98639]: pgmap v570: 129 pgs: 129 active+clean; 499 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 211 op/s 2023-10-18T12:34:25.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:24 smithi152 ceph-mon[134767]: pgmap v570: 129 pgs: 129 active+clean; 499 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 211 op/s 2023-10-18T12:34:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:24 smithi186 ceph-mon[135253]: pgmap v570: 129 pgs: 129 active+clean; 499 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 211 op/s 2023-10-18T12:34:25.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 944439 11.76 MB/sec execute 294 sec latency 15.635 ms 2023-10-18T12:34:26.862 INFO:tasks.workunit.client.0.smithi070.stdout: 1 946017 11.75 MB/sec execute 295 sec latency 79.419 ms 2023-10-18T12:34:27.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:26 smithi070 ceph-mon[98639]: pgmap v571: 129 pgs: 129 active+clean; 500 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 196 op/s 2023-10-18T12:34:27.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:26 smithi152 ceph-mon[134767]: pgmap v571: 129 pgs: 129 active+clean; 500 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 196 op/s 2023-10-18T12:34:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:26 smithi186 ceph-mon[135253]: pgmap v571: 129 pgs: 129 active+clean; 500 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 196 op/s 2023-10-18T12:34:27.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 948640 11.76 MB/sec execute 296 sec latency 25.001 ms 2023-10-18T12:34:28.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 950350 11.75 MB/sec execute 297 sec latency 19.377 ms 2023-10-18T12:34:29.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:28 smithi070 ceph-mon[98639]: pgmap v572: 129 pgs: 129 active+clean; 507 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:34:29.176 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:28 smithi152 ceph-mon[134767]: pgmap v572: 129 pgs: 129 active+clean; 507 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:34:29.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:28 smithi186 ceph-mon[135253]: pgmap v572: 129 pgs: 129 active+clean; 507 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:34:29.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 952810 11.76 MB/sec execute 298 sec latency 14.436 ms 2023-10-18T12:34:30.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:30 smithi070 ceph-mon[98639]: pgmap v573: 129 pgs: 129 active+clean; 510 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2023-10-18T12:34:30.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:30 smithi152 ceph-mon[134767]: pgmap v573: 129 pgs: 129 active+clean; 510 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2023-10-18T12:34:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:30 smithi186 ceph-mon[135253]: pgmap v573: 129 pgs: 129 active+clean; 510 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 202 op/s 2023-10-18T12:34:30.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 955381 11.76 MB/sec execute 299 sec latency 15.717 ms 2023-10-18T12:34:31.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 956851 11.75 MB/sec execute 300 sec latency 149.618 ms 2023-10-18T12:34:32.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 959420 11.76 MB/sec execute 301 sec latency 21.084 ms 2023-10-18T12:34:33.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:32 smithi070 ceph-mon[98639]: pgmap v574: 129 pgs: 129 active+clean; 517 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2023-10-18T12:34:33.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:32 smithi152 ceph-mon[134767]: pgmap v574: 129 pgs: 129 active+clean; 517 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2023-10-18T12:34:33.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:32 smithi186 ceph-mon[135253]: pgmap v574: 129 pgs: 129 active+clean; 517 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2023-10-18T12:34:33.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 960989 11.74 MB/sec execute 302 sec latency 19.268 ms 2023-10-18T12:34:34.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:34:34.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 -- 172.21.15.70:0/4103985814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 msgr2=0x7f253c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 --2- 172.21.15.70:0/4103985814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c068120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2538009990 tx=0x7f2538009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 -- 172.21.15.70:0/4103985814 shutdown_connections 2023-10-18T12:34:34.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 --2- 172.21.15.70:0/4103985814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 0x7f253c10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 --2- 172.21.15.70:0/4103985814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c068120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 --2- 172.21.15.70:0/4103985814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 0x7f253c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 -- 172.21.15.70:0/4103985814 >> 172.21.15.70:0/4103985814 conn(0x7f253c0767f0 msgr2=0x7f253c076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 -- 172.21.15.70:0/4103985814 shutdown_connections 2023-10-18T12:34:34.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.273+0000 7f25443ff700 1 -- 172.21.15.70:0/4103985814 wait complete. 2023-10-18T12:34:34.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.274+0000 7f25443ff700 1 Processor -- start 2023-10-18T12:34:34.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.274+0000 7f25443ff700 1 -- start start 2023-10-18T12:34:34.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f25443ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f25443ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 0x7f253c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f25443ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 0x7f253c100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f25443ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a4f70 con 0x7f253c105170 2023-10-18T12:34:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f25443ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a50e0 con 0x7f253c067d40 2023-10-18T12:34:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f25443ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a52a0 con 0x7f253c105b30 2023-10-18T12:34:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c0ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47722/0 (socket says 172.21.15.70:47722) 2023-10-18T12:34:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 -- 172.21.15.70:0/4208331719 learned_addr learned my addr 172.21.15.70:0/4208331719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254199a700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 0x7f253c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254299c700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 0x7f253c100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 -- 172.21.15.70:0/4208331719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 msgr2=0x7f253c100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 0x7f253c100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 -- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 msgr2=0x7f253c0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 0x7f253c0ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254219b700 1 -- 172.21.15.70:0/4208331719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380095f0 con 0x7f253c067d40 2023-10-18T12:34:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254299c700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 0x7f253c100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:34.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.275+0000 7f254199a700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 0x7f253c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:34.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.276+0000 7f254219b700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c0ff6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2538000c00 tx=0x7f2538005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.276+0000 7f25337fe700 1 -- 172.21.15.70:0/4208331719 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253801d070 con 0x7f253c067d40 2023-10-18T12:34:34.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.276+0000 7f25337fe700 1 -- 172.21.15.70:0/4208331719 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2538022470 con 0x7f253c067d40 2023-10-18T12:34:34.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.276+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c1a5520 con 0x7f253c067d40 2023-10-18T12:34:34.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.276+0000 7f25337fe700 1 -- 172.21.15.70:0/4208331719 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538017650 con 0x7f253c067d40 2023-10-18T12:34:34.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.276+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c1a59a0 con 0x7f253c067d40 2023-10-18T12:34:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.279+0000 7f25337fe700 1 -- 172.21.15.70:0/4208331719 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f25380225e0 con 0x7f253c067d40 2023-10-18T12:34:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.280+0000 7f2557c4fb80 1 Processor -- start 2023-10-18T12:34:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.280+0000 7f2557c4fb80 1 -- start start 2023-10-18T12:34:34.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa98be0 0x55b9bfa98fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfa93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfa93a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bfa066e0 con 0x55b9bfa99500 2023-10-18T12:34:34.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bf9ecc60 con 0x55b9bfa98be0 2023-10-18T12:34:34.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bf9ea580 con 0x55b9bfa935d0 2023-10-18T12:34:34.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfa93a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f25337fe700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25280a1670 0x7f25280a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfa93a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37812/0 (socket says 172.21.15.70:37812) 2023-10-18T12:34:34.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254299c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa98be0 0x55b9bfa98fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 -- 172.21.15.70:0/3000065813 learned_addr learned my addr 172.21.15.70:0/3000065813 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f25337fe700 1 -- 172.21.15.70:0/4208331719 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f25380ce6f0 con 0x7f253c067d40 2023-10-18T12:34:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254219b700 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfa93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 -- 172.21.15.70:0/3000065813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa98be0 msgr2=0x55b9bfa98fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa98be0 0x55b9bfa98fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 -- 172.21.15.70:0/3000065813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 msgr2=0x55b9bfa93090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfa93090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254199a700 1 -- 172.21.15.70:0/3000065813 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9bf9a33f0 con 0x55b9bfa935d0 2023-10-18T12:34:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.281+0000 7f254219b700 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfa93090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:34:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.282+0000 7f254299c700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25280a1670 0x7f25280a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.282+0000 7f254199a700 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfa93a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f252c0115f0 tx=0x7f252c011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158d0c74e825d094 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.282+0000 7f2530bfe700 1 -- 172.21.15.70:0/3000065813 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c004030 con 0x55b9bfa935d0 2023-10-18T12:34:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.282+0000 7f254299c700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25280a1670 0x7f25280a3b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f25340060b0 tx=0x7f2534009040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.282+0000 7f2530bfe700 1 -- 172.21.15.70:0/3000065813 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f252c003680 con 0x55b9bfa935d0 2023-10-18T12:34:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.282+0000 7f2530bfe700 1 -- 172.21.15.70:0/3000065813 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c019730 con 0x55b9bfa935d0 2023-10-18T12:34:34.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3000065813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 msgr2=0x55b9bfa93a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfa93a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f252c0115f0 tx=0x7f252c011900 comp rx=0 tx=0).stop 2023-10-18T12:34:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3000065813 shutdown_connections 2023-10-18T12:34:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfa93a50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa98be0 0x55b9bfa98fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3000065813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfa93090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3000065813 >> 172.21.15.70:0/3000065813 conn(0x55b9bfa07720 msgr2=0x55b9bfa8db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3000065813 shutdown_connections 2023-10-18T12:34:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.283+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3000065813 wait complete. 2023-10-18T12:34:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 -- start start 2023-10-18T12:34:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 Processor -- start 2023-10-18T12:34:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfa9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 0x55b9bfa9cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bf9e5e70 con 0x55b9bfa98be0 2023-10-18T12:34:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bf982710 con 0x55b9bfa935d0 2023-10-18T12:34:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bf9823b0 con 0x55b9bfa99500 2023-10-18T12:34:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.284+0000 7f2557c4fb80 10 client.0 _ll_register_callbacks cb 0x55b9bf8cca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254219b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 0x55b9bfa9cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254199a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfa9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 0x55b9bfa9cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37814/0 (socket says 172.21.15.70:37814) 2023-10-18T12:34:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254219b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa9f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47740/0 (socket says 172.21.15.70:47740) 2023-10-18T12:34:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 -- 172.21.15.70:0/463315336 learned_addr learned my addr 172.21.15.70:0/463315336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 -- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 msgr2=0x55b9bfa9f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa9f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 -- 172.21.15.70:0/463315336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 msgr2=0x55b9bfa9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfa9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 -- 172.21.15.70:0/463315336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c009d50 con 0x55b9bfa99500 2023-10-18T12:34:34.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254199a700 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfa9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:34.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.285+0000 7f254299c700 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 0x55b9bfa9cd70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f253400eb10 tx=0x7f2534004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233bfd94e21af1dd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2525ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:34:34.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2525ffb700 1 -- 172.21.15.70:0/463315336 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253401c040 con 0x55b9bfa99500 2023-10-18T12:34:34.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2525ffb700 1 -- 172.21.15.70:0/463315336 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f2534007da0 con 0x55b9bfa99500 2023-10-18T12:34:34.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9bf9eaef0 con 0x55b9bfa99500 2023-10-18T12:34:34.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2525ffb700 1 -- 172.21.15.70:0/463315336 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2534004920 con 0x55b9bfa99500 2023-10-18T12:34:34.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2557c4fb80 10 client.26743 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:34.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b9bf810de0 con 0x55b9bfa99500 2023-10-18T12:34:34.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.286+0000 7f2525ffb700 1 -- 172.21.15.70:0/463315336 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9bf9eaef0 con 0x55b9bfa99500 2023-10-18T12:34:34.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2525ffb700 1 -- 172.21.15.70:0/463315336 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f253402a070 con 0x55b9bfa99500 2023-10-18T12:34:34.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2557c4fb80 10 client.26743 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:34.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2557c4fb80 10 client.26743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:34.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2557c4fb80 20 client.26743 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:34.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfaacdb0 0x55b9bfaaf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2557c4fb80 4 client.26743 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:34.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9bfa628b0 con 0x55b9bfaacdb0 2023-10-18T12:34:34.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.287+0000 7f254219b700 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfaacdb0 0x55b9bfaaf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.288+0000 7f254219b700 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfaacdb0 0x55b9bfaaf1d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3bc710f626a218 server_cookie=881d2065c001a500 in_seq=0 out_seq=0 2023-10-18T12:34:34.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.288+0000 7f2525ffb700 10 client.26743 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:34.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.288+0000 7f2525ffb700 1 -- 172.21.15.70:0/463315336 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b9bfa628b0 con 0x55b9bfaacdb0 2023-10-18T12:34:34.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.288+0000 7f2525ffb700 10 client.26743 handle_command_reply: tid=0 2023-10-18T12:34:34.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 client.26743 shutdown 2023-10-18T12:34:34.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 msgr2=0x55b9bfa9cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 0x55b9bfa9cd70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f253400eb10 tx=0x7f2534004ad0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 shutdown_connections 2023-10-18T12:34:34.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfaacdb0 0x55b9bfaaf1d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa99500 0x55b9bfa9cd70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa9f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2525ffb700 0 client.26743 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/463315336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfa9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 >> 172.21.15.70:0/463315336 conn(0x55b9bfa07720 msgr2=0x55b9bfa92080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 shutdown_connections 2023-10-18T12:34:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 1 -- 172.21.15.70:0/463315336 wait complete. 2023-10-18T12:34:34.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.289+0000 7f2557c4fb80 20 client.26743 trim_cache size 0 max 16384 2023-10-18T12:34:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 Processor -- start 2023-10-18T12:34:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 -- start start 2023-10-18T12:34:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa8f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa98be0 0x55b9bfa885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfaa3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bf866920 con 0x55b9bfa99500 2023-10-18T12:34:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bfa8a230 con 0x55b9bfa935d0 2023-10-18T12:34:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.307+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bfa8e0c0 con 0x55b9bfa98be0 2023-10-18T12:34:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa8f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254199a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa98be0 0x55b9bfa885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254299c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfaa3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa8f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47750/0 (socket says 172.21.15.70:47750) 2023-10-18T12:34:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 -- 172.21.15.70:0/3155435402 learned_addr learned my addr 172.21.15.70:0/3155435402 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 -- 172.21.15.70:0/3155435402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa98be0 msgr2=0x55b9bfa885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa98be0 0x55b9bfa885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 -- 172.21.15.70:0/3155435402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 msgr2=0x55b9bfaa3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfaa3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 -- 172.21.15.70:0/3155435402 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9bf7f9c60 con 0x55b9bfa935d0 2023-10-18T12:34:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254299c700 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfaa3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f254219b700 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa8f740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2538006010 tx=0x7f25380d0b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a5ab1bf675de8b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.308+0000 7f25313ff700 1 -- 172.21.15.70:0/3155435402 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253809baa0 con 0x55b9bfa935d0 2023-10-18T12:34:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f25313ff700 1 -- 172.21.15.70:0/3155435402 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f253809b3a0 con 0x55b9bfa935d0 2023-10-18T12:34:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f25313ff700 1 -- 172.21.15.70:0/3155435402 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380228c0 con 0x55b9bfa935d0 2023-10-18T12:34:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3155435402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 msgr2=0x55b9bfa8f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa8f740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2538006010 tx=0x7f25380d0b00 comp rx=0 tx=0).stop 2023-10-18T12:34:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3155435402 shutdown_connections 2023-10-18T12:34:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa99500 0x55b9bfaa3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa935d0 0x55b9bfa8f740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/3155435402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa98be0 0x55b9bfa885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3155435402 >> 172.21.15.70:0/3155435402 conn(0x55b9bfa07720 msgr2=0x55b9bfa9bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3155435402 shutdown_connections 2023-10-18T12:34:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.309+0000 7f2557c4fb80 1 -- 172.21.15.70:0/3155435402 wait complete. 2023-10-18T12:34:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 -- start start 2023-10-18T12:34:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 Processor -- start 2023-10-18T12:34:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfaa30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfaa3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 0x55b9bfaffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bfaa3f20 con 0x55b9bfa98be0 2023-10-18T12:34:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bfb00120 con 0x55b9bfa99500 2023-10-18T12:34:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bfb00260 con 0x55b9bfa935d0 2023-10-18T12:34:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.310+0000 7f2557c4fb80 10 client.0 _ll_register_callbacks cb 0x55b9bf8cca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254199a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfaa3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254219b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfaa30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 0x55b9bfaffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254219b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfaa30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37844/0 (socket says 172.21.15.70:37844) 2023-10-18T12:34:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254199a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfaa3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48558/0 (socket says 172.21.15.70:48558) 2023-10-18T12:34:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254199a700 1 -- 172.21.15.70:0/702244822 learned_addr learned my addr 172.21.15.70:0/702244822 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 0x55b9bfaffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47766/0 (socket says 172.21.15.70:47766) 2023-10-18T12:34:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 -- 172.21.15.70:0/702244822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 msgr2=0x55b9bfaa30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfaa30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 -- 172.21.15.70:0/702244822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 msgr2=0x55b9bfaa3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfaa3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 -- 172.21.15.70:0/702244822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9bf810de0 con 0x55b9bfa99500 2023-10-18T12:34:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f254299c700 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 0x55b9bfaffbe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2534022640 tx=0x7f2534017720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800b8423e7f4aba5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.311+0000 7f2525ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2525ffb700 1 -- 172.21.15.70:0/702244822 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534017e40 con 0x55b9bfa99500 2023-10-18T12:34:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f252c003680 con 0x55b9bfa99500 2023-10-18T12:34:34.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2525ffb700 1 -- 172.21.15.70:0/702244822 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f25340227b0 con 0x55b9bfa99500 2023-10-18T12:34:34.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2525ffb700 1 -- 172.21.15.70:0/702244822 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2534009920 con 0x55b9bfa99500 2023-10-18T12:34:34.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 10 client.26753 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:34.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f253c1a59a0 con 0x55b9bfa99500 2023-10-18T12:34:34.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2525ffb700 1 -- 172.21.15.70:0/702244822 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f252c003680 con 0x55b9bfa99500 2023-10-18T12:34:34.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2525ffb700 1 -- 172.21.15.70:0/702244822 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f253400eb40 con 0x55b9bfa99500 2023-10-18T12:34:34.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 10 client.26753 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:34.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:34.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 20 client.26753 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:34.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfb01960 0x55b9bfb03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 4 client.26753 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:34:34.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.312+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b9bf9dee50 con 0x55b9bfb01960 2023-10-18T12:34:34.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f254219b700 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfb01960 0x55b9bfb03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f2525ffb700 10 client.26753 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:34.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f254219b700 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfb01960 0x55b9bfb03d80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1018ecdde86a6456 server_cookie=9192dbcfbaf15afe in_seq=0 out_seq=0 2023-10-18T12:34:34.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f2525ffb700 1 -- 172.21.15.70:0/702244822 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b9bf9dee50 con 0x55b9bfb01960 2023-10-18T12:34:34.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f2525ffb700 10 client.26753 handle_command_reply: tid=0 2023-10-18T12:34:34.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f2557c4fb80 1 client.26753 shutdown 2023-10-18T12:34:34.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 msgr2=0x55b9bfaffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.313+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 0x55b9bfaffbe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2534022640 tx=0x7f2534017720 comp rx=0 tx=0).stop 2023-10-18T12:34:34.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 shutdown_connections 2023-10-18T12:34:34.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b9bfb01960 0x55b9bfb03d80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9bfa99500 0x55b9bfaffbe0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9bfa935d0 0x55b9bfaa30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 --2- 172.21.15.70:0/702244822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b9bfa98be0 0x55b9bfaa3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 >> 172.21.15.70:0/702244822 conn(0x55b9bfa07720 msgr2=0x55b9bfa874b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2525ffb700 0 client.26753 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:34.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 shutdown_connections 2023-10-18T12:34:34.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 1 -- 172.21.15.70:0/702244822 wait complete. 2023-10-18T12:34:34.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f2557c4fb80 20 client.26753 trim_cache size 0 max 16384 2023-10-18T12:34:34.333 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:34:34.333 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:34:34.333 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:34:34.333 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:34:34.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25280a1670 msgr2=0x7f25280a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f25443ff700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25280a1670 0x7f25280a3b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f25340060b0 tx=0x7f2534009040 comp rx=0 tx=0).stop 2023-10-18T12:34:34.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 msgr2=0x7f253c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.314+0000 7f25443ff700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c0ff6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2538000c00 tx=0x7f2538005740 comp rx=0 tx=0).stop 2023-10-18T12:34:34.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 shutdown_connections 2023-10-18T12:34:34.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f253c105b30 0x7f253c100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25280a1670 0x7f25280a3b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f253c067d40 0x7f253c0ff6b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 --2- 172.21.15.70:0/4208331719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f253c105170 0x7f253c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 >> 172.21.15.70:0/4208331719 conn(0x7f253c0767f0 msgr2=0x7f253c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 shutdown_connections 2023-10-18T12:34:34.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.315+0000 7f25443ff700 1 -- 172.21.15.70:0/4208331719 wait complete. 2023-10-18T12:34:34.338 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:34:34.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:34:34.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 -- 172.21.15.70:0/313251029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e4101060 msgr2=0x7fe5e410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/313251029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e4101060 0x7fe5e410d570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00b0a0 tx=0x7fe5dc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 -- 172.21.15.70:0/313251029 shutdown_connections 2023-10-18T12:34:34.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/313251029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e4101060 0x7fe5e410d570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/313251029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41066b0 0x7fe5e4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/313251029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e41006a0 0x7fe5e4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 -- 172.21.15.70:0/313251029 >> 172.21.15.70:0/313251029 conn(0x7fe5e40fc1d0 msgr2=0x7fe5e40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 -- 172.21.15.70:0/313251029 shutdown_connections 2023-10-18T12:34:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.548+0000 7fe5e93ff700 1 -- 172.21.15.70:0/313251029 wait complete. 2023-10-18T12:34:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.549+0000 7fe5e93ff700 1 Processor -- start 2023-10-18T12:34:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.549+0000 7fe5e93ff700 1 -- start start 2023-10-18T12:34:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e93ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41006a0 0x7fe5e419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e93ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 0x7fe5e419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e93ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e41066b0 0x7fe5e41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e93ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4196f40 con 0x7fe5e4101060 2023-10-18T12:34:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e93ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41970b0 con 0x7fe5e41066b0 2023-10-18T12:34:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e93ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4197270 con 0x7fe5e41006a0 2023-10-18T12:34:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41006a0 0x7fe5e419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 0x7fe5e419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41006a0 0x7fe5e419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37860/0 (socket says 172.21.15.70:37860) 2023-10-18T12:34:34.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 0x7fe5e419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48568/0 (socket says 172.21.15.70:48568) 2023-10-18T12:34:34.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e2ffd700 1 -- 172.21.15.70:0/2018345526 learned_addr learned my addr 172.21.15.70:0/2018345526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:34.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e37fe700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e41066b0 0x7fe5e41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 -- 172.21.15.70:0/2018345526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41006a0 msgr2=0x7fe5e419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41006a0 0x7fe5e419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 -- 172.21.15.70:0/2018345526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e41066b0 msgr2=0x7fe5e41968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e41066b0 0x7fe5e41968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 -- 172.21.15.70:0/2018345526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009d00 con 0x7fe5e4101060 2023-10-18T12:34:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5e37fe700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e41066b0 0x7fe5e41968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.550+0000 7fe5dbfff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 0x7fe5e419ccf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00e9f0 tx=0x7fe5cc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.551+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc00c9c0 con 0x7fe5e4101060 2023-10-18T12:34:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.551+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4197550 con 0x7fe5e4101060 2023-10-18T12:34:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.551+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5cc010430 con 0x7fe5e4101060 2023-10-18T12:34:34.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.551+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc018680 con 0x7fe5e4101060 2023-10-18T12:34:34.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.551+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e41a37a0 con 0x7fe5e4101060 2023-10-18T12:34:34.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.552+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4108cc0 con 0x7fe5e4101060 2023-10-18T12:34:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.554+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe5cc010860 con 0x7fe5e4101060 2023-10-18T12:34:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.555+0000 7fe5e0ff9700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe5d00a15f0 0x7fe5d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.555+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe5cc014070 con 0x7fe5e4101060 2023-10-18T12:34:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.556+0000 7fe5e2ffd700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe5d00a15f0 0x7fe5d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.556+0000 7fe5e2ffd700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe5d00a15f0 0x7fe5d00a3ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005950 tx=0x7fe5d400b500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.559+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe5cc096550 con 0x7fe5e4101060 2023-10-18T12:34:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.821+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5e4101de0 con 0x7fe5e4101060 2023-10-18T12:34:34.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.822+0000 7fe5e0ff9700 1 -- 172.21.15.70:0/2018345526 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fe5cc099b70 con 0x7fe5e4101060 2023-10-18T12:34:34.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:34:34.824 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:34:34.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe5d00a15f0 msgr2=0x7fe5d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe5d00a15f0 0x7fe5d00a3ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005950 tx=0x7fe5d400b500 comp rx=0 tx=0).stop 2023-10-18T12:34:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 msgr2=0x7fe5e419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 0x7fe5e419ccf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00e9f0 tx=0x7fe5cc00ed00 comp rx=0 tx=0).stop 2023-10-18T12:34:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 shutdown_connections 2023-10-18T12:34:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe5d00a15f0 0x7fe5d00a3ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5e41066b0 0x7fe5e41968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5e41006a0 0x7fe5e419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 --2- 172.21.15.70:0/2018345526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4101060 0x7fe5e419ccf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 >> 172.21.15.70:0/2018345526 conn(0x7fe5e40fc1d0 msgr2=0x7fe5e40fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 shutdown_connections 2023-10-18T12:34:34.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:34.825+0000 7fe5e93ff700 1 -- 172.21.15.70:0/2018345526 wait complete. 2023-10-18T12:34:34.830 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:34:34.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:34:34.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 963429 11.75 MB/sec execute 303 sec latency 12.793 ms 2023-10-18T12:34:35.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 -- 172.21.15.70:0/118098303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde64072b90 msgr2=0x7fde64072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/118098303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde64072b90 0x7fde64072f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fde54009990 tx=0x7fde54009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 -- 172.21.15.70:0/118098303 shutdown_connections 2023-10-18T12:34:35.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/118098303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64111730 0x7fde64113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/118098303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde64072b90 0x7fde64072f70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/118098303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde640734b0 0x7fde641111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.071+0000 7fde6a1ff700 1 -- 172.21.15.70:0/118098303 >> 172.21.15.70:0/118098303 conn(0x7fde640fc1b0 msgr2=0x7fde640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.072+0000 7fde6a1ff700 1 -- 172.21.15.70:0/118098303 shutdown_connections 2023-10-18T12:34:35.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.072+0000 7fde6a1ff700 1 -- 172.21.15.70:0/118098303 wait complete. 2023-10-18T12:34:35.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 Processor -- start 2023-10-18T12:34:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 -- start start 2023-10-18T12:34:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 0x7fde6419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde640734b0 0x7fde6419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde64111730 0x7fde641a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641a2150 con 0x7fde64111730 2023-10-18T12:34:35.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641a22c0 con 0x7fde640734b0 2023-10-18T12:34:35.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde6a1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641a2480 con 0x7fde64072b90 2023-10-18T12:34:35.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 0x7fde6419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 0x7fde6419d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37864/0 (socket says 172.21.15.70:37864) 2023-10-18T12:34:35.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde640734b0 0x7fde6419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde637fe700 1 -- 172.21.15.70:0/3721478433 learned_addr learned my addr 172.21.15.70:0/3721478433 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:35.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.073+0000 7fde63fff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde64111730 0x7fde641a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde637fe700 1 -- 172.21.15.70:0/3721478433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde640734b0 msgr2=0x7fde6419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde637fe700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde640734b0 0x7fde6419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde637fe700 1 -- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde64111730 msgr2=0x7fde641a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde637fe700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde64111730 0x7fde641a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde637fe700 1 -- 172.21.15.70:0/3721478433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde540095f0 con 0x7fde64072b90 2023-10-18T12:34:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde640734b0 0x7fde6419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:35.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde637fe700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 0x7fde6419d280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde54005910 tx=0x7fde54004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.074+0000 7fde60ff9700 1 -- 172.21.15.70:0/3721478433 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5401d070 con 0x7fde64072b90 2023-10-18T12:34:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.075+0000 7fde60ff9700 1 -- 172.21.15.70:0/3721478433 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde54022470 con 0x7fde64072b90 2023-10-18T12:34:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.075+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde641a2700 con 0x7fde64072b90 2023-10-18T12:34:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.075+0000 7fde60ff9700 1 -- 172.21.15.70:0/3721478433 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54017650 con 0x7fde64072b90 2023-10-18T12:34:35.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.075+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde641a9fa0 con 0x7fde64072b90 2023-10-18T12:34:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.078+0000 7fde60ff9700 1 -- 172.21.15.70:0/3721478433 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fde540225e0 con 0x7fde64072b90 2023-10-18T12:34:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.079+0000 7fde60ff9700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fde4c0a1670 0x7fde4c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.079+0000 7fde7dbddb80 1 Processor -- start 2023-10-18T12:34:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.079+0000 7fde60ff9700 1 -- 172.21.15.70:0/3721478433 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fde540ce720 con 0x7fde64072b90 2023-10-18T12:34:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.079+0000 7fde7dbddb80 1 -- start start 2023-10-18T12:34:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fde4c0a1670 0x7fde4c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 0x557b4134afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b41345090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b413455d0 0x557b41345a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b412b86e0 con 0x557b4134b500 2023-10-18T12:34:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4129ec60 con 0x557b4134abe0 2023-10-18T12:34:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4129c580 con 0x557b413455d0 2023-10-18T12:34:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 0x557b4134afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 0x557b4134afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47802/0 (socket says 172.21.15.70:47802) 2023-10-18T12:34:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 -- 172.21.15.70:0/527341482 learned_addr learned my addr 172.21.15.70:0/527341482 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde63fff700 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b413455d0 0x557b41345a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde62ffd700 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b41345090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fde4c0a1670 0x7fde4c0a3b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fde6419e780 tx=0x7fde58010870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 -- 172.21.15.70:0/527341482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b413455d0 msgr2=0x557b41345a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b413455d0 0x557b41345a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 -- 172.21.15.70:0/527341482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 msgr2=0x557b41345090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.080+0000 7fde637fe700 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b41345090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.081+0000 7fde637fe700 1 -- 172.21.15.70:0/527341482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b412553f0 con 0x557b4134abe0 2023-10-18T12:34:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.081+0000 7fde637fe700 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 0x557b4134afc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fde5409b360 tx=0x7fde54009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ed0adbd9140eb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.081+0000 7fde49ffb700 1 -- 172.21.15.70:0/527341482 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5409ad30 con 0x557b4134abe0 2023-10-18T12:34:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.081+0000 7fde49ffb700 1 -- 172.21.15.70:0/527341482 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5409bad0 con 0x557b4134abe0 2023-10-18T12:34:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.081+0000 7fde49ffb700 1 -- 172.21.15.70:0/527341482 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540ce300 con 0x557b4134abe0 2023-10-18T12:34:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 -- 172.21.15.70:0/527341482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 msgr2=0x557b4134afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 0x557b4134afc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fde5409b360 tx=0x7fde54009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 -- 172.21.15.70:0/527341482 shutdown_connections 2023-10-18T12:34:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b413455d0 0x557b41345a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134abe0 0x557b4134afc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/527341482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b41345090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 -- 172.21.15.70:0/527341482 >> 172.21.15.70:0/527341482 conn(0x557b412b9720 msgr2=0x557b4133fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 -- 172.21.15.70:0/527341482 shutdown_connections 2023-10-18T12:34:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 -- 172.21.15.70:0/527341482 wait complete. 2023-10-18T12:34:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.082+0000 7fde7dbddb80 1 -- start start 2023-10-18T12:34:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 Processor -- start 2023-10-18T12:34:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41351410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b4134ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b41297e70 con 0x557b4134b500 2023-10-18T12:34:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b41234710 con 0x557b413455d0 2023-10-18T12:34:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b412343b0 con 0x557b4134abe0 2023-10-18T12:34:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde7dbddb80 10 client.0 _ll_register_callbacks cb 0x557b4117ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41351410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde63fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b4134ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37890/0 (socket says 172.21.15.70:37890) 2023-10-18T12:34:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde62ffd700 1 -- 172.21.15.70:0/3940507522 learned_addr learned my addr 172.21.15.70:0/3940507522 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.083+0000 7fde637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41351410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47814/0 (socket says 172.21.15.70:47814) 2023-10-18T12:34:35.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde637fe700 1 -- 172.21.15.70:0/3940507522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 msgr2=0x557b4134e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde637fe700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde637fe700 1 -- 172.21.15.70:0/3940507522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 msgr2=0x557b4134ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde637fe700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b4134ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde637fe700 1 -- 172.21.15.70:0/3940507522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde641a9fa0 con 0x557b413455d0 2023-10-18T12:34:35.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde63fff700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b4134ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde637fe700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41351410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x557b41351af0 tx=0x7fde54022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18303a842be31244 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde42ffd700 1 -- 172.21.15.70:0/3940507522 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5409ad30 con 0x557b413455d0 2023-10-18T12:34:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b4129cef0 con 0x557b413455d0 2023-10-18T12:34:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde42ffd700 1 -- 172.21.15.70:0/3940507522 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fde5409b3d0 con 0x557b413455d0 2023-10-18T12:34:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde42ffd700 1 -- 172.21.15.70:0/3940507522 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde540ce510 con 0x557b413455d0 2023-10-18T12:34:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde7dbddb80 10 client.26771 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.084+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557b410c2de0 con 0x557b413455d0 2023-10-18T12:34:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde42ffd700 1 -- 172.21.15.70:0/3940507522 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b4129cef0 con 0x557b413455d0 2023-10-18T12:34:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde42ffd700 1 -- 172.21.15.70:0/3940507522 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fde5409a8c0 con 0x557b413455d0 2023-10-18T12:34:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde7dbddb80 10 client.26771 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde7dbddb80 10 client.26771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde7dbddb80 20 client.26771 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b4135e590 0x557b413609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde7dbddb80 4 client.26771 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b413148b0 con 0x557b4135e590 2023-10-18T12:34:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.085+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b4135e590 0x557b413609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.086+0000 7fde42ffd700 10 client.26771 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.086+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b4135e590 0x557b413609b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba55246b6e5928a server_cookie=b35df9abefeff4e5 in_seq=0 out_seq=0 2023-10-18T12:34:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde42ffd700 1 -- 172.21.15.70:0/3940507522 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b413148b0 con 0x557b4135e590 2023-10-18T12:34:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde42ffd700 10 client.26771 handle_command_reply: tid=0 2023-10-18T12:34:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 client.26771 shutdown 2023-10-18T12:34:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 msgr2=0x557b41351410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41351410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x557b41351af0 tx=0x7fde54022af0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 shutdown_connections 2023-10-18T12:34:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b4135e590 0x557b413609b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b4134ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41351410 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3940507522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 >> 172.21.15.70:0/3940507522 conn(0x557b412b9720 msgr2=0x557b41344080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde42ffd700 0 client.26771 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 shutdown_connections 2023-10-18T12:34:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.087+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3940507522 wait complete. 2023-10-18T12:34:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.088+0000 7fde7dbddb80 20 client.26771 trim_cache size 0 max 16384 2023-10-18T12:34:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 Processor -- start 2023-10-18T12:34:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 -- start start 2023-10-18T12:34:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b413455d0 0x557b41355c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134b500 0x557b4134d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4133aaf0 con 0x557b413455d0 2023-10-18T12:34:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b41118920 con 0x557b4134b500 2023-10-18T12:34:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4134da50 con 0x557b4134abe0 2023-10-18T12:34:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37896/0 (socket says 172.21.15.70:37896) 2023-10-18T12:34:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde62ffd700 1 -- 172.21.15.70:0/3758828020 learned_addr learned my addr 172.21.15.70:0/3758828020 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde63fff700 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134b500 0x557b4134d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.101+0000 7fde637fe700 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b413455d0 0x557b41355c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde62ffd700 1 -- 172.21.15.70:0/3758828020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134b500 msgr2=0x557b4134d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134b500 0x557b4134d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde62ffd700 1 -- 172.21.15.70:0/3758828020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b413455d0 msgr2=0x557b41355c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b413455d0 0x557b41355c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde62ffd700 1 -- 172.21.15.70:0/3758828020 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b410abc60 con 0x557b4134abe0 2023-10-18T12:34:35.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134cfd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde58005950 tx=0x7fde58015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884ce6dfc0617275 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:35.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde4a7fc700 1 -- 172.21.15.70:0/3758828020 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5802e070 con 0x557b4134abe0 2023-10-18T12:34:35.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde4a7fc700 1 -- 172.21.15.70:0/3758828020 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde580044e0 con 0x557b4134abe0 2023-10-18T12:34:35.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.102+0000 7fde4a7fc700 1 -- 172.21.15.70:0/3758828020 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58024040 con 0x557b4134abe0 2023-10-18T12:34:35.115 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:34 smithi070 ceph-mon[98639]: pgmap v575: 129 pgs: 129 active+clean; 521 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 229 op/s 2023-10-18T12:34:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3758828020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 msgr2=0x557b4134cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134cfd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde58005950 tx=0x7fde58015000 comp rx=0 tx=0).stop 2023-10-18T12:34:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3758828020 shutdown_connections 2023-10-18T12:34:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b4134b500 0x557b4134d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b413455d0 0x557b41355c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3758828020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b4134cfd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.103+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3758828020 >> 172.21.15.70:0/3758828020 conn(0x557b412b9720 msgr2=0x557b413448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.104+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3758828020 shutdown_connections 2023-10-18T12:34:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.104+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3758828020 wait complete. 2023-10-18T12:34:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.104+0000 7fde7dbddb80 1 -- start start 2023-10-18T12:34:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.104+0000 7fde7dbddb80 1 Processor -- start 2023-10-18T12:34:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41353c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b41354150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b413b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b41354a10 con 0x557b4134b500 2023-10-18T12:34:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b413b18c0 con 0x557b413455d0 2023-10-18T12:34:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b413b1a00 con 0x557b4134abe0 2023-10-18T12:34:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde7dbddb80 10 client.0 _ll_register_callbacks cb 0x557b4117ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41353c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b41354150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41353c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47842/0 (socket says 172.21.15.70:47842) 2023-10-18T12:34:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b41354150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37906/0 (socket says 172.21.15.70:37906) 2023-10-18T12:34:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde637fe700 1 -- 172.21.15.70:0/3996473161 learned_addr learned my addr 172.21.15.70:0/3996473161 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde63fff700 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b413b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde637fe700 1 -- 172.21.15.70:0/3996473161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 msgr2=0x557b41354150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde637fe700 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b41354150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.105+0000 7fde637fe700 1 -- 172.21.15.70:0/3996473161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 msgr2=0x557b413b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde637fe700 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b413b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde637fe700 1 -- 172.21.15.70:0/3996473161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde58013170 con 0x557b413455d0 2023-10-18T12:34:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde637fe700 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41353c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fde54006010 tx=0x7fde540cc010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0b179c744aa793 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde42ffd700 1 -- 172.21.15.70:0/3996473161 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5409ad30 con 0x557b413455d0 2023-10-18T12:34:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde42ffd700 1 -- 172.21.15.70:0/3996473161 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fde5409b3d0 con 0x557b413455d0 2023-10-18T12:34:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde5409bad0 con 0x557b413455d0 2023-10-18T12:34:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde42ffd700 1 -- 172.21.15.70:0/3996473161 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5409b1a0 con 0x557b413455d0 2023-10-18T12:34:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde7dbddb80 10 client.26777 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557b410e9250 con 0x557b413455d0 2023-10-18T12:34:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.106+0000 7fde42ffd700 1 -- 172.21.15.70:0/3996473161 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5409bad0 con 0x557b413455d0 2023-10-18T12:34:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde42ffd700 1 -- 172.21.15.70:0/3996473161 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fde540971c0 con 0x557b413455d0 2023-10-18T12:34:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde7dbddb80 10 client.26777 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde7dbddb80 10 client.26777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde7dbddb80 20 client.26777 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b413b2c60 0x557b413b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde7dbddb80 4 client.26777 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:34:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557b41290e50 con 0x557b413b2c60 2023-10-18T12:34:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b413b2c60 0x557b413b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde62ffd700 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b413b2c60 0x557b413b5050 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18019f934520f56a server_cookie=6d7533231d0d69a3 in_seq=0 out_seq=0 2023-10-18T12:34:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.107+0000 7fde42ffd700 10 client.26777 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde42ffd700 1 -- 172.21.15.70:0/3996473161 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557b41290e50 con 0x557b413b2c60 2023-10-18T12:34:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde42ffd700 10 client.26777 handle_command_reply: tid=0 2023-10-18T12:34:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 client.26777 shutdown 2023-10-18T12:34:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 msgr2=0x557b41353c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41353c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fde54006010 tx=0x7fde540cc010 comp rx=0 tx=0).stop 2023-10-18T12:34:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 shutdown_connections 2023-10-18T12:34:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557b413b2c60 0x557b413b5050 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4134b500 0x557b413b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557b413455d0 0x557b41353c10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde42ffd700 0 client.26777 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 --2- 172.21.15.70:0/3996473161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557b4134abe0 0x557b41354150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 >> 172.21.15.70:0/3996473161 conn(0x557b412b9720 msgr2=0x557b41344440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 shutdown_connections 2023-10-18T12:34:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 1 -- 172.21.15.70:0/3996473161 wait complete. 2023-10-18T12:34:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.108+0000 7fde7dbddb80 20 client.26777 trim_cache size 0 max 16384 2023-10-18T12:34:35.131 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:34:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fde4c0a1670 msgr2=0x7fde4c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fde4c0a1670 0x7fde4c0a3b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fde6419e780 tx=0x7fde58010870 comp rx=0 tx=0).stop 2023-10-18T12:34:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 msgr2=0x7fde6419d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 0x7fde6419d280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde54005910 tx=0x7fde54004730 comp rx=0 tx=0).stop 2023-10-18T12:34:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 shutdown_connections 2023-10-18T12:34:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde64111730 0x7fde641a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde64072b90 0x7fde6419d280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fde4c0a1670 0x7fde4c0a3b30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 --2- 172.21.15.70:0/3721478433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde640734b0 0x7fde6419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 >> 172.21.15.70:0/3721478433 conn(0x7fde640fc1b0 msgr2=0x7fde640fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 shutdown_connections 2023-10-18T12:34:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:35.109+0000 7fde6a1ff700 1 -- 172.21.15.70:0/3721478433 wait complete. 2023-10-18T12:34:35.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:34 smithi152 ceph-mon[134767]: pgmap v575: 129 pgs: 129 active+clean; 521 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 229 op/s 2023-10-18T12:34:35.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:34 smithi186 ceph-mon[135253]: pgmap v575: 129 pgs: 129 active+clean; 521 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 229 op/s 2023-10-18T12:34:35.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 965843 11.75 MB/sec execute 304 sec latency 16.709 ms 2023-10-18T12:34:36.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:35 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2018345526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:34:36.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:34:36.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:34:36.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:35 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2018345526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:34:36.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:35 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2018345526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:34:36.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 -- 172.21.15.70:0/604898826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c106730 msgr2=0x7fa22c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 --2- 172.21.15.70:0/604898826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c106730 0x7fa22c106b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009990 tx=0x7fa21c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 -- 172.21.15.70:0/604898826 shutdown_connections 2023-10-18T12:34:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 --2- 172.21.15.70:0/604898826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c1010e0 0x7fa22c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 --2- 172.21.15.70:0/604898826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c106730 0x7fa22c106b10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 --2- 172.21.15.70:0/604898826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa22c100720 0x7fa22c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 -- 172.21.15.70:0/604898826 >> 172.21.15.70:0/604898826 conn(0x7fa22c0fc1b0 msgr2=0x7fa22c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 -- 172.21.15.70:0/604898826 shutdown_connections 2023-10-18T12:34:36.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.350+0000 7fa2323ff700 1 -- 172.21.15.70:0/604898826 wait complete. 2023-10-18T12:34:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.351+0000 7fa2323ff700 1 Processor -- start 2023-10-18T12:34:36.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 -- start start 2023-10-18T12:34:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 0x7fa22c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c1010e0 0x7fa22c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa22c106730 0x7fa22c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19dc90 con 0x7fa22c1010e0 2023-10-18T12:34:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19de00 con 0x7fa22c100720 2023-10-18T12:34:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa2323ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19dfc0 con 0x7fa22c106730 2023-10-18T12:34:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 0x7fa22c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa223fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c1010e0 0x7fa22c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 0x7fa22c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47868/0 (socket says 172.21.15.70:47868) 2023-10-18T12:34:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa22bfff700 1 -- 172.21.15.70:0/3729433568 learned_addr learned my addr 172.21.15.70:0/3729433568 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.352+0000 7fa23099c700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa22c106730 0x7fa22c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa22bfff700 1 -- 172.21.15.70:0/3729433568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa22c106730 msgr2=0x7fa22c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa22c106730 0x7fa22c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa22bfff700 1 -- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c1010e0 msgr2=0x7fa22c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c1010e0 0x7fa22c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa22bfff700 1 -- 172.21.15.70:0/3729433568 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0095f0 con 0x7fa22c100720 2023-10-18T12:34:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 0x7fa22c198dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa21c000c00 tx=0x7fa21c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa229ffb700 1 -- 172.21.15.70:0/3729433568 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c01d070 con 0x7fa22c100720 2023-10-18T12:34:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c19e240 con 0x7fa22c100720 2023-10-18T12:34:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa229ffb700 1 -- 172.21.15.70:0/3729433568 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21c022470 con 0x7fa22c100720 2023-10-18T12:34:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa229ffb700 1 -- 172.21.15.70:0/3729433568 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c017650 con 0x7fa22c100720 2023-10-18T12:34:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.353+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c1a5ae0 con 0x7fa22c100720 2023-10-18T12:34:36.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.357+0000 7fa229ffb700 1 -- 172.21.15.70:0/3729433568 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa21c0225e0 con 0x7fa22c100720 2023-10-18T12:34:36.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.358+0000 7fa229ffb700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa2180a15f0 0x7fa2180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.358+0000 7fa229ffb700 1 -- 172.21.15.70:0/3729433568 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa21c0ce760 con 0x7fa22c100720 2023-10-18T12:34:36.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.358+0000 7fa245d7db80 1 Processor -- start 2023-10-18T12:34:36.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.358+0000 7fa223fff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa2180a15f0 0x7fa2180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.358+0000 7fa245d7db80 1 -- start start 2023-10-18T12:34:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899575110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561899575650 0x561899575ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa245d7db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618994e8760 con 0x56189957ac60 2023-10-18T12:34:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa245d7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618994cece0 con 0x56189957b580 2023-10-18T12:34:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa245d7db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618994cc600 con 0x561899575650 2023-10-18T12:34:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48650/0 (socket says 172.21.15.70:48650) 2023-10-18T12:34:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 -- 172.21.15.70:0/1997889572 learned_addr learned my addr 172.21.15.70:0/1997889572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa223fff700 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899575110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa23099c700 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561899575650 0x561899575ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa223fff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa2180a15f0 0x7fa2180a3ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa22c19a2c0 tx=0x7fa214009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 -- 172.21.15.70:0/1997889572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561899575650 msgr2=0x561899575ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561899575650 0x561899575ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 -- 172.21.15.70:0/1997889572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 msgr2=0x561899575110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899575110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.359+0000 7fa22bfff700 1 -- 172.21.15.70:0/1997889572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561899485470 con 0x56189957ac60 2023-10-18T12:34:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.360+0000 7fa22bfff700 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957b040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa21c09b170 tx=0x7fa21c097830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdfbe97753876e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.360+0000 7fa2227fc700 1 -- 172.21.15.70:0/1997889572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c09add0 con 0x56189957ac60 2023-10-18T12:34:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.360+0000 7fa2227fc700 1 -- 172.21.15.70:0/1997889572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21c09bb70 con 0x56189957ac60 2023-10-18T12:34:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.360+0000 7fa2227fc700 1 -- 172.21.15.70:0/1997889572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0ce3e0 con 0x56189957ac60 2023-10-18T12:34:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.360+0000 7fa245d7db80 1 -- 172.21.15.70:0/1997889572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 msgr2=0x56189957b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.360+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957b040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa21c09b170 tx=0x7fa21c097830 comp rx=0 tx=0).stop 2023-10-18T12:34:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 -- 172.21.15.70:0/1997889572 shutdown_connections 2023-10-18T12:34:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561899575650 0x561899575ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957b040 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1997889572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899575110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 -- 172.21.15.70:0/1997889572 >> 172.21.15.70:0/1997889572 conn(0x5618994e97a0 msgr2=0x56189956fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 -- 172.21.15.70:0/1997889572 shutdown_connections 2023-10-18T12:34:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 -- 172.21.15.70:0/1997889572 wait complete. 2023-10-18T12:34:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.361+0000 7fa245d7db80 1 -- start start 2023-10-18T12:34:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 Processor -- start 2023-10-18T12:34:36.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 0x561899581530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x56189957ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618994c7ef0 con 0x56189957ac60 2023-10-18T12:34:36.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561899464790 con 0x561899575650 2023-10-18T12:34:36.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561899464430 con 0x56189957b580 2023-10-18T12:34:36.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa245d7db80 10 client.0 _ll_register_callbacks cb 0x5618993aeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:36.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 0x561899581530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 0x561899581530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47878/0 (socket says 172.21.15.70:47878) 2023-10-18T12:34:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa223fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.362+0000 7fa22bfff700 1 -- 172.21.15.70:0/3160923662 learned_addr learned my addr 172.21.15.70:0/3160923662 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa23099c700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x56189957ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa22bfff700 1 -- 172.21.15.70:0/3160923662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 msgr2=0x56189957ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x56189957ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa22bfff700 1 -- 172.21.15.70:0/3160923662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 msgr2=0x56189957e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa22bfff700 1 -- 172.21.15.70:0/3160923662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c1a5ae0 con 0x561899575650 2023-10-18T12:34:36.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa23099c700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x56189957ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:34:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa223fff700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 0x561899581530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa21c097970 tx=0x7fa21c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad3422e3beffea5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa20b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3160923662 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0b3ea0 con 0x561899575650 2023-10-18T12:34:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3160923662 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa21c09b470 con 0x561899575650 2023-10-18T12:34:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618994ccf70 con 0x561899575650 2023-10-18T12:34:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3160923662 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa21c09add0 con 0x561899575650 2023-10-18T12:34:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa245d7db80 10 client.26789 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.363+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5618992f2e60 con 0x561899575650 2023-10-18T12:34:36.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3160923662 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618994ccf70 con 0x561899575650 2023-10-18T12:34:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3160923662 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa21c0b2070 con 0x561899575650 2023-10-18T12:34:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa245d7db80 10 client.26789 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa245d7db80 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa245d7db80 20 client.26789 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:36.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56189958e770 0x561899590b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa245d7db80 4 client.26789 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561899544930 con 0x56189958e770 2023-10-18T12:34:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.364+0000 7fa223fff700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56189958e770 0x561899590b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.365+0000 7fa20b7fe700 10 client.26789 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.365+0000 7fa223fff700 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56189958e770 0x561899590b90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06fd5c8dabe03a server_cookie=39390482b7cb14dd in_seq=0 out_seq=0 2023-10-18T12:34:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3160923662 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561899544930 con 0x56189958e770 2023-10-18T12:34:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa20b7fe700 10 client.26789 handle_command_reply: tid=0 2023-10-18T12:34:36.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 client.26789 shutdown 2023-10-18T12:34:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 msgr2=0x561899581530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 0x561899581530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa21c097970 tx=0x7fa21c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 shutdown_connections 2023-10-18T12:34:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56189958e770 0x561899590b90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x56189957ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561899575650 0x561899581530 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3160923662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56189957ac60 0x56189957e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa20b7fe700 0 client.26789 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 >> 172.21.15.70:0/3160923662 conn(0x5618994e97a0 msgr2=0x561899574100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 shutdown_connections 2023-10-18T12:34:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 1 -- 172.21.15.70:0/3160923662 wait complete. 2023-10-18T12:34:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.367+0000 7fa245d7db80 20 client.26789 trim_cache size 0 max 16384 2023-10-18T12:34:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.385+0000 7fa245d7db80 1 Processor -- start 2023-10-18T12:34:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.385+0000 7fa245d7db80 1 -- start start 2023-10-18T12:34:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x56189957e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 0x561899585cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x561899578730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa245d7db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618993489a0 con 0x561899575650 2023-10-18T12:34:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa245d7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189956c310 con 0x56189957ac60 2023-10-18T12:34:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa245d7db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618995861f0 con 0x56189957b580 2023-10-18T12:34:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x56189957e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 0x561899585cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x56189957e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48670/0 (socket says 172.21.15.70:48670) 2023-10-18T12:34:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 0x561899585cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47890/0 (socket says 172.21.15.70:47890) 2023-10-18T12:34:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa22bfff700 1 -- 172.21.15.70:0/1129109058 learned_addr learned my addr 172.21.15.70:0/1129109058 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa23099c700 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x561899578730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 -- 172.21.15.70:0/1129109058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 msgr2=0x561899578730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x561899578730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 -- 172.21.15.70:0/1129109058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 msgr2=0x56189957e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x56189957e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa223fff700 1 -- 172.21.15.70:0/1129109058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618992dbce0 con 0x56189957ac60 2023-10-18T12:34:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa22bfff700 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x56189957e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.386+0000 7fa23099c700 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x561899578730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.387+0000 7fa223fff700 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 0x561899585cb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa2140094f0 tx=0x7fa214014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78074ef36f853040 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.387+0000 7fa222ffd700 1 -- 172.21.15.70:0/1129109058 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214022030 con 0x56189957ac60 2023-10-18T12:34:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.387+0000 7fa222ffd700 1 -- 172.21.15.70:0/1129109058 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa214023040 con 0x56189957ac60 2023-10-18T12:34:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.387+0000 7fa222ffd700 1 -- 172.21.15.70:0/1129109058 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21402b540 con 0x56189957ac60 2023-10-18T12:34:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 -- 172.21.15.70:0/1129109058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 msgr2=0x561899585cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 0x561899585cb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa2140094f0 tx=0x7fa214014000 comp rx=0 tx=0).stop 2023-10-18T12:34:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 -- 172.21.15.70:0/1129109058 shutdown_connections 2023-10-18T12:34:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957b580 0x561899578730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x56189957e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 --2- 172.21.15.70:0/1129109058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957ac60 0x561899585cb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 -- 172.21.15.70:0/1129109058 >> 172.21.15.70:0/1129109058 conn(0x5618994e97a0 msgr2=0x56189957e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 -- 172.21.15.70:0/1129109058 shutdown_connections 2023-10-18T12:34:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 -- 172.21.15.70:0/1129109058 wait complete. 2023-10-18T12:34:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.388+0000 7fa245d7db80 1 -- start start 2023-10-18T12:34:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 Processor -- start 2023-10-18T12:34:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x5618995dca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 0x5618995dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899584c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561899585250 con 0x561899575650 2023-10-18T12:34:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618995853c0 con 0x56189957b580 2023-10-18T12:34:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561899585580 con 0x56189957ac60 2023-10-18T12:34:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa245d7db80 10 client.0 _ll_register_callbacks cb 0x5618993aeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa23099c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899584c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x5618995dca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa223fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 0x5618995dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa23099c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899584c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47900/0 (socket says 172.21.15.70:47900) 2023-10-18T12:34:36.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa223fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 0x5618995dcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37970/0 (socket says 172.21.15.70:37970) 2023-10-18T12:34:36.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.389+0000 7fa23099c700 1 -- 172.21.15.70:0/3785954757 learned_addr learned my addr 172.21.15.70:0/3785954757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa223fff700 1 -- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 msgr2=0x561899584c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa223fff700 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899584c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa223fff700 1 -- 172.21.15.70:0/3785954757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 msgr2=0x5618995dca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa223fff700 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x5618995dca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa223fff700 1 -- 172.21.15.70:0/3785954757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618992f2e60 con 0x56189957ac60 2023-10-18T12:34:36.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa223fff700 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 0x5618995dcfd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x56189957cd40 tx=0x7fa21402ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aff32f0e4d87732 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa20b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:34:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3785954757 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214022040 con 0x56189957ac60 2023-10-18T12:34:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa21c09bb70 con 0x56189957ac60 2023-10-18T12:34:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3785954757 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa214039070 con 0x56189957ac60 2023-10-18T12:34:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.390+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3785954757 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa21402b2e0 con 0x56189957ac60 2023-10-18T12:34:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 10 client.26791 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa224009240 con 0x56189957ac60 2023-10-18T12:34:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3785954757 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21c09bb70 con 0x56189957ac60 2023-10-18T12:34:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3785954757 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa214031070 con 0x56189957ac60 2023-10-18T12:34:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 10 client.26791 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 20 client.26791 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5618995e49e0 0x5618995e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 4 client.26791 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:34:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5618994c0ed0 con 0x5618995e49e0 2023-10-18T12:34:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.391+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5618995e49e0 0x5618995e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.392+0000 7fa20b7fe700 10 client.26791 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.392+0000 7fa22bfff700 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5618995e49e0 0x5618995e6e00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d53bb28332f37d server_cookie=75f6640dae6dd74b in_seq=0 out_seq=0 2023-10-18T12:34:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa20b7fe700 1 -- 172.21.15.70:0/3785954757 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5618994c0ed0 con 0x5618995e49e0 2023-10-18T12:34:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa20b7fe700 10 client.26791 handle_command_reply: tid=0 2023-10-18T12:34:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 client.26791 shutdown 2023-10-18T12:34:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 msgr2=0x5618995dcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 0x5618995dcfd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x56189957cd40 tx=0x7fa21402ba60 comp rx=0 tx=0).stop 2023-10-18T12:34:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 shutdown_connections 2023-10-18T12:34:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5618995e49e0 0x5618995e6e00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56189957b580 0x561899584c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561899575650 0x5618995dca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 --2- 172.21.15.70:0/3785954757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56189957ac60 0x5618995dcfd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 >> 172.21.15.70:0/3785954757 conn(0x5618994e97a0 msgr2=0x561899578730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa20b7fe700 0 client.26791 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 shutdown_connections 2023-10-18T12:34:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 1 -- 172.21.15.70:0/3785954757 wait complete. 2023-10-18T12:34:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.393+0000 7fa245d7db80 20 client.26791 trim_cache size 0 max 16384 2023-10-18T12:34:36.414 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:34:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:34:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "806215bc-7b89-4001-a342-a4a49191e3ba", 2023-10-18T12:34:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:34:36.415 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:34:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa2180a15f0 msgr2=0x7fa2180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa2180a15f0 0x7fa2180a3ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa22c19a2c0 tx=0x7fa214009260 comp rx=0 tx=0).stop 2023-10-18T12:34:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 msgr2=0x7fa22c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 0x7fa22c198dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa21c000c00 tx=0x7fa21c005740 comp rx=0 tx=0).stop 2023-10-18T12:34:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 shutdown_connections 2023-10-18T12:34:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa22c106730 0x7fa22c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa2180a15f0 0x7fa2180a3ab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa22c100720 0x7fa22c198dc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 --2- 172.21.15.70:0/3729433568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa22c1010e0 0x7fa22c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 >> 172.21.15.70:0/3729433568 conn(0x7fa22c0fc1b0 msgr2=0x7fa22c0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 shutdown_connections 2023-10-18T12:34:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.394+0000 7fa2323ff700 1 -- 172.21.15.70:0/3729433568 wait complete. 2023-10-18T12:34:36.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:34:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 -- 172.21.15.70:0/3994670707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 msgr2=0x7f5030100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 --2- 172.21.15.70:0/3994670707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f5030100500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5028009f90 tx=0x7f502800b330 comp rx=0 tx=0).stop 2023-10-18T12:34:36.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 -- 172.21.15.70:0/3994670707 shutdown_connections 2023-10-18T12:34:36.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 --2- 172.21.15.70:0/3994670707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f503010d190 0x7f503010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 --2- 172.21.15.70:0/3994670707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f5030100500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 --2- 172.21.15.70:0/3994670707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5030100a40 0x7f503010cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 -- 172.21.15.70:0/3994670707 >> 172.21.15.70:0/3994670707 conn(0x7f50300767f0 msgr2=0x7f5030076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 -- 172.21.15.70:0/3994670707 shutdown_connections 2023-10-18T12:34:36.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.688+0000 7f5036dff700 1 -- 172.21.15.70:0/3994670707 wait complete. 2023-10-18T12:34:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.690+0000 7f5036dff700 1 Processor -- start 2023-10-18T12:34:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.690+0000 7f5036dff700 1 -- start start 2023-10-18T12:34:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5036dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f503010caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5036dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 0x7f5030109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5036dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f503010d190 0x7f503010a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5036dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503010a930 con 0x7f5030100a40 2023-10-18T12:34:36.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5036dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50301a97a0 con 0x7f503010d190 2023-10-18T12:34:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5036dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50301a98e0 con 0x7f5030100120 2023-10-18T12:34:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f502ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 0x7f5030109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f502ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 0x7f5030109bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48700/0 (socket says 172.21.15.70:48700) 2023-10-18T12:34:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f502ffff700 1 -- 172.21.15.70:0/674605435 learned_addr learned my addr 172.21.15.70:0/674605435 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.691+0000 7f5034b9b700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f503010caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.692+0000 7f503539c700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f503010d190 0x7f503010a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.693+0000 7f5034b9b700 1 -- 172.21.15.70:0/674605435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f503010d190 msgr2=0x7f503010a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.693+0000 7f5034b9b700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f503010d190 0x7f503010a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.693+0000 7f5034b9b700 1 -- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 msgr2=0x7f5030109bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.693+0000 7f5034b9b700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 0x7f5030109bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.693+0000 7f5034b9b700 1 -- 172.21.15.70:0/674605435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028009c40 con 0x7f5030100120 2023-10-18T12:34:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.693+0000 7f502ffff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 0x7f5030109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:36.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.694+0000 7f5034b9b700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f503010caf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5028000f80 tx=0x7f50280079d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.695+0000 7f502dffb700 1 -- 172.21.15.70:0/674605435 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800e040 con 0x7f5030100120 2023-10-18T12:34:36.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.695+0000 7f502dffb700 1 -- 172.21.15.70:0/674605435 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502801b450 con 0x7f5030100120 2023-10-18T12:34:36.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.695+0000 7f502dffb700 1 -- 172.21.15.70:0/674605435 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50280052a0 con 0x7f5030100120 2023-10-18T12:34:36.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.695+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50301a9ae0 con 0x7f5030100120 2023-10-18T12:34:36.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.696+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50301a9f10 con 0x7f5030100120 2023-10-18T12:34:36.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.702+0000 7f502dffb700 1 -- 172.21.15.70:0/674605435 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5028005400 con 0x7f5030100120 2023-10-18T12:34:36.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.706+0000 7f502dffb700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f50180a15f0 0x7f50180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.707+0000 7f502dffb700 1 -- 172.21.15.70:0/674605435 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f50280cf640 con 0x7f5030100120 2023-10-18T12:34:36.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.707+0000 7f504a71cb80 1 Processor -- start 2023-10-18T12:34:36.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.708+0000 7f502ffff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f50180a15f0 0x7f50180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 -- start start 2023-10-18T12:34:36.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba137500 0x55c6ba131090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 0x55c6ba131a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba0a46e0 con 0x55c6ba1315d0 2023-10-18T12:34:36.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba08ac60 con 0x55c6ba136be0 2023-10-18T12:34:36.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.709+0000 7f504a71cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba088580 con 0x55c6ba137500 2023-10-18T12:34:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f503539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 0x55c6ba131a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f503539c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 0x55c6ba131a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48706/0 (socket says 172.21.15.70:48706) 2023-10-18T12:34:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f503539c700 1 -- 172.21.15.70:0/3230890747 learned_addr learned my addr 172.21.15.70:0/3230890747 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f502ffff700 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba137500 0x55c6ba131090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f503539c700 1 -- 172.21.15.70:0/3230890747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba137500 msgr2=0x55c6ba131090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.710+0000 7f503539c700 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba137500 0x55c6ba131090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.711+0000 7f503539c700 1 -- 172.21.15.70:0/3230890747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 msgr2=0x55c6ba136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.711+0000 7f503539c700 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba136fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.711+0000 7f503539c700 1 -- 172.21.15.70:0/3230890747 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ba0413f0 con 0x55c6ba1315d0 2023-10-18T12:34:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.711+0000 7f502ffff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f50180a15f0 0x7f50180a3ab0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f503010b2f0 tx=0x7f502000b4a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.711+0000 7f503539c700 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 0x55c6ba131a50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5024009c30 tx=0x7f5024009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1a63969698d52b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.712+0000 7f5016ffd700 1 -- 172.21.15.70:0/3230890747 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024011d60 con 0x55c6ba1315d0 2023-10-18T12:34:36.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.712+0000 7f5016ffd700 1 -- 172.21.15.70:0/3230890747 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50240044e0 con 0x55c6ba1315d0 2023-10-18T12:34:36.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.712+0000 7f5016ffd700 1 -- 172.21.15.70:0/3230890747 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024019430 con 0x55c6ba1315d0 2023-10-18T12:34:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 -- 172.21.15.70:0/3230890747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 msgr2=0x55c6ba131a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 0x55c6ba131a50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5024009c30 tx=0x7f5024009f40 comp rx=0 tx=0).stop 2023-10-18T12:34:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 -- 172.21.15.70:0/3230890747 shutdown_connections 2023-10-18T12:34:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba1315d0 0x55c6ba131a50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3230890747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba137500 0x55c6ba131090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.713+0000 7f504a71cb80 1 -- 172.21.15.70:0/3230890747 >> 172.21.15.70:0/3230890747 conn(0x55c6ba0a5720 msgr2=0x55c6ba12bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.714+0000 7f504a71cb80 1 -- 172.21.15.70:0/3230890747 shutdown_connections 2023-10-18T12:34:36.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.714+0000 7f504a71cb80 1 -- 172.21.15.70:0/3230890747 wait complete. 2023-10-18T12:34:36.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.714+0000 7f504a71cb80 1 -- start start 2023-10-18T12:34:36.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.715+0000 7f504a71cb80 1 Processor -- start 2023-10-18T12:34:36.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba13a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba13adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 0x55c6ba13b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba083e70 con 0x55c6ba137500 2023-10-18T12:34:36.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba020710 con 0x55c6ba136be0 2023-10-18T12:34:36.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba0203b0 con 0x55c6ba1315d0 2023-10-18T12:34:36.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f504a71cb80 10 client.0 _ll_register_callbacks cb 0x55c6b9f6aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f502ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba13adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f502ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba13adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47942/0 (socket says 172.21.15.70:47942) 2023-10-18T12:34:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.716+0000 7f502ffff700 1 -- 172.21.15.70:0/3954516622 learned_addr learned my addr 172.21.15.70:0/3954516622 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba13a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f502ffff700 1 -- 172.21.15.70:0/3954516622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 msgr2=0x55c6ba13a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f502ffff700 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba13a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f502ffff700 1 -- 172.21.15.70:0/3954516622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 msgr2=0x55c6ba13b300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T12:34:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f502ffff700 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 0x55c6ba13b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f502ffff700 1 -- 172.21.15.70:0/3954516622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50240098e0 con 0x55c6ba136be0 2023-10-18T12:34:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f502ffff700 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba13adc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f50200119d0 tx=0x7f50200047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d41727e528329e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.717+0000 7f500ffff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:34:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.718+0000 7f500ffff700 1 -- 172.21.15.70:0/3954516622 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502001ea50 con 0x55c6ba136be0 2023-10-18T12:34:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.718+0000 7f500ffff700 1 -- 172.21.15.70:0/3954516622 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5020017070 con 0x55c6ba136be0 2023-10-18T12:34:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.718+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6ba088ef0 con 0x55c6ba136be0 2023-10-18T12:34:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.718+0000 7f500ffff700 1 -- 172.21.15.70:0/3954516622 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f502001ebb0 con 0x55c6ba136be0 2023-10-18T12:34:36.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.718+0000 7f504a71cb80 10 client.26807 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:36.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.718+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c6b9eaede0 con 0x55c6ba136be0 2023-10-18T12:34:36.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f500ffff700 1 -- 172.21.15.70:0/3954516622 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6ba088ef0 con 0x55c6ba136be0 2023-10-18T12:34:36.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f500ffff700 1 -- 172.21.15.70:0/3954516622 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5020032cb0 con 0x55c6ba136be0 2023-10-18T12:34:36.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f504a71cb80 10 client.26807 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:36.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f504a71cb80 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:36.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f504a71cb80 20 client.26807 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:36.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba15bfd0 0x55c6ba15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f504a71cb80 4 client.26807 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:34:36.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.719+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6ba1008b0 con 0x55c6ba15bfd0 2023-10-18T12:34:36.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.720+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba15bfd0 0x55c6ba15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.720+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba15bfd0 0x55c6ba15e3f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c75f37735c480d server_cookie=9a037892cbb67d04 in_seq=0 out_seq=0 2023-10-18T12:34:36.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.720+0000 7f500ffff700 10 client.26807 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.721+0000 7f500ffff700 1 -- 172.21.15.70:0/3954516622 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6ba1008b0 con 0x55c6ba15bfd0 2023-10-18T12:34:36.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.721+0000 7f500ffff700 10 client.26807 handle_command_reply: tid=0 2023-10-18T12:34:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.721+0000 7f504a71cb80 1 client.26807 shutdown 2023-10-18T12:34:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.722+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 msgr2=0x55c6ba13adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.722+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba13adc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f50200119d0 tx=0x7f50200047c0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.722+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 shutdown_connections 2023-10-18T12:34:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba15bfd0 0x55c6ba15e3f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 0x55c6ba13b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba13a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3954516622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba136be0 0x55c6ba13adc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 >> 172.21.15.70:0/3954516622 conn(0x55c6ba0a5720 msgr2=0x55c6ba130080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f500ffff700 0 client.26807 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 shutdown_connections 2023-10-18T12:34:36.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 1 -- 172.21.15.70:0/3954516622 wait complete. 2023-10-18T12:34:36.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.723+0000 7f504a71cb80 20 client.26807 trim_cache size 0 max 16384 2023-10-18T12:34:36.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.754+0000 7f504a71cb80 1 Processor -- start 2023-10-18T12:34:36.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.754+0000 7f504a71cb80 1 -- start start 2023-10-18T12:34:36.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba136be0 0x55c6ba12b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 0x55c6ba139190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f504a71cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b9f04920 con 0x55c6ba136be0 2023-10-18T12:34:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f504a71cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba158a50 con 0x55c6ba137500 2023-10-18T12:34:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f504a71cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba13be80 con 0x55c6ba1315d0 2023-10-18T12:34:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.755+0000 7f503539c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 0x55c6ba139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f503539c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 0x55c6ba139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47950/0 (socket says 172.21.15.70:47950) 2023-10-18T12:34:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f503539c700 1 -- 172.21.15.70:0/3176488352 learned_addr learned my addr 172.21.15.70:0/3176488352 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f502ffff700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba136be0 0x55c6ba12b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f5034b9b700 1 -- 172.21.15.70:0/3176488352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 msgr2=0x55c6ba139190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 0x55c6ba139190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f5034b9b700 1 -- 172.21.15.70:0/3176488352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba136be0 msgr2=0x55c6ba12b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba136be0 0x55c6ba12b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.756+0000 7f5034b9b700 1 -- 172.21.15.70:0/3176488352 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6b9e97c60 con 0x55c6ba1315d0 2023-10-18T12:34:36.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.757+0000 7f503539c700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 0x55c6ba139190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:34:36.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.757+0000 7f502ffff700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba136be0 0x55c6ba12b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:34:36.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.761+0000 7f5034b9b700 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba141ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50280cca30 tx=0x7f5028098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcb13a7f3c61881 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.762+0000 7f50177fe700 1 -- 172.21.15.70:0/3176488352 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502809c350 con 0x55c6ba1315d0 2023-10-18T12:34:36.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.762+0000 7f50177fe700 1 -- 172.21.15.70:0/3176488352 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50280b2070 con 0x55c6ba1315d0 2023-10-18T12:34:36.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.762+0000 7f50177fe700 1 -- 172.21.15.70:0/3176488352 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502809c350 con 0x55c6ba1315d0 2023-10-18T12:34:36.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.763+0000 7f504a71cb80 1 -- 172.21.15.70:0/3176488352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 msgr2=0x55c6ba141ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.763+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba141ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50280cca30 tx=0x7f5028098040 comp rx=0 tx=0).stop 2023-10-18T12:34:36.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.764+0000 7f504a71cb80 1 -- 172.21.15.70:0/3176488352 shutdown_connections 2023-10-18T12:34:36.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.764+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba137500 0x55c6ba139190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.764+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba1315d0 0x55c6ba141ee0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.764+0000 7f504a71cb80 1 --2- 172.21.15.70:0/3176488352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba136be0 0x55c6ba12b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.764+0000 7f504a71cb80 1 -- 172.21.15.70:0/3176488352 >> 172.21.15.70:0/3176488352 conn(0x55c6ba0a5720 msgr2=0x55c6ba139e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.765+0000 7f504a71cb80 1 -- 172.21.15.70:0/3176488352 shutdown_connections 2023-10-18T12:34:36.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.765+0000 7f504a71cb80 1 -- 172.21.15.70:0/3176488352 wait complete. 2023-10-18T12:34:36.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.765+0000 7f504a71cb80 1 -- start start 2023-10-18T12:34:36.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.766+0000 7f504a71cb80 1 Processor -- start 2023-10-18T12:34:36.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.766+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba1315d0 0x55c6ba1400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 0x55c6ba1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f504a71cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 0x55c6ba1aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f5034b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba1315d0 0x55c6ba1400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f504a71cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba1af360 con 0x55c6ba137500 2023-10-18T12:34:36.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f504a71cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba1af4a0 con 0x55c6ba1315d0 2023-10-18T12:34:36.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f504a71cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ba1af5e0 con 0x55c6ba136be0 2023-10-18T12:34:36.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f504a71cb80 10 client.0 _ll_register_callbacks cb 0x7f5020011580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:34:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f502ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 0x55c6ba1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f502ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 0x55c6ba1405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:38038/0 (socket says 172.21.15.70:38038) 2023-10-18T12:34:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.768+0000 7f502ffff700 1 -- 172.21.15.70:0/178790062 learned_addr learned my addr 172.21.15.70:0/178790062 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:34:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.769+0000 7f502ffff700 1 -- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba1315d0 msgr2=0x55c6ba1400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.769+0000 7f502ffff700 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba1315d0 0x55c6ba1400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.769+0000 7f502ffff700 1 -- 172.21.15.70:0/178790062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 msgr2=0x55c6ba1aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:34:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.769+0000 7f502ffff700 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 0x55c6ba1aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.769+0000 7f502ffff700 1 -- 172.21.15.70:0/178790062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6ba0413f0 con 0x55c6ba136be0 2023-10-18T12:34:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.770+0000 7f500ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:34:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.770+0000 7f502ffff700 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 0x55c6ba1405f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5020028530 tx=0x7f5020028750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2081498c8e02ccc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:34:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.770+0000 7f500ffff700 1 -- 172.21.15.70:0/178790062 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020038070 con 0x55c6ba136be0 2023-10-18T12:34:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.770+0000 7f500ffff700 1 -- 172.21.15.70:0/178790062 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5020017040 con 0x55c6ba136be0 2023-10-18T12:34:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.771+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50240044e0 con 0x55c6ba136be0 2023-10-18T12:34:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.771+0000 7f500ffff700 1 -- 172.21.15.70:0/178790062 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5020028b80 con 0x55c6ba136be0 2023-10-18T12:34:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.771+0000 7f504a71cb80 10 client.26806 fetch_fsmap learned FSMap version 20 2023-10-18T12:34:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.771+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f50301a9f10 con 0x55c6ba136be0 2023-10-18T12:34:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.772+0000 7f500ffff700 1 -- 172.21.15.70:0/178790062 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50240044e0 con 0x55c6ba136be0 2023-10-18T12:34:36.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f500ffff700 1 -- 172.21.15.70:0/178790062 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5020032500 con 0x55c6ba136be0 2023-10-18T12:34:36.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f504a71cb80 10 client.26806 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:34:36.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f504a71cb80 10 client.26806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:34:36.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f504a71cb80 20 client.26806 populate_metadata read hostname 'smithi070' 2023-10-18T12:34:36.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f504a71cb80 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba1b0ca0 0x55c6ba1b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:34:36.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f504a71cb80 4 client.26806 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:34:36.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.773+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6ba07ce50 con 0x55c6ba1b0ca0 2023-10-18T12:34:36.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.779+0000 7f5034b9b700 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba1b0ca0 0x55c6ba1b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:34:36.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.782+0000 7f500ffff700 10 client.26806 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.782+0000 7f5034b9b700 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba1b0ca0 0x55c6ba1b30c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171a9d8acc500488 server_cookie=79eb70f0b8dbbdf8 in_seq=0 out_seq=0 2023-10-18T12:34:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.785+0000 7f500ffff700 1 -- 172.21.15.70:0/178790062 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c6ba07ce50 con 0x55c6ba1b0ca0 2023-10-18T12:34:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.785+0000 7f500ffff700 10 client.26806 handle_command_reply: tid=0 2023-10-18T12:34:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.785+0000 7f504a71cb80 1 client.26806 shutdown 2023-10-18T12:34:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 msgr2=0x55c6ba1405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 0x55c6ba1405f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5020028530 tx=0x7f5020028750 comp rx=0 tx=0).stop 2023-10-18T12:34:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 shutdown_connections 2023-10-18T12:34:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c6ba1b0ca0 0x55c6ba1b30c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6ba137500 0x55c6ba1aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6ba1315d0 0x55c6ba1400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 --2- 172.21.15.70:0/178790062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6ba136be0 0x55c6ba1405f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.786+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 >> 172.21.15.70:0/178790062 conn(0x55c6ba0a5720 msgr2=0x55c6ba139120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.787+0000 7f500ffff700 0 client.26806 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:34:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.787+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 shutdown_connections 2023-10-18T12:34:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.787+0000 7f504a71cb80 1 -- 172.21.15.70:0/178790062 wait complete. 2023-10-18T12:34:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.787+0000 7f504a71cb80 20 client.26806 trim_cache size 0 max 16384 2023-10-18T12:34:36.792 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:34:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:34:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:34:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "806215bc-7b89-4001-a342-a4a49191e3ba": { 2023-10-18T12:34:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:34:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "806215bc-7b89-4001-a342-a4a49191e3ba", 2023-10-18T12:34:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:34:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:34:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:34:36.794 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:34:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f50180a15f0 msgr2=0x7f50180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f50180a15f0 0x7f50180a3ab0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f503010b2f0 tx=0x7f502000b4a0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 msgr2=0x7f503010caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:34:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f503010caf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5028000f80 tx=0x7f50280079d0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 shutdown_connections 2023-10-18T12:34:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f50180a15f0 0x7f50180a3ab0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f503010d190 0x7f503010a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5030100120 0x7f503010caf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 --2- 172.21.15.70:0/674605435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5030100a40 0x7f5030109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:34:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.790+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 >> 172.21.15.70:0/674605435 conn(0x7f50300767f0 msgr2=0x7f503010e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:34:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.791+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 shutdown_connections 2023-10-18T12:34:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:34:36.791+0000 7f5036dff700 1 -- 172.21.15.70:0/674605435 wait complete. 2023-10-18T12:34:36.830 INFO:tasks.cephfs.filesystem:scrub status for tag:806215bc-7b89-4001-a342-a4a49191e3ba - {'path': '/', 'tag': '806215bc-7b89-4001-a342-a4a49191e3ba', 'options': 'recursive,force'} 2023-10-18T12:34:36.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 967167 11.74 MB/sec execute 305 sec latency 78.313 ms 2023-10-18T12:34:37.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:36 smithi070 ceph-mon[98639]: pgmap v576: 129 pgs: 129 active+clean; 525 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 221 op/s 2023-10-18T12:34:37.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:36 smithi152 ceph-mon[134767]: pgmap v576: 129 pgs: 129 active+clean; 525 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 221 op/s 2023-10-18T12:34:37.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:36 smithi186 ceph-mon[135253]: pgmap v576: 129 pgs: 129 active+clean; 525 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 221 op/s 2023-10-18T12:34:37.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 969297 11.74 MB/sec execute 306 sec latency 37.909 ms 2023-10-18T12:34:38.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:37 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:34:38.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:37 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:38.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:37 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:34:38.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:37 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:38.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:37 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:34:38.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:37 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:38.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:37 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:34:38.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:37 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:38.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:37 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:34:38.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:37 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:38.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:37 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:34:38.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:37 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:34:38.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 971192 11.73 MB/sec execute 307 sec latency 13.404 ms 2023-10-18T12:34:39.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:38 smithi070 ceph-mon[98639]: pgmap v577: 129 pgs: 129 active+clean; 523 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2023-10-18T12:34:39.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:38 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:34:39.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:38 smithi152 ceph-mon[134767]: pgmap v577: 129 pgs: 129 active+clean; 523 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2023-10-18T12:34:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:38 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:34:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:38 smithi186 ceph-mon[135253]: pgmap v577: 129 pgs: 129 active+clean; 523 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2023-10-18T12:34:39.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:38 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:34:39.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 973717 11.75 MB/sec execute 308 sec latency 18.881 ms 2023-10-18T12:34:40.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:39 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:34:40.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:39 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:34:40.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:39 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:34:40.863 INFO:tasks.workunit.client.0.smithi070.stdout: 1 975559 11.73 MB/sec execute 309 sec latency 14.282 ms 2023-10-18T12:34:41.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:40 smithi070 ceph-mon[98639]: pgmap v578: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2023-10-18T12:34:41.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:34:41.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:34:41.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:40 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:34:41.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:40 smithi152 ceph-mon[134767]: pgmap v578: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2023-10-18T12:34:41.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:34:41.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:34:41.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:40 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:34:41.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:40 smithi186 ceph-mon[135253]: pgmap v578: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2023-10-18T12:34:41.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:34:41.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:34:41.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:40 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:34:41.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 977761 11.73 MB/sec execute 310 sec latency 15.160 ms 2023-10-18T12:34:42.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 979472 11.72 MB/sec execute 311 sec latency 65.941 ms 2023-10-18T12:34:43.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:42 smithi070 ceph-mon[98639]: pgmap v579: 129 pgs: 129 active+clean; 499 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 209 op/s 2023-10-18T12:34:43.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:42 smithi152 ceph-mon[134767]: pgmap v579: 129 pgs: 129 active+clean; 499 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 209 op/s 2023-10-18T12:34:43.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:42 smithi186 ceph-mon[135253]: pgmap v579: 129 pgs: 129 active+clean; 499 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 209 op/s 2023-10-18T12:34:43.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 981550 11.72 MB/sec execute 312 sec latency 12.686 ms 2023-10-18T12:34:44.865 INFO:tasks.workunit.client.0.smithi070.stdout: 1 984317 11.74 MB/sec execute 313 sec latency 20.541 ms 2023-10-18T12:34:45.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:44 smithi070 ceph-mon[98639]: pgmap v580: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 232 op/s 2023-10-18T12:34:45.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:44 smithi152 ceph-mon[134767]: pgmap v580: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 232 op/s 2023-10-18T12:34:45.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:44 smithi186 ceph-mon[135253]: pgmap v580: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 232 op/s 2023-10-18T12:34:45.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 986086 11.72 MB/sec execute 314 sec latency 13.059 ms 2023-10-18T12:34:46.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 988373 11.73 MB/sec execute 315 sec latency 16.419 ms 2023-10-18T12:34:47.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:46 smithi070 ceph-mon[98639]: pgmap v581: 129 pgs: 129 active+clean; 504 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 221 op/s 2023-10-18T12:34:47.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:46 smithi152 ceph-mon[134767]: pgmap v581: 129 pgs: 129 active+clean; 504 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 221 op/s 2023-10-18T12:34:47.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:46 smithi186 ceph-mon[135253]: pgmap v581: 129 pgs: 129 active+clean; 504 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 221 op/s 2023-10-18T12:34:47.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 990102 11.71 MB/sec execute 316 sec latency 98.092 ms 2023-10-18T12:34:48.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 992269 11.72 MB/sec execute 317 sec latency 13.150 ms 2023-10-18T12:34:49.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:48 smithi070 ceph-mon[98639]: pgmap v582: 129 pgs: 129 active+clean; 507 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:34:49.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:48 smithi152 ceph-mon[134767]: pgmap v582: 129 pgs: 129 active+clean; 507 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:34:49.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:48 smithi186 ceph-mon[135253]: pgmap v582: 129 pgs: 129 active+clean; 507 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:34:49.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 994934 11.73 MB/sec execute 318 sec latency 26.899 ms 2023-10-18T12:34:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:50 smithi070 ceph-mon[98639]: pgmap v583: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 220 op/s 2023-10-18T12:34:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:50 smithi152 ceph-mon[134767]: pgmap v583: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 220 op/s 2023-10-18T12:34:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:50 smithi186 ceph-mon[135253]: pgmap v583: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 220 op/s 2023-10-18T12:34:50.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 996433 11.72 MB/sec execute 319 sec latency 106.646 ms 2023-10-18T12:34:51.864 INFO:tasks.workunit.client.0.smithi070.stdout: 1 998883 11.72 MB/sec execute 320 sec latency 16.103 ms 2023-10-18T12:34:52.866 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1000290 11.70 MB/sec execute 321 sec latency 61.922 ms 2023-10-18T12:34:53.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:52 smithi070 ceph-mon[98639]: pgmap v584: 129 pgs: 129 active+clean; 535 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 240 op/s 2023-10-18T12:34:53.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:52 smithi152 ceph-mon[134767]: pgmap v584: 129 pgs: 129 active+clean; 535 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 240 op/s 2023-10-18T12:34:53.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:52 smithi186 ceph-mon[135253]: pgmap v584: 129 pgs: 129 active+clean; 535 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 240 op/s 2023-10-18T12:34:53.866 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1002337 11.71 MB/sec execute 322 sec latency 82.130 ms 2023-10-18T12:34:54.866 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1004350 11.70 MB/sec execute 323 sec latency 42.807 ms 2023-10-18T12:34:55.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:54 smithi070 ceph-mon[98639]: pgmap v585: 129 pgs: 129 active+clean; 541 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 250 op/s 2023-10-18T12:34:55.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:54 smithi152 ceph-mon[134767]: pgmap v585: 129 pgs: 129 active+clean; 541 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 250 op/s 2023-10-18T12:34:55.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:54 smithi186 ceph-mon[135253]: pgmap v585: 129 pgs: 129 active+clean; 541 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 250 op/s 2023-10-18T12:34:55.867 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1006281 11.70 MB/sec execute 324 sec latency 43.774 ms 2023-10-18T12:34:56.866 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1009064 11.72 MB/sec execute 325 sec latency 16.361 ms 2023-10-18T12:34:57.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:56 smithi070 ceph-mon[98639]: pgmap v586: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 188 op/s 2023-10-18T12:34:57.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:56 smithi152 ceph-mon[134767]: pgmap v586: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 188 op/s 2023-10-18T12:34:57.225 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:56 smithi186 ceph-mon[135253]: pgmap v586: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 188 op/s 2023-10-18T12:34:57.867 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1010189 11.69 MB/sec execute 326 sec latency 58.401 ms 2023-10-18T12:34:58.867 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1012654 11.70 MB/sec execute 327 sec latency 87.526 ms 2023-10-18T12:34:59.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:34:58 smithi070 ceph-mon[98639]: pgmap v587: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 176 op/s 2023-10-18T12:34:59.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:34:58 smithi152 ceph-mon[134767]: pgmap v587: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 176 op/s 2023-10-18T12:34:59.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:34:58 smithi186 ceph-mon[135253]: pgmap v587: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 176 op/s 2023-10-18T12:34:59.867 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1014201 11.69 MB/sec execute 328 sec latency 16.917 ms 2023-10-18T12:35:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:00 smithi070 ceph-mon[98639]: pgmap v588: 129 pgs: 129 active+clean; 547 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 222 op/s 2023-10-18T12:35:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:00 smithi152 ceph-mon[134767]: pgmap v588: 129 pgs: 129 active+clean; 547 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 222 op/s 2023-10-18T12:35:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:00 smithi186 ceph-mon[135253]: pgmap v588: 129 pgs: 129 active+clean; 547 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 222 op/s 2023-10-18T12:35:00.867 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1015759 11.68 MB/sec execute 329 sec latency 84.878 ms 2023-10-18T12:35:01.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1016784 11.66 MB/sec execute 330 sec latency 65.309 ms 2023-10-18T12:35:02.867 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1018466 11.64 MB/sec execute 331 sec latency 116.352 ms 2023-10-18T12:35:03.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:02 smithi070 ceph-mon[98639]: pgmap v589: 129 pgs: 129 active+clean; 547 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 223 op/s 2023-10-18T12:35:03.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:02 smithi152 ceph-mon[134767]: pgmap v589: 129 pgs: 129 active+clean; 547 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 223 op/s 2023-10-18T12:35:03.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:02 smithi186 ceph-mon[135253]: pgmap v589: 129 pgs: 129 active+clean; 547 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 223 op/s 2023-10-18T12:35:03.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1020308 11.65 MB/sec execute 332 sec latency 92.429 ms 2023-10-18T12:35:04.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1022096 11.63 MB/sec execute 333 sec latency 69.555 ms 2023-10-18T12:35:05.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:04 smithi070 ceph-mon[98639]: pgmap v590: 129 pgs: 129 active+clean; 554 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2023-10-18T12:35:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:04 smithi070 ceph-mon[98639]: osdmap e98: 12 total, 12 up, 12 in 2023-10-18T12:35:05.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:04 smithi152 ceph-mon[134767]: pgmap v590: 129 pgs: 129 active+clean; 554 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2023-10-18T12:35:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:04 smithi152 ceph-mon[134767]: osdmap e98: 12 total, 12 up, 12 in 2023-10-18T12:35:05.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:04 smithi186 ceph-mon[135253]: pgmap v590: 129 pgs: 129 active+clean; 554 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2023-10-18T12:35:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:04 smithi186 ceph-mon[135253]: osdmap e98: 12 total, 12 up, 12 in 2023-10-18T12:35:05.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1023930 11.63 MB/sec execute 334 sec latency 42.137 ms 2023-10-18T12:35:06.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:35:06.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1026490 11.64 MB/sec execute 335 sec latency 16.630 ms 2023-10-18T12:35:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 -- 172.21.15.70:0/3458080792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 msgr2=0x7fece8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 --2- 172.21.15.70:0/3458080792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 0x7fece8106b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fece4009990 tx=0x7fece4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 -- 172.21.15.70:0/3458080792 shutdown_connections 2023-10-18T12:35:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 --2- 172.21.15.70:0/3458080792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece81010e0 0x7fece810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 --2- 172.21.15.70:0/3458080792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 0x7fece8106b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 --2- 172.21.15.70:0/3458080792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece8100720 0x7fece8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.040+0000 7fecefbff700 1 -- 172.21.15.70:0/3458080792 >> 172.21.15.70:0/3458080792 conn(0x7fece80fc1b0 msgr2=0x7fece80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.041+0000 7fecefbff700 1 -- 172.21.15.70:0/3458080792 shutdown_connections 2023-10-18T12:35:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.041+0000 7fecefbff700 1 -- 172.21.15.70:0/3458080792 wait complete. 2023-10-18T12:35:07.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 Processor -- start 2023-10-18T12:35:07.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 -- start start 2023-10-18T12:35:07.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 0x7fece8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece81010e0 0x7fece81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 0x7fece819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece819dc90 con 0x7fece81010e0 2023-10-18T12:35:07.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece819de00 con 0x7fece8106730 2023-10-18T12:35:07.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecefbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece819dfc0 con 0x7fece8100720 2023-10-18T12:35:07.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7feced99b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 0x7fece8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7feced99b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 0x7fece8198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41962/0 (socket says 172.21.15.70:41962) 2023-10-18T12:35:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7feced99b700 1 -- 172.21.15.70:0/1083201569 learned_addr learned my addr 172.21.15.70:0/1083201569 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7feced19a700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece81010e0 0x7fece81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.042+0000 7fecee19c700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 0x7fece819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7feced99b700 1 -- 172.21.15.70:0/1083201569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 msgr2=0x7fece819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7feced99b700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 0x7fece819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7feced99b700 1 -- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece81010e0 msgr2=0x7fece81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7feced99b700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece81010e0 0x7fece81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7feced99b700 1 -- 172.21.15.70:0/1083201569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40095f0 con 0x7fece8100720 2023-10-18T12:35:07.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7feced99b700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 0x7fece8198d70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fece4005910 tx=0x7fece4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7fecdeffd700 1 -- 172.21.15.70:0/1083201569 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece401d070 con 0x7fece8100720 2023-10-18T12:35:07.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece819e240 con 0x7fece8100720 2023-10-18T12:35:07.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7fecdeffd700 1 -- 172.21.15.70:0/1083201569 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece4022470 con 0x7fece8100720 2023-10-18T12:35:07.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.043+0000 7fecdeffd700 1 -- 172.21.15.70:0/1083201569 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4017650 con 0x7fece8100720 2023-10-18T12:35:07.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.044+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece81a5ae0 con 0x7fece8100720 2023-10-18T12:35:07.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.046+0000 7fecdeffd700 1 -- 172.21.15.70:0/1083201569 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fece40225e0 con 0x7fece8100720 2023-10-18T12:35:07.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 Processor -- start 2023-10-18T12:35:07.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fecdeffd700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fecd40a15f0 0x7fecd40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 -- start start 2023-10-18T12:35:07.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fecdeffd700 1 -- 172.21.15.70:0/1083201569 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fece40ce730 con 0x7fece8100720 2023-10-18T12:35:07.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fecee19c700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fecd40a15f0 0x7fecd40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 0x55bb6a363fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a364500 0x55bb6a35e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a35ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a2d16e0 con 0x55bb6a364500 2023-10-18T12:35:07.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a2b7c60 con 0x55bb6a363be0 2023-10-18T12:35:07.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7fed034d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a2b5580 con 0x55bb6a35e5d0 2023-10-18T12:35:07.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.048+0000 7feced99b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 0x55bb6a363fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 0x55bb6a363fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47836/0 (socket says 172.21.15.70:47836) 2023-10-18T12:35:07.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 -- 172.21.15.70:0/3263266851 learned_addr learned my addr 172.21.15.70:0/3263266851 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:07.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7fecee19c700 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a35ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced19a700 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a364500 0x55bb6a35e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 -- 172.21.15.70:0/3263266851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 msgr2=0x55bb6a35ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a35ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 -- 172.21.15.70:0/3263266851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a364500 msgr2=0x55bb6a35e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a364500 0x55bb6a35e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 -- 172.21.15.70:0/3263266851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6a26e3f0 con 0x55bb6a363be0 2023-10-18T12:35:07.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced19a700 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a364500 0x55bb6a35e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:07.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7fecee19c700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fecd40a15f0 0x7fecd40a3ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fece000f870 tx=0x7fece000fc30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7feced99b700 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 0x55bb6a363fc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fece40a3280 tx=0x7fece4022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b490d2124d5ef741 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7fecd3fff700 1 -- 172.21.15.70:0/3263266851 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece409bbb0 con 0x55bb6a363be0 2023-10-18T12:35:07.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7fecd3fff700 1 -- 172.21.15.70:0/3263266851 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece409b4b0 con 0x55bb6a363be0 2023-10-18T12:35:07.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.049+0000 7fecd3fff700 1 -- 172.21.15.70:0/3263266851 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece40ce530 con 0x55bb6a363be0 2023-10-18T12:35:07.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 -- 172.21.15.70:0/3263266851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 msgr2=0x55bb6a363fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 0x55bb6a363fc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fece40a3280 tx=0x7fece4022f90 comp rx=0 tx=0).stop 2023-10-18T12:35:07.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 -- 172.21.15.70:0/3263266851 shutdown_connections 2023-10-18T12:35:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a35ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a363be0 0x55bb6a363fc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3263266851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a364500 0x55bb6a35e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 -- 172.21.15.70:0/3263266851 >> 172.21.15.70:0/3263266851 conn(0x55bb6a2d2720 msgr2=0x55bb6a358b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 -- 172.21.15.70:0/3263266851 shutdown_connections 2023-10-18T12:35:07.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.050+0000 7fed034d3b80 1 -- 172.21.15.70:0/3263266851 wait complete. 2023-10-18T12:35:07.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 -- start start 2023-10-18T12:35:07.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 Processor -- start 2023-10-18T12:35:07.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a3676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a364500 0x55bb6a367c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a2b0e70 con 0x55bb6a363be0 2023-10-18T12:35:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a24d710 con 0x55bb6a364500 2023-10-18T12:35:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a24d3b0 con 0x55bb6a35e5d0 2023-10-18T12:35:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.051+0000 7fed034d3b80 10 client.0 _ll_register_callbacks cb 0x55bb6a197a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a3676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a3676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58342/0 (socket says 172.21.15.70:58342) 2023-10-18T12:35:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 -- 172.21.15.70:0/4192156646 learned_addr learned my addr 172.21.15.70:0/4192156646 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7fecee19c700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a364500 0x55bb6a367c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced99b700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 -- 172.21.15.70:0/4192156646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 msgr2=0x55bb6a36a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 -- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a364500 msgr2=0x55bb6a367c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a364500 0x55bb6a367c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 -- 172.21.15.70:0/4192156646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece81a5ae0 con 0x55bb6a363be0 2023-10-18T12:35:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7feced19a700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a3676d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fecd80126d0 tx=0x7fecd80129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9d817d5ccb5aeb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7fecd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:35:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.052+0000 7fecd17fa700 1 -- 172.21.15.70:0/4192156646 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8017070 con 0x55bb6a363be0 2023-10-18T12:35:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fecd17fa700 1 -- 172.21.15.70:0/4192156646 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fecd8004050 con 0x55bb6a363be0 2023-10-18T12:35:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb6a2b5ef0 con 0x55bb6a363be0 2023-10-18T12:35:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fecd17fa700 1 -- 172.21.15.70:0/4192156646 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd8004210 con 0x55bb6a363be0 2023-10-18T12:35:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 10 client.16776 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bb6a0dbde0 con 0x55bb6a363be0 2023-10-18T12:35:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fecd17fa700 1 -- 172.21.15.70:0/4192156646 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb6a2b5ef0 con 0x55bb6a363be0 2023-10-18T12:35:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fecd17fa700 1 -- 172.21.15.70:0/4192156646 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fecd8010430 con 0x55bb6a363be0 2023-10-18T12:35:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 10 client.16776 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 20 client.16776 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a377b90 0x55bb6a379fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 4 client.16776 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.053+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb6a32d8b0 con 0x55bb6a377b90 2023-10-18T12:35:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.054+0000 7feced99b700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a377b90 0x55bb6a379fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.054+0000 7fecd17fa700 10 client.16776 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:07.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.054+0000 7feced99b700 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a377b90 0x55bb6a379fb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1ba3f278e32e31 server_cookie=72a66059c767a5dd in_seq=0 out_seq=0 2023-10-18T12:35:07.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fecd17fa700 1 -- 172.21.15.70:0/4192156646 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bb6a32d8b0 con 0x55bb6a377b90 2023-10-18T12:35:07.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fecd17fa700 10 client.16776 handle_command_reply: tid=0 2023-10-18T12:35:07.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 client.16776 shutdown 2023-10-18T12:35:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 msgr2=0x55bb6a3676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a3676d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fecd80126d0 tx=0x7fecd80129e0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 shutdown_connections 2023-10-18T12:35:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a377b90 0x55bb6a379fb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a364500 0x55bb6a367c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fecd17fa700 0 client.16776 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 --2- 172.21.15.70:0/4192156646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a3676d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 >> 172.21.15.70:0/4192156646 conn(0x55bb6a2d2720 msgr2=0x55bb6a35d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 shutdown_connections 2023-10-18T12:35:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.055+0000 7fed034d3b80 1 -- 172.21.15.70:0/4192156646 wait complete. 2023-10-18T12:35:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.056+0000 7fed034d3b80 20 client.16776 trim_cache size 0 max 16384 2023-10-18T12:35:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 Processor -- start 2023-10-18T12:35:07.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 -- start start 2023-10-18T12:35:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a365fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a366500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a353af0 con 0x55bb6a363be0 2023-10-18T12:35:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a131920 con 0x55bb6a35e5d0 2023-10-18T12:35:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7fed034d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a35f030 con 0x55bb6a364500 2023-10-18T12:35:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7feced19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a365fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7feced99b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.073+0000 7feced19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a365fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58344/0 (socket says 172.21.15.70:58344) 2023-10-18T12:35:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7fecee19c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a366500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced99b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:47852/0 (socket says 172.21.15.70:47852) 2023-10-18T12:35:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 -- 172.21.15.70:0/2369092581 learned_addr learned my addr 172.21.15.70:0/2369092581 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 -- 172.21.15.70:0/2369092581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 msgr2=0x55bb6a366500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a366500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 -- 172.21.15.70:0/2369092581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 msgr2=0x55bb6a36ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 -- 172.21.15.70:0/2369092581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6a0c4c60 con 0x55bb6a363be0 2023-10-18T12:35:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced99b700 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7feced19a700 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a365fc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fecd8010840 tx=0x7fecd8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb6e4eaa4032e82 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7fecdcbff700 1 -- 172.21.15.70:0/2369092581 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8017040 con 0x55bb6a363be0 2023-10-18T12:35:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7fecdcbff700 1 -- 172.21.15.70:0/2369092581 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecd8012d30 con 0x55bb6a363be0 2023-10-18T12:35:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.074+0000 7fecdcbff700 1 -- 172.21.15.70:0/2369092581 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd80038e0 con 0x55bb6a363be0 2023-10-18T12:35:07.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 -- 172.21.15.70:0/2369092581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 msgr2=0x55bb6a365fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a365fc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fecd8010840 tx=0x7fecd8009f30 comp rx=0 tx=0).stop 2023-10-18T12:35:07.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 -- 172.21.15.70:0/2369092581 shutdown_connections 2023-10-18T12:35:07.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a366500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 --2- 172.21.15.70:0/2369092581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a365fc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 -- 172.21.15.70:0/2369092581 >> 172.21.15.70:0/2369092581 conn(0x55bb6a2d2720 msgr2=0x55bb6a35d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 -- 172.21.15.70:0/2369092581 shutdown_connections 2023-10-18T12:35:07.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 -- 172.21.15.70:0/2369092581 wait complete. 2023-10-18T12:35:07.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.075+0000 7fed034d3b80 1 -- start start 2023-10-18T12:35:07.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 Processor -- start 2023-10-18T12:35:07.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a36cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a36d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a36d920 con 0x55bb6a363be0 2023-10-18T12:35:07.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a36da90 con 0x55bb6a35e5d0 2023-10-18T12:35:07.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a36dc50 con 0x55bb6a364500 2023-10-18T12:35:07.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fed034d3b80 10 client.0 _ll_register_callbacks cb 0x55bb6a197a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:07.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fecee19c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a36d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7feced19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a36cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fecee19c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a36d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41994/0 (socket says 172.21.15.70:41994) 2023-10-18T12:35:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.076+0000 7fecee19c700 1 -- 172.21.15.70:0/3574936023 learned_addr learned my addr 172.21.15.70:0/3574936023 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecee19c700 1 -- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 msgr2=0x55bb6a36e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecee19c700 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecee19c700 1 -- 172.21.15.70:0/3574936023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 msgr2=0x55bb6a36cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecee19c700 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a36cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecee19c700 1 -- 172.21.15.70:0/3574936023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb6a26e3f0 con 0x55bb6a364500 2023-10-18T12:35:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecee19c700 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a36d3e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fece0018e00 tx=0x7fece0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0bfbbdb74910690 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:35:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecd17fa700 1 -- 172.21.15.70:0/3574936023 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece000e040 con 0x55bb6a364500 2023-10-18T12:35:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fece409b4b0 con 0x55bb6a364500 2023-10-18T12:35:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.077+0000 7fecd17fa700 1 -- 172.21.15.70:0/3574936023 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fece0004db0 con 0x55bb6a364500 2023-10-18T12:35:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fecd17fa700 1 -- 172.21.15.70:0/3574936023 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fece00093f0 con 0x55bb6a364500 2023-10-18T12:35:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 10 client.26824 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bb6a0dbde0 con 0x55bb6a364500 2023-10-18T12:35:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fecd17fa700 1 -- 172.21.15.70:0/3574936023 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece409b4b0 con 0x55bb6a364500 2023-10-18T12:35:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fecd17fa700 1 -- 172.21.15.70:0/3574936023 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fece001aa90 con 0x55bb6a364500 2023-10-18T12:35:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 10 client.26824 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 20 client.26824 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a3ccee0 0x55bb6a3cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 4 client.26824 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:35:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb6a2a9e50 con 0x55bb6a3ccee0 2023-10-18T12:35:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.078+0000 7feced99b700 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a3ccee0 0x55bb6a3cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7feced99b700 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a3ccee0 0x55bb6a3cf300 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4ece472c59589b server_cookie=2086a15d4b7de08b in_seq=0 out_seq=0 2023-10-18T12:35:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fecd17fa700 10 client.26824 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fecd17fa700 1 -- 172.21.15.70:0/3574936023 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bb6a2a9e50 con 0x55bb6a3ccee0 2023-10-18T12:35:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fecd17fa700 10 client.26824 handle_command_reply: tid=0 2023-10-18T12:35:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 client.26824 shutdown 2023-10-18T12:35:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 msgr2=0x55bb6a36d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a36d3e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fece0018e00 tx=0x7fece0004b40 comp rx=0 tx=0).stop 2023-10-18T12:35:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 shutdown_connections 2023-10-18T12:35:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55bb6a3ccee0 0x55bb6a3cf300 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb6a364500 0x55bb6a36d3e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bb6a35e5d0 0x55bb6a36e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fecd17fa700 0 client.26824 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 --2- 172.21.15.70:0/3574936023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bb6a363be0 0x55bb6a36cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 >> 172.21.15.70:0/3574936023 conn(0x55bb6a2d2720 msgr2=0x55bb6a3616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 shutdown_connections 2023-10-18T12:35:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.079+0000 7fed034d3b80 1 -- 172.21.15.70:0/3574936023 wait complete. 2023-10-18T12:35:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fed034d3b80 20 client.26824 trim_cache size 0 max 16384 2023-10-18T12:35:07.097 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:35:07.097 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:35:07.097 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:35:07.097 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:35:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fecd40a15f0 msgr2=0x7fecd40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fecd40a15f0 0x7fecd40a3ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fece000f870 tx=0x7fece000fc30 comp rx=0 tx=0).stop 2023-10-18T12:35:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 msgr2=0x7fece8198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 0x7fece8198d70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fece4005910 tx=0x7fece4004730 comp rx=0 tx=0).stop 2023-10-18T12:35:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 shutdown_connections 2023-10-18T12:35:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fecd40a15f0 0x7fecd40a3ab0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fece8106730 0x7fece819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece8100720 0x7fece8198d70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 --2- 172.21.15.70:0/1083201569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fece81010e0 0x7fece81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 >> 172.21.15.70:0/1083201569 conn(0x7fece80fc1b0 msgr2=0x7fece810be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 shutdown_connections 2023-10-18T12:35:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.080+0000 7fecefbff700 1 -- 172.21.15.70:0/1083201569 wait complete. 2023-10-18T12:35:07.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:06 smithi070 ceph-mon[98639]: pgmap v592: 129 pgs: 129 active+clean; 553 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2023-10-18T12:35:07.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:35:07.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:35:07.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:06 smithi152 ceph-mon[134767]: pgmap v592: 129 pgs: 129 active+clean; 553 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2023-10-18T12:35:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:06 smithi186 ceph-mon[135253]: pgmap v592: 129 pgs: 129 active+clean; 553 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2023-10-18T12:35:07.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 -- 172.21.15.70:0/3390090551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 msgr2=0x7feb00113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 --2- 172.21.15.70:0/3390090551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 0x7feb00113aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009f90 tx=0x7feaf800b330 comp rx=0 tx=0).stop 2023-10-18T12:35:07.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 -- 172.21.15.70:0/3390090551 shutdown_connections 2023-10-18T12:35:07.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 --2- 172.21.15.70:0/3390090551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 0x7feb00113aa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 --2- 172.21.15.70:0/3390090551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb00072b90 0x7feb00072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 --2- 172.21.15.70:0/3390090551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb000734b0 0x7feb00111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 -- 172.21.15.70:0/3390090551 >> 172.21.15.70:0/3390090551 conn(0x7feb000fc1d0 msgr2=0x7feb000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 -- 172.21.15.70:0/3390090551 shutdown_connections 2023-10-18T12:35:07.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.350+0000 7feb069ff700 1 -- 172.21.15.70:0/3390090551 wait complete. 2023-10-18T12:35:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.351+0000 7feb069ff700 1 Processor -- start 2023-10-18T12:35:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.351+0000 7feb069ff700 1 -- start start 2023-10-18T12:35:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.351+0000 7feb069ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 0x7feb0019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feb069ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb000734b0 0x7feb0019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feb069ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 0x7feb001a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feb069ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb001a2220 con 0x7feb001116b0 2023-10-18T12:35:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feb069ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb001a2390 con 0x7feb000734b0 2023-10-18T12:35:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feb069ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb001a2550 con 0x7feb00072b90 2023-10-18T12:35:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 0x7feb0019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 0x7feb0019d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42012/0 (socket says 172.21.15.70:42012) 2023-10-18T12:35:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 -- 172.21.15.70:0/1230355573 learned_addr learned my addr 172.21.15.70:0/1230355573 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feaff7fe700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb000734b0 0x7feb0019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feb04f9c700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 0x7feb001a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 -- 172.21.15.70:0/1230355573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb000734b0 msgr2=0x7feb0019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb000734b0 0x7feb0019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 -- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 msgr2=0x7feb001a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 0x7feb001a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feafffff700 1 -- 172.21.15.70:0/1230355573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00095f0 con 0x7feb00072b90 2023-10-18T12:35:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.352+0000 7feaff7fe700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb000734b0 0x7feb0019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.353+0000 7feafffff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 0x7feb0019d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feaf000ec10 tx=0x7feaf000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.353+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000cd40 con 0x7feb00072b90 2023-10-18T12:35:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.353+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaf00044e0 con 0x7feb00072b90 2023-10-18T12:35:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.353+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf8009c40 con 0x7feb00072b90 2023-10-18T12:35:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.353+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0010690 con 0x7feb00072b90 2023-10-18T12:35:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.353+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb001a2a20 con 0x7feb00072b90 2023-10-18T12:35:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.354+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb00103bd0 con 0x7feb00072b90 2023-10-18T12:35:07.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.357+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7feaf0003710 con 0x7feb00072b90 2023-10-18T12:35:07.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.359+0000 7feafd7fa700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feae80a1670 0x7feae80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:07.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.359+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7feaf0014070 con 0x7feb00072b90 2023-10-18T12:35:07.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.359+0000 7feaff7fe700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feae80a1670 0x7feae80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:07.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.360+0000 7feaff7fe700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feae80a1670 0x7feae80a3b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb0019e850 tx=0x7feaf4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:07.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.363+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feaf0096940 con 0x7feb00072b90 2023-10-18T12:35:07.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.699+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb000742d0 con 0x7feb00072b90 2023-10-18T12:35:07.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.700+0000 7feafd7fa700 1 -- 172.21.15.70:0/1230355573 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7feaf0099f60 con 0x7feb00072b90 2023-10-18T12:35:07.702 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:35:07.702 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:35:07.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feae80a1670 msgr2=0x7feae80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feae80a1670 0x7feae80a3b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb0019e850 tx=0x7feaf4009260 comp rx=0 tx=0).stop 2023-10-18T12:35:07.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 msgr2=0x7feb0019d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:07.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 0x7feb0019d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feaf000ec10 tx=0x7feaf000ef20 comp rx=0 tx=0).stop 2023-10-18T12:35:07.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 shutdown_connections 2023-10-18T12:35:07.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb001116b0 0x7feb001a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb00072b90 0x7feb0019d300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7feae80a1670 0x7feae80a3b30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 --2- 172.21.15.70:0/1230355573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb000734b0 0x7feb0019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:07.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.702+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 >> 172.21.15.70:0/1230355573 conn(0x7feb000fc1d0 msgr2=0x7feb000fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:07.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.703+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 shutdown_connections 2023-10-18T12:35:07.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:07.703+0000 7feb069ff700 1 -- 172.21.15.70:0/1230355573 wait complete. 2023-10-18T12:35:07.707 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:35:07.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:35:07.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1027752 11.63 MB/sec execute 336 sec latency 69.843 ms 2023-10-18T12:35:08.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/1669033111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78103310 msgr2=0x7f2b7810f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/1669033111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78103310 0x7f2b7810f820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000b0a0 tx=0x7f2b7000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/1669033111 shutdown_connections 2023-10-18T12:35:08.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/1669033111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78103310 0x7f2b7810f820 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/1669033111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78108960 0x7f2b78108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/1669033111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78102950 0x7f2b78102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/1669033111 >> 172.21.15.70:0/1669033111 conn(0x7f2b780fe400 msgr2=0x7f2b78100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:08.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/1669033111 shutdown_connections 2023-10-18T12:35:08.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.020+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/1669033111 wait complete. 2023-10-18T12:35:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.021+0000 7f2b7d7ff700 1 Processor -- start 2023-10-18T12:35:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.021+0000 7f2b7d7ff700 1 -- start start 2023-10-18T12:35:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b7d7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 0x7f2b78198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b7d7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78103310 0x7f2b78199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b7d7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78108960 0x7f2b7819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b7d7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7819dd20 con 0x7f2b78108960 2023-10-18T12:35:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b7d7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7819de90 con 0x7f2b78102950 2023-10-18T12:35:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b7d7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7819e050 con 0x7f2b78103310 2023-10-18T12:35:08.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 0x7f2b78198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 0x7f2b78198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58598/0 (socket says 172.21.15.70:58598) 2023-10-18T12:35:08.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 -- 172.21.15.70:0/552448391 learned_addr learned my addr 172.21.15.70:0/552448391 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:08.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b777fe700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78108960 0x7f2b7819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b767fc700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78103310 0x7f2b78199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 -- 172.21.15.70:0/552448391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78103310 msgr2=0x7f2b78199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78103310 0x7f2b78199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 -- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78108960 msgr2=0x7f2b7819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78108960 0x7f2b7819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b76ffd700 1 -- 172.21.15.70:0/552448391 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68009520 con 0x7f2b78102950 2023-10-18T12:35:08.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.022+0000 7f2b767fc700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78103310 0x7f2b78199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:08.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.023+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 0x7f2b78198e50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800eb30 tx=0x7f2b6800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:08.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.023+0000 7f2b5ffff700 1 -- 172.21.15.70:0/552448391 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68009880 con 0x7f2b78102950 2023-10-18T12:35:08.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.023+0000 7f2b5ffff700 1 -- 172.21.15.70:0/552448391 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b680044e0 con 0x7f2b78102950 2023-10-18T12:35:08.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.023+0000 7f2b5ffff700 1 -- 172.21.15.70:0/552448391 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68003e90 con 0x7f2b78102950 2023-10-18T12:35:08.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.023+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b70009d00 con 0x7f2b78102950 2023-10-18T12:35:08.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.023+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b781a5990 con 0x7f2b78102950 2023-10-18T12:35:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.026+0000 7f2b5ffff700 1 -- 172.21.15.70:0/552448391 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2b68004000 con 0x7f2b78102950 2023-10-18T12:35:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.027+0000 7f2b911deb80 1 Processor -- start 2023-10-18T12:35:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.027+0000 7f2b911deb80 1 -- start start 2023-10-18T12:35:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b3cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b37a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b911deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80aaa6e0 con 0x564f80b375d0 2023-10-18T12:35:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b911deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80a90c60 con 0x564f80b3d500 2023-10-18T12:35:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b911deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80a8e580 con 0x564f80b3cbe0 2023-10-18T12:35:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b37090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58600/0 (socket says 172.21.15.70:58600) 2023-10-18T12:35:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 -- 172.21.15.70:0/638780972 learned_addr learned my addr 172.21.15.70:0/638780972 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b767fc700 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b37a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b5ffff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b600a1670 0x7f2b600a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b5ffff700 1 -- 172.21.15.70:0/552448391 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f2b68014070 con 0x7f2b78102950 2023-10-18T12:35:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 -- 172.21.15.70:0/638780972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 msgr2=0x564f80b3cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b3cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 -- 172.21.15.70:0/638780972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 msgr2=0x564f80b37a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b37a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b76ffd700 1 -- 172.21.15.70:0/638780972 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f80a473f0 con 0x564f80b3d500 2023-10-18T12:35:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.028+0000 7f2b777fe700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b600a1670 0x7f2b600a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.029+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b37090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2b680cac70 tx=0x7f2b68003930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3883a5c8b3e4845f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.029+0000 7f2b5d7fa700 1 -- 172.21.15.70:0/638780972 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68085070 con 0x564f80b3d500 2023-10-18T12:35:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.029+0000 7f2b5d7fa700 1 -- 172.21.15.70:0/638780972 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b68099ef0 con 0x564f80b3d500 2023-10-18T12:35:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.029+0000 7f2b5d7fa700 1 -- 172.21.15.70:0/638780972 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b680a2920 con 0x564f80b3d500 2023-10-18T12:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 -- 172.21.15.70:0/638780972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 msgr2=0x564f80b37090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b777fe700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b600a1670 0x7f2b600a3b30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009cd0 tx=0x7f2b70009c60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b37090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2b680cac70 tx=0x7f2b68003930 comp rx=0 tx=0).stop 2023-10-18T12:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 -- 172.21.15.70:0/638780972 shutdown_connections 2023-10-18T12:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b37a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b3cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 --2- 172.21.15.70:0/638780972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b37090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 -- 172.21.15.70:0/638780972 >> 172.21.15.70:0/638780972 conn(0x564f80aab720 msgr2=0x564f80b31b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 -- 172.21.15.70:0/638780972 shutdown_connections 2023-10-18T12:35:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 -- 172.21.15.70:0/638780972 wait complete. 2023-10-18T12:35:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 -- start start 2023-10-18T12:35:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.030+0000 7f2b911deb80 1 Processor -- start 2023-10-18T12:35:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b40760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b40d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80a89e70 con 0x564f80b375d0 2023-10-18T12:35:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80a26710 con 0x564f80b3d500 2023-10-18T12:35:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80a263b0 con 0x564f80b3cbe0 2023-10-18T12:35:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b911deb80 10 client.0 _ll_register_callbacks cb 0x564f80970a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b767fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b40760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b767fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b40760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36346/0 (socket says 172.21.15.70:36346) 2023-10-18T12:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b767fc700 1 -- 172.21.15.70:0/1297216009 learned_addr learned my addr 172.21.15.70:0/1297216009 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.031+0000 7f2b777fe700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b40d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b767fc700 1 -- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 msgr2=0x564f80b40d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b767fc700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b40d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b767fc700 1 -- 172.21.15.70:0/1297216009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 msgr2=0x564f80b43420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b767fc700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b43420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b767fc700 1 -- 172.21.15.70:0/1297216009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b781a5990 con 0x564f80b3cbe0 2023-10-18T12:35:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b43420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b767fc700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b40760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0104f0 tx=0x7f2b6c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cbac0729e78426e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b5aff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:35:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b5aff5700 1 -- 172.21.15.70:0/1297216009 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c010e80 con 0x564f80b3cbe0 2023-10-18T12:35:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f80a8eef0 con 0x564f80b3cbe0 2023-10-18T12:35:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b5aff5700 1 -- 172.21.15.70:0/1297216009 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f2b6c014640 con 0x564f80b3cbe0 2023-10-18T12:35:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b5aff5700 1 -- 172.21.15.70:0/1297216009 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b6c014800 con 0x564f80b3cbe0 2023-10-18T12:35:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.032+0000 7f2b911deb80 10 client.26836 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564f808b4de0 con 0x564f80b3cbe0 2023-10-18T12:35:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b5aff5700 1 -- 172.21.15.70:0/1297216009 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f80a8eef0 con 0x564f80b3cbe0 2023-10-18T12:35:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b5aff5700 1 -- 172.21.15.70:0/1297216009 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2b6c013b30 con 0x564f80b3cbe0 2023-10-18T12:35:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 10 client.26836 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 10 client.26836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 20 client.26836 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80b50d90 0x564f80b531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 4 client.26836 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f80b068b0 con 0x564f80b50d90 2023-10-18T12:35:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.033+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80b50d90 0x564f80b531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.034+0000 7f2b5aff5700 10 client.26836 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.034+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80b50d90 0x564f80b531b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49235cd63cbba97a server_cookie=f94f9613cad16fa6 in_seq=0 out_seq=0 2023-10-18T12:35:08.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b5aff5700 1 -- 172.21.15.70:0/1297216009 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f80b068b0 con 0x564f80b50d90 2023-10-18T12:35:08.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b5aff5700 10 client.26836 handle_command_reply: tid=0 2023-10-18T12:35:08.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 client.26836 shutdown 2023-10-18T12:35:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 msgr2=0x564f80b40760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b40760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0104f0 tx=0x7f2b6c010800 comp rx=0 tx=0).stop 2023-10-18T12:35:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 shutdown_connections 2023-10-18T12:35:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80b50d90 0x564f80b531b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80b40d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b43420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b5aff5700 0 client.26836 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 --2- 172.21.15.70:0/1297216009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b40760 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 >> 172.21.15.70:0/1297216009 conn(0x564f80aab720 msgr2=0x564f80b36080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 shutdown_connections 2023-10-18T12:35:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 1 -- 172.21.15.70:0/1297216009 wait complete. 2023-10-18T12:35:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.035+0000 7f2b911deb80 20 client.26836 trim_cache size 0 max 16384 2023-10-18T12:35:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 Processor -- start 2023-10-18T12:35:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 -- start start 2023-10-18T12:35:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3cbe0 0x564f80b31810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3d500 0x564f80b3f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8090a920 con 0x564f80b375d0 2023-10-18T12:35:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80b36700 con 0x564f80b3cbe0 2023-10-18T12:35:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b911deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80b41670 con 0x564f80b3d500 2023-10-18T12:35:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b767fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3cbe0 0x564f80b31810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b47e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45544/0 (socket says 172.21.15.70:45544) 2023-10-18T12:35:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.049+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3610767845 learned_addr learned my addr 172.21.15.70:0/3610767845 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b777fe700 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3d500 0x564f80b3f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3610767845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3d500 msgr2=0x564f80b3f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3d500 0x564f80b3f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3610767845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3cbe0 msgr2=0x564f80b31810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3cbe0 0x564f80b31810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3610767845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f8089dc60 con 0x564f80b375d0 2023-10-18T12:35:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b777fe700 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3d500 0x564f80b3f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:35:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b767fc700 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3cbe0 0x564f80b31810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b47e50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2b68096a90 tx=0x7f2b680962a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d2e50be62c11ed server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b5dffb700 1 -- 172.21.15.70:0/3610767845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800cc90 con 0x564f80b375d0 2023-10-18T12:35:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b5dffb700 1 -- 172.21.15.70:0/3610767845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b680a2920 con 0x564f80b375d0 2023-10-18T12:35:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.050+0000 7f2b5dffb700 1 -- 172.21.15.70:0/3610767845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68098a50 con 0x564f80b375d0 2023-10-18T12:35:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 -- 172.21.15.70:0/3610767845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 msgr2=0x564f80b47e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b47e50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2b68096a90 tx=0x7f2b680962a0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 -- 172.21.15.70:0/3610767845 shutdown_connections 2023-10-18T12:35:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3d500 0x564f80b3f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b47e50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3610767845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3cbe0 0x564f80b31810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 -- 172.21.15.70:0/3610767845 >> 172.21.15.70:0/3610767845 conn(0x564f80aab720 msgr2=0x564f80b36930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 -- 172.21.15.70:0/3610767845 shutdown_connections 2023-10-18T12:35:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 -- 172.21.15.70:0/3610767845 wait complete. 2023-10-18T12:35:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.051+0000 7f2b911deb80 1 -- start start 2023-10-18T12:35:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 Processor -- start 2023-10-18T12:35:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b3baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80ba3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80b3c6f0 con 0x564f80b375d0 2023-10-18T12:35:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80ba4120 con 0x564f80b3d500 2023-10-18T12:35:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80ba4290 con 0x564f80b3cbe0 2023-10-18T12:35:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b911deb80 10 client.0 _ll_register_callbacks cb 0x564f80970a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b3baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b3baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45560/0 (socket says 172.21.15.70:45560) 2023-10-18T12:35:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3443390263 learned_addr learned my addr 172.21.15.70:0/3443390263 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.052+0000 7f2b777fe700 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80ba3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3443390263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 msgr2=0x564f80b3bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 msgr2=0x564f80ba3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80ba3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b76ffd700 1 -- 172.21.15.70:0/3443390263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f808b4de0 con 0x564f80b375d0 2023-10-18T12:35:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b76ffd700 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b3baa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2b6809a060 tx=0x7f2b6809e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c166fe939c0e6802 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b5aff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:35:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b5aff5700 1 -- 172.21.15.70:0/3443390263 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800cc90 con 0x564f80b375d0 2023-10-18T12:35:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b5aff5700 1 -- 172.21.15.70:0/3443390263 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f2b680caf80 con 0x564f80b375d0 2023-10-18T12:35:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f80b38040 con 0x564f80b375d0 2023-10-18T12:35:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b5aff5700 1 -- 172.21.15.70:0/3443390263 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b680cb140 con 0x564f80b375d0 2023-10-18T12:35:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b911deb80 10 client.16806 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.053+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564f80a473f0 con 0x564f80b375d0 2023-10-18T12:35:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b5aff5700 1 -- 172.21.15.70:0/3443390263 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f80b38040 con 0x564f80b375d0 2023-10-18T12:35:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b5aff5700 1 -- 172.21.15.70:0/3443390263 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2b680a2490 con 0x564f80b375d0 2023-10-18T12:35:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b911deb80 10 client.16806 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b911deb80 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b911deb80 20 client.16806 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80ba5590 0x564f80ba7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b911deb80 4 client.16806 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:35:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f2b68099ef0 con 0x564f80ba5590 2023-10-18T12:35:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.054+0000 7f2b767fc700 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80ba5590 0x564f80ba7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:08.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b767fc700 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80ba5590 0x564f80ba7980 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1daf2d2738eabf9 server_cookie=e5eb4e9d5f42ce5b in_seq=0 out_seq=0 2023-10-18T12:35:08.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b5aff5700 10 client.16806 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b5aff5700 1 -- 172.21.15.70:0/3443390263 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f2b68099ef0 con 0x564f80ba5590 2023-10-18T12:35:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b5aff5700 10 client.16806 handle_command_reply: tid=0 2023-10-18T12:35:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 client.16806 shutdown 2023-10-18T12:35:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 msgr2=0x564f80b3baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b3baa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2b6809a060 tx=0x7f2b6809e6e0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 shutdown_connections 2023-10-18T12:35:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564f80ba5590 0x564f80ba7980 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f80b3d500 0x564f80ba3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564f80b375d0 0x564f80b3baa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 --2- 172.21.15.70:0/3443390263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564f80b3cbe0 0x564f80b3bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b5aff5700 0 client.16806 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.055+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 >> 172.21.15.70:0/3443390263 conn(0x564f80aab720 msgr2=0x564f80b31810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 shutdown_connections 2023-10-18T12:35:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b911deb80 1 -- 172.21.15.70:0/3443390263 wait complete. 2023-10-18T12:35:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b911deb80 20 client.16806 trim_cache size 0 max 16384 2023-10-18T12:35:08.076 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:35:08.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b600a1670 msgr2=0x7f2b600a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b600a1670 0x7f2b600a3b30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009cd0 tx=0x7f2b70009c60 comp rx=0 tx=0).stop 2023-10-18T12:35:08.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 msgr2=0x7f2b78198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:08.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 0x7f2b78198e50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800eb30 tx=0x7f2b6800ee40 comp rx=0 tx=0).stop 2023-10-18T12:35:08.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 shutdown_connections 2023-10-18T12:35:08.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b78108960 0x7f2b7819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2b600a1670 0x7f2b600a3b30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b78102950 0x7f2b78198e50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 --2- 172.21.15.70:0/552448391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b78103310 0x7f2b78199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.056+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 >> 172.21.15.70:0/552448391 conn(0x7f2b780fe400 msgr2=0x7f2b780feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.057+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 shutdown_connections 2023-10-18T12:35:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:08.057+0000 7f2b7d7ff700 1 -- 172.21.15.70:0/552448391 wait complete. 2023-10-18T12:35:08.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:07 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1230355573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:35:08.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:07 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1230355573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:35:08.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:07 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1230355573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:35:08.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1030443 11.64 MB/sec execute 337 sec latency 22.968 ms 2023-10-18T12:35:09.081 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:35:09.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:35:09.145 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:08 smithi070 ceph-mon[98639]: pgmap v593: 129 pgs: 129 active+clean; 555 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 250 op/s 2023-10-18T12:35:09.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:08 smithi152 ceph-mon[134767]: pgmap v593: 129 pgs: 129 active+clean; 555 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 250 op/s 2023-10-18T12:35:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:08 smithi186 ceph-mon[135253]: pgmap v593: 129 pgs: 129 active+clean; 555 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 250 op/s 2023-10-18T12:35:09.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 -- 172.21.15.70:0/4118675815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 msgr2=0x7f4d94100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 --2- 172.21.15.70:0/4118675815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d94100ba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009990 tx=0x7f4d88009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 -- 172.21.15.70:0/4118675815 shutdown_connections 2023-10-18T12:35:09.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 --2- 172.21.15.70:0/4118675815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d941010e0 0x7f4d9410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 --2- 172.21.15.70:0/4118675815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94106730 0x7f4d94106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 --2- 172.21.15.70:0/4118675815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d94100ba0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.291+0000 7f4d98dff700 1 -- 172.21.15.70:0/4118675815 >> 172.21.15.70:0/4118675815 conn(0x7f4d940fc1b0 msgr2=0x7f4d940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.292+0000 7f4d98dff700 1 -- 172.21.15.70:0/4118675815 shutdown_connections 2023-10-18T12:35:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.292+0000 7f4d98dff700 1 -- 172.21.15.70:0/4118675815 wait complete. 2023-10-18T12:35:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.293+0000 7f4d98dff700 1 Processor -- start 2023-10-18T12:35:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.293+0000 7f4d98dff700 1 -- start start 2023-10-18T12:35:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d98dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d9419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d98dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d941010e0 0x7f4d9419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d98dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94106730 0x7f4d941969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d98dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d94197030 con 0x7f4d941010e0 2023-10-18T12:35:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d98dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d941971a0 con 0x7f4d94100720 2023-10-18T12:35:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d98dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d94197360 con 0x7f4d94106730 2023-10-18T12:35:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d9419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d9419c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58640/0 (socket says 172.21.15.70:58640) 2023-10-18T12:35:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 -- 172.21.15.70:0/2025399420 learned_addr learned my addr 172.21.15.70:0/2025399420 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d941010e0 0x7f4d9419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94106730 0x7f4d941969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 -- 172.21.15.70:0/2025399420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94106730 msgr2=0x7f4d941969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94106730 0x7f4d941969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 -- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d941010e0 msgr2=0x7f4d9419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d941010e0 0x7f4d9419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.294+0000 7f4d9259c700 1 -- 172.21.15.70:0/2025399420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880095f0 con 0x7f4d94100720 2023-10-18T12:35:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.295+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d9419c8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400e9a0 tx=0x7f4d8400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.295+0000 7f4d837fe700 1 -- 172.21.15.70:0/2025399420 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84009790 con 0x7f4d94100720 2023-10-18T12:35:09.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.295+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d94197640 con 0x7f4d94100720 2023-10-18T12:35:09.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.295+0000 7f4d837fe700 1 -- 172.21.15.70:0/2025399420 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d84010430 con 0x7f4d94100720 2023-10-18T12:35:09.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.295+0000 7f4d837fe700 1 -- 172.21.15.70:0/2025399420 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84018680 con 0x7f4d94100720 2023-10-18T12:35:09.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.295+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941a3890 con 0x7f4d94100720 2023-10-18T12:35:09.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.299+0000 7f4d837fe700 1 -- 172.21.15.70:0/2025399420 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4d840105a0 con 0x7f4d94100720 2023-10-18T12:35:09.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 Processor -- start 2023-10-18T12:35:09.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 -- start start 2023-10-18T12:35:09.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21941760 con 0x55cc219ceb90 2023-10-18T12:35:09.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21927ce0 con 0x55cc219d1cb0 2023-10-18T12:35:09.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21925600 con 0x55cc219d2090 2023-10-18T12:35:09.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.301+0000 7f4d837fe700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d7c0a15f0 0x7f4d7c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36364/0 (socket says 172.21.15.70:36364) 2023-10-18T12:35:09.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 -- 172.21.15.70:0/1012277928 learned_addr learned my addr 172.21.15.70:0/1012277928 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d7c0a15f0 0x7f4d7c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d837fe700 1 -- 172.21.15.70:0/2025399420 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f4d84014070 con 0x7f4d94100720 2023-10-18T12:35:09.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 -- 172.21.15.70:0/1012277928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 msgr2=0x55cc219ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 -- 172.21.15.70:0/1012277928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 msgr2=0x55cc219e2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 -- 172.21.15.70:0/1012277928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc218de470 con 0x55cc219d2090 2023-10-18T12:35:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.302+0000 7f4d9259c700 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219ce650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d84096740 tx=0x7f4d8406d1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74e8f92b031894e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.303+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d7c0a15f0 0x7f4d7c0a3ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009ec0 tx=0x7f4d8c00b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.303+0000 7f4d80ff9700 1 -- 172.21.15.70:0/1012277928 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8409a940 con 0x55cc219d2090 2023-10-18T12:35:09.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.303+0000 7f4d80ff9700 1 -- 172.21.15.70:0/1012277928 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d84065ab0 con 0x55cc219d2090 2023-10-18T12:35:09.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.303+0000 7f4d80ff9700 1 -- 172.21.15.70:0/1012277928 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d840cac30 con 0x55cc219d2090 2023-10-18T12:35:09.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1012277928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 msgr2=0x55cc219ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219ce650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d84096740 tx=0x7f4d8406d1b0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1012277928 shutdown_connections 2023-10-18T12:35:09.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1012277928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219ce650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1012277928 >> 172.21.15.70:0/1012277928 conn(0x55cc219427a0 msgr2=0x55cc219c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1012277928 shutdown_connections 2023-10-18T12:35:09.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.304+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1012277928 wait complete. 2023-10-18T12:35:09.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.305+0000 7f4dac78eb80 1 -- start start 2023-10-18T12:35:09.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.305+0000 7f4dac78eb80 1 Processor -- start 2023-10-18T12:35:09.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21920ef0 con 0x55cc219ceb90 2023-10-18T12:35:09.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc218bd790 con 0x55cc219d1cb0 2023-10-18T12:35:09.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc218bd430 con 0x55cc219d2090 2023-10-18T12:35:09.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4dac78eb80 10 client.0 _ll_register_callbacks cb 0x55cc21807b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:09.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4d91d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4d91d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219e0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58672/0 (socket says 172.21.15.70:58672) 2023-10-18T12:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.306+0000 7f4d91d9b700 1 -- 172.21.15.70:0/2918185768 learned_addr learned my addr 172.21.15.70:0/2918185768 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d91d9b700 1 -- 172.21.15.70:0/2918185768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 msgr2=0x55cc219e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d91d9b700 1 -- 172.21.15.70:0/2918185768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 msgr2=0x55cc219e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d91d9b700 1 -- 172.21.15.70:0/2918185768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941a3890 con 0x55cc219d1cb0 2023-10-18T12:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219e0140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d88015000 tx=0x7f4d88005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd6f62596195ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.307+0000 7f4d79ffb700 1 -- 172.21.15.70:0/2918185768 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8801d070 con 0x55cc219d1cb0 2023-10-18T12:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc21925f70 con 0x55cc219d1cb0 2023-10-18T12:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4d79ffb700 1 -- 172.21.15.70:0/2918185768 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f4d88004500 con 0x55cc219d1cb0 2023-10-18T12:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4d79ffb700 1 -- 172.21.15.70:0/2918185768 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d880058e0 con 0x55cc219d1cb0 2023-10-18T12:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4dac78eb80 10 client.26861 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cc2174be60 con 0x55cc219d1cb0 2023-10-18T12:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4d79ffb700 1 -- 172.21.15.70:0/2918185768 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc21925f70 con 0x55cc219d1cb0 2023-10-18T12:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.308+0000 7f4d79ffb700 1 -- 172.21.15.70:0/2918185768 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4d88024850 con 0x55cc219d1cb0 2023-10-18T12:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4dac78eb80 10 client.26861 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:09.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4dac78eb80 10 client.26861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:09.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4dac78eb80 20 client.26861 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:09.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc219f07d0 0x55cc219f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4dac78eb80 4 client.26861 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:09.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc2199d930 con 0x55cc219f07d0 2023-10-18T12:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc219f07d0 0x55cc219f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4d9259c700 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc219f07d0 0x55cc219f2bf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af37a3aa1ef4c6b server_cookie=112ebd9d628abb4e in_seq=0 out_seq=0 2023-10-18T12:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.309+0000 7f4d79ffb700 10 client.26861 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.310+0000 7f4d79ffb700 1 -- 172.21.15.70:0/2918185768 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc2199d930 con 0x55cc219f07d0 2023-10-18T12:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.310+0000 7f4d79ffb700 10 client.26861 handle_command_reply: tid=0 2023-10-18T12:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.310+0000 7f4dac78eb80 1 client.26861 shutdown 2023-10-18T12:35:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 msgr2=0x55cc219e0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219e0140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d88015000 tx=0x7f4d88005800 comp rx=0 tx=0).stop 2023-10-18T12:35:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 shutdown_connections 2023-10-18T12:35:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc219f07d0 0x55cc219f2bf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d2090 0x55cc219e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/2918185768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d1cb0 0x55cc219e0140 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4d79ffb700 0 client.26861 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 >> 172.21.15.70:0/2918185768 conn(0x55cc219427a0 msgr2=0x55cc219cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 shutdown_connections 2023-10-18T12:35:09.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 1 -- 172.21.15.70:0/2918185768 wait complete. 2023-10-18T12:35:09.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.311+0000 7f4dac78eb80 20 client.26861 trim_cache size 0 max 16384 2023-10-18T12:35:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.331+0000 7f4dac78eb80 1 Processor -- start 2023-10-18T12:35:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 -- start start 2023-10-18T12:35:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 0x55cc219e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219d2090 0x55cc219def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc217a19a0 con 0x55cc219d2090 2023-10-18T12:35:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc219df490 con 0x55cc219ceb90 2023-10-18T12:35:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc219ce9c0 con 0x55cc219d1cb0 2023-10-18T12:35:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 0x55cc219e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 0x55cc219e76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58678/0 (socket says 172.21.15.70:58678) 2023-10-18T12:35:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4d9259c700 1 -- 172.21.15.70:0/831799058 learned_addr learned my addr 172.21.15.70:0/831799058 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.332+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219d2090 0x55cc219def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d9259c700 1 -- 172.21.15.70:0/831799058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 msgr2=0x55cc219dea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d9259c700 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219dea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d9259c700 1 -- 172.21.15.70:0/831799058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219d2090 msgr2=0x55cc219def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d9259c700 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219d2090 0x55cc219def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d9259c700 1 -- 172.21.15.70:0/831799058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc21734ce0 con 0x55cc219ceb90 2023-10-18T12:35:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219d2090 0x55cc219def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:09.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d9259c700 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 0x55cc219e76d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d840caa60 tx=0x7f4d84065c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25a89d3c2b5c55c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d817fa700 1 -- 172.21.15.70:0/831799058 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8409e390 con 0x55cc219ceb90 2023-10-18T12:35:09.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d817fa700 1 -- 172.21.15.70:0/831799058 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d840b1030 con 0x55cc219ceb90 2023-10-18T12:35:09.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.333+0000 7f4d817fa700 1 -- 172.21.15.70:0/831799058 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d840cac30 con 0x55cc219ceb90 2023-10-18T12:35:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 -- 172.21.15.70:0/831799058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 msgr2=0x55cc219e76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 0x55cc219e76d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d840caa60 tx=0x7f4d84065c20 comp rx=0 tx=0).stop 2023-10-18T12:35:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 -- 172.21.15.70:0/831799058 shutdown_connections 2023-10-18T12:35:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219d2090 0x55cc219def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219ceb90 0x55cc219e76d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/831799058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 -- 172.21.15.70:0/831799058 >> 172.21.15.70:0/831799058 conn(0x55cc219427a0 msgr2=0x55cc219db470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 -- 172.21.15.70:0/831799058 shutdown_connections 2023-10-18T12:35:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 -- 172.21.15.70:0/831799058 wait complete. 2023-10-18T12:35:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 -- start start 2023-10-18T12:35:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.334+0000 7f4dac78eb80 1 Processor -- start 2023-10-18T12:35:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 0x55cc219e6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21a44630 con 0x55cc219ceb90 2023-10-18T12:35:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21a44770 con 0x55cc219d2090 2023-10-18T12:35:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21a448b0 con 0x55cc219d1cb0 2023-10-18T12:35:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4dac78eb80 10 client.0 _ll_register_callbacks cb 0x55cc21807b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 0x55cc219e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4d9259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 0x55cc219e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58690/0 (socket says 172.21.15.70:58690) 2023-10-18T12:35:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4d92d9d700 1 -- 172.21.15.70:0/1477432676 learned_addr learned my addr 172.21.15.70:0/1477432676 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.335+0000 7f4d91d9b700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d92d9d700 1 -- 172.21.15.70:0/1477432676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 msgr2=0x55cc219e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d92d9d700 1 -- 172.21.15.70:0/1477432676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 msgr2=0x55cc219e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d92d9d700 1 -- 172.21.15.70:0/1477432676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc218de470 con 0x55cc219d2090 2023-10-18T12:35:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d9259c700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d92d9d700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 0x55cc219e6810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c01ae00 tx=0x7f4d8c009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cfbc0caf496a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d79ffb700 1 -- 172.21.15.70:0/1477432676 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00e070 con 0x55cc219d2090 2023-10-18T12:35:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d84065ab0 con 0x55cc219d2090 2023-10-18T12:35:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.336+0000 7f4d79ffb700 1 -- 172.21.15.70:0/1477432676 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f4d8c01bc50 con 0x55cc219d2090 2023-10-18T12:35:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4d79ffb700 1 -- 172.21.15.70:0/1477432676 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d8c01be10 con 0x55cc219d2090 2023-10-18T12:35:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 10 client.26867 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cc217722d0 con 0x55cc219d2090 2023-10-18T12:35:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4d79ffb700 1 -- 172.21.15.70:0/1477432676 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d84065ab0 con 0x55cc219d2090 2023-10-18T12:35:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4d79ffb700 1 -- 172.21.15.70:0/1477432676 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4d8c0045c0 con 0x55cc219d2090 2023-10-18T12:35:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 10 client.26867 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 20 client.26867 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc21a460b0 0x55cc21a484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 4 client.26867 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:35:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cc21919ed0 con 0x55cc21a460b0 2023-10-18T12:35:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.337+0000 7f4d9259c700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc21a460b0 0x55cc21a484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.338+0000 7f4d79ffb700 10 client.26867 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.338+0000 7f4d9259c700 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc21a460b0 0x55cc21a484d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b6b33cf28a7707 server_cookie=d0f299250a819a83 in_seq=0 out_seq=0 2023-10-18T12:35:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.341+0000 7f4d79ffb700 1 -- 172.21.15.70:0/1477432676 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cc21919ed0 con 0x55cc21a460b0 2023-10-18T12:35:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.341+0000 7f4d79ffb700 10 client.26867 handle_command_reply: tid=0 2023-10-18T12:35:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.341+0000 7f4dac78eb80 1 client.26867 shutdown 2023-10-18T12:35:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 msgr2=0x55cc219e6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 0x55cc219e6810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c01ae00 tx=0x7f4d8c009e50 comp rx=0 tx=0).stop 2023-10-18T12:35:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 shutdown_connections 2023-10-18T12:35:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55cc21a460b0 0x55cc21a484d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc219d2090 0x55cc219e6810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4d79ffb700 0 client.26867 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc219ceb90 0x55cc219e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 --2- 172.21.15.70:0/1477432676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc219d1cb0 0x55cc219e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 >> 172.21.15.70:0/1477432676 conn(0x55cc219427a0 msgr2=0x55cc219db470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 shutdown_connections 2023-10-18T12:35:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 1 -- 172.21.15.70:0/1477432676 wait complete. 2023-10-18T12:35:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4dac78eb80 20 client.26867 trim_cache size 0 max 16384 2023-10-18T12:35:09.355 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:35:09.355 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:35:09.356 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "dcf95a4c-1b02-471c-9db4-b5ef21f4f799", 2023-10-18T12:35:09.356 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:35:09.356 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:35:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d7c0a15f0 msgr2=0x7f4d7c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4d98dff700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d7c0a15f0 0x7f4d7c0a3ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009ec0 tx=0x7f4d8c00b040 comp rx=0 tx=0).stop 2023-10-18T12:35:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 msgr2=0x7f4d9419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.342+0000 7f4d98dff700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d9419c8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400e9a0 tx=0x7f4d8400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 shutdown_connections 2023-10-18T12:35:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94106730 0x7f4d941969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4d7c0a15f0 0x7f4d7c0a3ab0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d94100720 0x7f4d9419c8a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 --2- 172.21.15.70:0/2025399420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d941010e0 0x7f4d9419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 >> 172.21.15.70:0/2025399420 conn(0x7f4d940fc1b0 msgr2=0x7f4d940fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 shutdown_connections 2023-10-18T12:35:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.343+0000 7f4d98dff700 1 -- 172.21.15.70:0/2025399420 wait complete. 2023-10-18T12:35:09.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:35:09.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 -- 172.21.15.70:0/2135496913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c100710 msgr2=0x7f998c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 --2- 172.21.15.70:0/2135496913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c100710 0x7f998c100b90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9980009990 tx=0x7f9980009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 -- 172.21.15.70:0/2135496913 shutdown_connections 2023-10-18T12:35:09.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 --2- 172.21.15.70:0/2135496913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f998c1010d0 0x7f998c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 --2- 172.21.15.70:0/2135496913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c106720 0x7f998c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 --2- 172.21.15.70:0/2135496913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c100710 0x7f998c100b90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 -- 172.21.15.70:0/2135496913 >> 172.21.15.70:0/2135496913 conn(0x7f998c0fc200 msgr2=0x7f998c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.593+0000 7f99923ff700 1 -- 172.21.15.70:0/2135496913 shutdown_connections 2023-10-18T12:35:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.594+0000 7f99923ff700 1 -- 172.21.15.70:0/2135496913 wait complete. 2023-10-18T12:35:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.594+0000 7f99923ff700 1 Processor -- start 2023-10-18T12:35:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 -- start start 2023-10-18T12:35:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 0x7f998c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c1010d0 0x7f998c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f998c106720 0x7f998c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c19dcb0 con 0x7f998c1010d0 2023-10-18T12:35:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c19de20 con 0x7f998c106720 2023-10-18T12:35:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f99923ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c19dfe0 con 0x7f998c100710 2023-10-18T12:35:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f998b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c1010d0 0x7f998c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f998bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 0x7f998c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f998bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 0x7f998c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36428/0 (socket says 172.21.15.70:36428) 2023-10-18T12:35:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f999099c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f998c106720 0x7f998c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.595+0000 7f998bfff700 1 -- 172.21.15.70:0/1513174079 learned_addr learned my addr 172.21.15.70:0/1513174079 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f998bfff700 1 -- 172.21.15.70:0/1513174079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f998c106720 msgr2=0x7f998c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f998bfff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f998c106720 0x7f998c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f998bfff700 1 -- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c1010d0 msgr2=0x7f998c1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f998bfff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c1010d0 0x7f998c1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f998bfff700 1 -- 172.21.15.70:0/1513174079 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800095f0 con 0x7f998c100710 2023-10-18T12:35:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f998bfff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 0x7f998c198e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9a0 tx=0x7f997c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f99897fa700 1 -- 172.21.15.70:0/1513174079 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c009790 con 0x7f998c100710 2023-10-18T12:35:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c19e2c0 con 0x7f998c100710 2023-10-18T12:35:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f99897fa700 1 -- 172.21.15.70:0/1513174079 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997c010430 con 0x7f998c100710 2023-10-18T12:35:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f99897fa700 1 -- 172.21.15.70:0/1513174079 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c018680 con 0x7f998c100710 2023-10-18T12:35:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.596+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c1a5b00 con 0x7f998c100710 2023-10-18T12:35:09.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.601+0000 7f99897fa700 1 -- 172.21.15.70:0/1513174079 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f997c010810 con 0x7f998c100710 2023-10-18T12:35:09.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.602+0000 7f99a5d3cb80 1 Processor -- start 2023-10-18T12:35:09.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.602+0000 7f99a5d3cb80 1 -- start start 2023-10-18T12:35:09.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.602+0000 7f99897fa700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f99740a1670 0x7f99740a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99897fa700 1 -- 172.21.15.70:0/1513174079 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f997c014070 con 0x7f998c100710 2023-10-18T12:35:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9655fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9650090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda9650a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda95c36e0 con 0x55fda9656500 2023-10-18T12:35:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda95a9c60 con 0x55fda96505d0 2023-10-18T12:35:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda95a7580 con 0x55fda9655be0 2023-10-18T12:35:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f999099c700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f99740a1670 0x7f99740a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9655fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9650090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9655fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36430/0 (socket says 172.21.15.70:36430) 2023-10-18T12:35:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 -- 172.21.15.70:0/449596217 learned_addr learned my addr 172.21.15.70:0/449596217 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f999099c700 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda9650a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 -- 172.21.15.70:0/449596217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 msgr2=0x55fda9650a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda9650a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 -- 172.21.15.70:0/449596217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 msgr2=0x55fda9650090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9650090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.603+0000 7f998bfff700 1 -- 172.21.15.70:0/449596217 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fda95603f0 con 0x55fda9655be0 2023-10-18T12:35:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.604+0000 7f998bfff700 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9655fc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f997c096c40 tx=0x7f997c099b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f197f6e01945c6b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.604+0000 7f999099c700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f99740a1670 0x7f99740a3b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f998400f870 tx=0x7f998400fc30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.604+0000 7f99727fc700 1 -- 172.21.15.70:0/449596217 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0cd070 con 0x55fda9655be0 2023-10-18T12:35:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.604+0000 7f99727fc700 1 -- 172.21.15.70:0/449596217 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997c09e120 con 0x55fda9655be0 2023-10-18T12:35:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.604+0000 7f99727fc700 1 -- 172.21.15.70:0/449596217 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c09a430 con 0x55fda9655be0 2023-10-18T12:35:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.604+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/449596217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 msgr2=0x55fda9655fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9655fc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f997c096c40 tx=0x7f997c099b30 comp rx=0 tx=0).stop 2023-10-18T12:35:09.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/449596217 shutdown_connections 2023-10-18T12:35:09.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda9650a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9655fc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/449596217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9650090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/449596217 >> 172.21.15.70:0/449596217 conn(0x55fda95c4720 msgr2=0x55fda964ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/449596217 shutdown_connections 2023-10-18T12:35:09.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/449596217 wait complete. 2023-10-18T12:35:09.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.605+0000 7f99a5d3cb80 1 -- start start 2023-10-18T12:35:09.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.606+0000 7f99a5d3cb80 1 Processor -- start 2023-10-18T12:35:09.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.606+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda965c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.606+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda96598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9659fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda95a2e70 con 0x55fda9656500 2023-10-18T12:35:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda953f710 con 0x55fda96505d0 2023-10-18T12:35:09.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda953f3b0 con 0x55fda9655be0 2023-10-18T12:35:09.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f99a5d3cb80 10 client.0 _ll_register_callbacks cb 0x55fda9489a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:09.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda965c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda965c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58734/0 (socket says 172.21.15.70:58734) 2023-10-18T12:35:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 -- 172.21.15.70:0/1917149782 learned_addr learned my addr 172.21.15.70:0/1917149782 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f999099c700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9659fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998b7fe700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda96598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 -- 172.21.15.70:0/1917149782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 msgr2=0x55fda96598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda96598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 -- 172.21.15.70:0/1917149782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 msgr2=0x55fda9659fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9659fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 -- 172.21.15.70:0/1917149782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c1a5b00 con 0x55fda96505d0 2023-10-18T12:35:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f999099c700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9659fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f998bfff700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda965c560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f997c009230 tx=0x7f997c09a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c2efcf6112f507 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f996b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:09.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.607+0000 7f996b7fe700 1 -- 172.21.15.70:0/1917149782 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c07b020 con 0x55fda96505d0 2023-10-18T12:35:09.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fda95a7ef0 con 0x55fda96505d0 2023-10-18T12:35:09.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f996b7fe700 1 -- 172.21.15.70:0/1917149782 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f997c09a430 con 0x55fda96505d0 2023-10-18T12:35:09.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f996b7fe700 1 -- 172.21.15.70:0/1917149782 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f997c0cd040 con 0x55fda96505d0 2023-10-18T12:35:09.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 10 client.26879 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:09.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fda93cdde0 con 0x55fda96505d0 2023-10-18T12:35:09.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f996b7fe700 1 -- 172.21.15.70:0/1917149782 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fda95a7ef0 con 0x55fda96505d0 2023-10-18T12:35:09.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f996b7fe700 1 -- 172.21.15.70:0/1917149782 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f997c0cd2c0 con 0x55fda96505d0 2023-10-18T12:35:09.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 10 client.26879 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:09.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:09.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 20 client.26879 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:09.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda9669b90 0x55fda966bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 4 client.26879 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:09.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.608+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fda961f8b0 con 0x55fda9669b90 2023-10-18T12:35:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.609+0000 7f998b7fe700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda9669b90 0x55fda966bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.609+0000 7f998b7fe700 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda9669b90 0x55fda966bf80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9910fd5bf0c95ee server_cookie=f773d4724d35d08f in_seq=0 out_seq=0 2023-10-18T12:35:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.609+0000 7f996b7fe700 10 client.26879 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.610+0000 7f996b7fe700 1 -- 172.21.15.70:0/1917149782 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fda961f8b0 con 0x55fda9669b90 2023-10-18T12:35:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.610+0000 7f996b7fe700 10 client.26879 handle_command_reply: tid=0 2023-10-18T12:35:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.610+0000 7f99a5d3cb80 1 client.26879 shutdown 2023-10-18T12:35:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.610+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 msgr2=0x55fda965c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.610+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda965c560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f997c009230 tx=0x7f997c09a290 comp rx=0 tx=0).stop 2023-10-18T12:35:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.610+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 shutdown_connections 2023-10-18T12:35:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda9669b90 0x55fda966bf80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda9656500 0x55fda9659fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f996b7fe700 0 client.26879 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda96505d0 0x55fda965c560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1917149782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda96598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 >> 172.21.15.70:0/1917149782 conn(0x55fda95c4720 msgr2=0x55fda964f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 shutdown_connections 2023-10-18T12:35:09.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1917149782 wait complete. 2023-10-18T12:35:09.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.611+0000 7f99a5d3cb80 20 client.26879 trim_cache size 0 max 16384 2023-10-18T12:35:09.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 Processor -- start 2023-10-18T12:35:09.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 -- start start 2023-10-18T12:35:09.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda9660eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9653d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda964f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda9423920 con 0x55fda96505d0 2023-10-18T12:35:09.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda965ae50 con 0x55fda9656500 2023-10-18T12:35:09.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda964c3e0 con 0x55fda9655be0 2023-10-18T12:35:09.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f998b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9653d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.629+0000 7f998bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda9660eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f999099c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda964f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda9660eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45656/0 (socket says 172.21.15.70:45656) 2023-10-18T12:35:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998bfff700 1 -- 172.21.15.70:0/886028503 learned_addr learned my addr 172.21.15.70:0/886028503 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9653d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36450/0 (socket says 172.21.15.70:36450) 2023-10-18T12:35:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 -- 172.21.15.70:0/886028503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 msgr2=0x55fda964f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda964f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 -- 172.21.15.70:0/886028503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 msgr2=0x55fda9660eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda9660eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 -- 172.21.15.70:0/886028503 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fda93b6c60 con 0x55fda9655be0 2023-10-18T12:35:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f999099c700 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda964f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.630+0000 7f998b7fe700 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9653d60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f998000b2e0 tx=0x7f9980009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f2d4b2a509d068 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f9972ffd700 1 -- 172.21.15.70:0/886028503 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998001d070 con 0x55fda9655be0 2023-10-18T12:35:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f9972ffd700 1 -- 172.21.15.70:0/886028503 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f998001e040 con 0x55fda9655be0 2023-10-18T12:35:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f9972ffd700 1 -- 172.21.15.70:0/886028503 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000fa60 con 0x55fda9655be0 2023-10-18T12:35:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/886028503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 msgr2=0x55fda9653d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9653d60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f998000b2e0 tx=0x7f9980009310 comp rx=0 tx=0).stop 2023-10-18T12:35:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/886028503 shutdown_connections 2023-10-18T12:35:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda964f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda9660eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/886028503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda9653d60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/886028503 >> 172.21.15.70:0/886028503 conn(0x55fda95c4720 msgr2=0x55fda96509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/886028503 shutdown_connections 2023-10-18T12:35:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.631+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/886028503 wait complete. 2023-10-18T12:35:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.632+0000 7f99a5d3cb80 1 -- start start 2023-10-18T12:35:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.632+0000 7f99a5d3cb80 1 Processor -- start 2023-10-18T12:35:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda965f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda965faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda9660030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda96c62b0 con 0x55fda96505d0 2023-10-18T12:35:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda96c6420 con 0x55fda9656500 2023-10-18T12:35:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fda96c65e0 con 0x55fda9655be0 2023-10-18T12:35:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f99a5d3cb80 10 client.0 _ll_register_callbacks cb 0x55fda9489a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda9660030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f998bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda965f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda9660030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58754/0 (socket says 172.21.15.70:58754) 2023-10-18T12:35:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 -- 172.21.15.70:0/1991242924 learned_addr learned my addr 172.21.15.70:0/1991242924 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f998b7fe700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda965faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 -- 172.21.15.70:0/1991242924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 msgr2=0x55fda965faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda965faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 -- 172.21.15.70:0/1991242924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 msgr2=0x55fda965f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda965f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f999099c700 1 -- 172.21.15.70:0/1991242924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998400fdb0 con 0x55fda9656500 2023-10-18T12:35:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.633+0000 7f998bfff700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda965f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f999099c700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda9660030 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9984018e80 tx=0x7f9984004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9465e4dd09f2c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f996b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f996b7fe700 1 -- 172.21.15.70:0/1991242924 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400e040 con 0x55fda9656500 2023-10-18T12:35:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f997c09e120 con 0x55fda9656500 2023-10-18T12:35:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f996b7fe700 1 -- 172.21.15.70:0/1991242924 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f9984004db0 con 0x55fda9656500 2023-10-18T12:35:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f996b7fe700 1 -- 172.21.15.70:0/1991242924 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9984008e90 con 0x55fda9656500 2023-10-18T12:35:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f99a5d3cb80 10 client.26885 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f998c10dee0 con 0x55fda9656500 2023-10-18T12:35:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f996b7fe700 1 -- 172.21.15.70:0/1991242924 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997c09e120 con 0x55fda9656500 2023-10-18T12:35:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.634+0000 7f996b7fe700 1 -- 172.21.15.70:0/1991242924 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f998402d030 con 0x55fda9656500 2023-10-18T12:35:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f99a5d3cb80 10 client.26885 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f99a5d3cb80 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f99a5d3cb80 20 client.26885 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda96c7eb0 0x55fda96ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f99a5d3cb80 4 client.26885 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:35:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fda959be50 con 0x55fda96c7eb0 2023-10-18T12:35:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f998bfff700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda96c7eb0 0x55fda96ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f996b7fe700 10 client.26885 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.635+0000 7f998bfff700 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda96c7eb0 0x55fda96ca2d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e041813dd8158c71 server_cookie=f0b1aeccffd33749 in_seq=0 out_seq=0 2023-10-18T12:35:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f996b7fe700 1 -- 172.21.15.70:0/1991242924 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fda959be50 con 0x55fda96c7eb0 2023-10-18T12:35:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f996b7fe700 10 client.26885 handle_command_reply: tid=0 2023-10-18T12:35:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 client.26885 shutdown 2023-10-18T12:35:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 msgr2=0x55fda9660030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda9660030 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9984018e80 tx=0x7f9984004b40 comp rx=0 tx=0).stop 2023-10-18T12:35:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 shutdown_connections 2023-10-18T12:35:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55fda96c7eb0 0x55fda96ca2d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fda9656500 0x55fda9660030 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fda96505d0 0x55fda965f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f996b7fe700 0 client.26885 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:09.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 --2- 172.21.15.70:0/1991242924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fda9655be0 0x55fda965faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 >> 172.21.15.70:0/1991242924 conn(0x55fda95c4720 msgr2=0x55fda9653d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 shutdown_connections 2023-10-18T12:35:09.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 1 -- 172.21.15.70:0/1991242924 wait complete. 2023-10-18T12:35:09.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.638+0000 7f99a5d3cb80 20 client.26885 trim_cache size 0 max 16384 2023-10-18T12:35:09.653 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:35:09.653 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:35:09.653 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:35:09.653 INFO:teuthology.orchestra.run.smithi070.stdout: "dcf95a4c-1b02-471c-9db4-b5ef21f4f799": { 2023-10-18T12:35:09.654 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:35:09.654 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "dcf95a4c-1b02-471c-9db4-b5ef21f4f799", 2023-10-18T12:35:09.654 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:35:09.654 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:35:09.654 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:35:09.655 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:35:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f99740a1670 msgr2=0x7f99740a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f99740a1670 0x7f99740a3b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f998400f870 tx=0x7f998400fc30 comp rx=0 tx=0).stop 2023-10-18T12:35:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 msgr2=0x7f998c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 0x7f998c198e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9a0 tx=0x7f997c00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 shutdown_connections 2023-10-18T12:35:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f99740a1670 0x7f99740a3b30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f998c106720 0x7f998c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f998c100710 0x7f998c198e80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 --2- 172.21.15.70:0/1513174079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998c1010d0 0x7f998c1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 >> 172.21.15.70:0/1513174079 conn(0x7f998c0fc200 msgr2=0x7f998c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 shutdown_connections 2023-10-18T12:35:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:09.639+0000 7f99923ff700 1 -- 172.21.15.70:0/1513174079 wait complete. 2023-10-18T12:35:09.660 INFO:tasks.cephfs.filesystem:scrub status for tag:dcf95a4c-1b02-471c-9db4-b5ef21f4f799 - {'path': '/', 'tag': 'dcf95a4c-1b02-471c-9db4-b5ef21f4f799', 'options': 'recursive,force'} 2023-10-18T12:35:09.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1032124 11.62 MB/sec execute 338 sec latency 13.673 ms 2023-10-18T12:35:10.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:10 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:35:10.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:10 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:10.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:10 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:35:10.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:10 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:10.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:10 smithi070 ceph-mon[98639]: pgmap v594: 129 pgs: 129 active+clean; 566 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 265 op/s 2023-10-18T12:35:10.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:10 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:35:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:10 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:10 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:35:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:10 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:10 smithi152 ceph-mon[134767]: pgmap v594: 129 pgs: 129 active+clean; 566 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 265 op/s 2023-10-18T12:35:10.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:10 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:35:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:10 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:10 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:35:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:10 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:10 smithi186 ceph-mon[135253]: pgmap v594: 129 pgs: 129 active+clean; 566 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 265 op/s 2023-10-18T12:35:10.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1034444 11.63 MB/sec execute 339 sec latency 31.265 ms 2023-10-18T12:35:11.868 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1036546 11.63 MB/sec execute 340 sec latency 74.043 ms 2023-10-18T12:35:12.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1038059 11.62 MB/sec execute 341 sec latency 49.486 ms 2023-10-18T12:35:13.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:12 smithi070 ceph-mon[98639]: pgmap v595: 129 pgs: 129 active+clean; 566 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2023-10-18T12:35:13.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:12 smithi152 ceph-mon[134767]: pgmap v595: 129 pgs: 129 active+clean; 566 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2023-10-18T12:35:13.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:12 smithi186 ceph-mon[135253]: pgmap v595: 129 pgs: 129 active+clean; 566 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2023-10-18T12:35:13.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1040244 11.62 MB/sec execute 342 sec latency 33.216 ms 2023-10-18T12:35:14.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1042006 11.61 MB/sec execute 343 sec latency 12.200 ms 2023-10-18T12:35:15.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:14 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:35:15.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:14 smithi070 ceph-mon[98639]: pgmap v596: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2023-10-18T12:35:15.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:14 smithi070 ceph-mon[98639]: osdmap e99: 12 total, 12 up, 12 in 2023-10-18T12:35:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:14 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:35:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:14 smithi152 ceph-mon[134767]: pgmap v596: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2023-10-18T12:35:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:14 smithi152 ceph-mon[134767]: osdmap e99: 12 total, 12 up, 12 in 2023-10-18T12:35:15.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:14 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:35:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:14 smithi186 ceph-mon[135253]: pgmap v596: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2023-10-18T12:35:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:14 smithi186 ceph-mon[135253]: osdmap e99: 12 total, 12 up, 12 in 2023-10-18T12:35:15.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1044524 11.62 MB/sec execute 344 sec latency 16.024 ms 2023-10-18T12:35:16.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1046179 11.61 MB/sec execute 345 sec latency 13.593 ms 2023-10-18T12:35:17.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:16 smithi070 ceph-mon[98639]: pgmap v598: 129 pgs: 129 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-18T12:35:17.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:16 smithi152 ceph-mon[134767]: pgmap v598: 129 pgs: 129 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-18T12:35:17.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:16 smithi186 ceph-mon[135253]: pgmap v598: 129 pgs: 129 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-18T12:35:17.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1048332 11.62 MB/sec execute 346 sec latency 56.502 ms 2023-10-18T12:35:18.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1050026 11.60 MB/sec execute 347 sec latency 82.451 ms 2023-10-18T12:35:19.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:18 smithi070 ceph-mon[98639]: pgmap v599: 129 pgs: 129 active+clean; 576 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-10-18T12:35:19.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:18 smithi152 ceph-mon[134767]: pgmap v599: 129 pgs: 129 active+clean; 576 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-10-18T12:35:19.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:18 smithi186 ceph-mon[135253]: pgmap v599: 129 pgs: 129 active+clean; 576 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-10-18T12:35:19.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1052243 11.61 MB/sec execute 348 sec latency 34.457 ms 2023-10-18T12:35:20.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:20 smithi070 ceph-mon[98639]: pgmap v600: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 217 op/s 2023-10-18T12:35:20.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:20 smithi152 ceph-mon[134767]: pgmap v600: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 217 op/s 2023-10-18T12:35:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:20 smithi186 ceph-mon[135253]: pgmap v600: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 217 op/s 2023-10-18T12:35:20.869 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1054878 11.61 MB/sec execute 349 sec latency 14.948 ms 2023-10-18T12:35:21.870 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1056664 11.61 MB/sec execute 350 sec latency 26.852 ms 2023-10-18T12:35:22.870 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1058843 11.61 MB/sec execute 351 sec latency 131.465 ms 2023-10-18T12:35:23.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:22 smithi070 ceph-mon[98639]: pgmap v601: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 217 op/s 2023-10-18T12:35:23.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:22 smithi152 ceph-mon[134767]: pgmap v601: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 217 op/s 2023-10-18T12:35:23.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:22 smithi186 ceph-mon[135253]: pgmap v601: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 217 op/s 2023-10-18T12:35:23.871 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1060158 11.60 MB/sec execute 352 sec latency 120.031 ms 2023-10-18T12:35:24.870 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1062545 11.60 MB/sec execute 353 sec latency 52.701 ms 2023-10-18T12:35:25.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:24 smithi070 ceph-mon[98639]: pgmap v602: 129 pgs: 129 active+clean; 588 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 200 op/s 2023-10-18T12:35:25.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:24 smithi152 ceph-mon[134767]: pgmap v602: 129 pgs: 129 active+clean; 588 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 200 op/s 2023-10-18T12:35:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:24 smithi186 ceph-mon[135253]: pgmap v602: 129 pgs: 129 active+clean; 588 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 200 op/s 2023-10-18T12:35:25.871 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1064566 11.59 MB/sec execute 354 sec latency 16.017 ms 2023-10-18T12:35:26.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1066479 11.59 MB/sec execute 355 sec latency 19.744 ms 2023-10-18T12:35:27.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:26 smithi152 ceph-mon[134767]: pgmap v603: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 218 op/s 2023-10-18T12:35:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:26 smithi186 ceph-mon[135253]: pgmap v603: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 218 op/s 2023-10-18T12:35:27.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:26 smithi070 ceph-mon[98639]: pgmap v603: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 218 op/s 2023-10-18T12:35:27.871 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1069060 11.60 MB/sec execute 356 sec latency 17.427 ms 2023-10-18T12:35:28.871 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1070393 11.59 MB/sec execute 357 sec latency 48.692 ms 2023-10-18T12:35:29.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:28 smithi152 ceph-mon[134767]: pgmap v604: 129 pgs: 129 active+clean; 555 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2023-10-18T12:35:29.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:28 smithi186 ceph-mon[135253]: pgmap v604: 129 pgs: 129 active+clean; 555 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2023-10-18T12:35:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:28 smithi070 ceph-mon[98639]: pgmap v604: 129 pgs: 129 active+clean; 555 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2023-10-18T12:35:29.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1072998 11.60 MB/sec execute 358 sec latency 50.538 ms 2023-10-18T12:35:30.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:30 smithi070 ceph-mon[98639]: pgmap v605: 129 pgs: 129 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2023-10-18T12:35:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:30 smithi152 ceph-mon[134767]: pgmap v605: 129 pgs: 129 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2023-10-18T12:35:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:30 smithi186 ceph-mon[135253]: pgmap v605: 129 pgs: 129 active+clean; 563 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2023-10-18T12:35:30.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1074873 11.58 MB/sec execute 359 sec latency 15.585 ms 2023-10-18T12:35:31.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1077060 11.59 MB/sec execute 360 sec latency 16.023 ms 2023-10-18T12:35:32.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1079536 11.59 MB/sec execute 361 sec latency 28.187 ms 2023-10-18T12:35:33.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:32 smithi070 ceph-mon[98639]: pgmap v606: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 224 op/s 2023-10-18T12:35:33.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:32 smithi152 ceph-mon[134767]: pgmap v606: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 224 op/s 2023-10-18T12:35:33.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:32 smithi186 ceph-mon[135253]: pgmap v606: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 224 op/s 2023-10-18T12:35:33.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1081103 11.58 MB/sec execute 362 sec latency 33.461 ms 2023-10-18T12:35:34.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1083632 11.60 MB/sec execute 363 sec latency 31.476 ms 2023-10-18T12:35:35.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:34 smithi070 ceph-mon[98639]: pgmap v607: 129 pgs: 129 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 230 op/s 2023-10-18T12:35:35.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:34 smithi152 ceph-mon[134767]: pgmap v607: 129 pgs: 129 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 230 op/s 2023-10-18T12:35:35.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:34 smithi186 ceph-mon[135253]: pgmap v607: 129 pgs: 129 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 230 op/s 2023-10-18T12:35:35.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1085400 11.58 MB/sec execute 364 sec latency 14.240 ms 2023-10-18T12:35:36.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1087750 11.59 MB/sec execute 365 sec latency 18.008 ms 2023-10-18T12:35:37.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:36 smithi070 ceph-mon[98639]: pgmap v608: 129 pgs: 129 active+clean; 570 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-18T12:35:37.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:36 smithi152 ceph-mon[134767]: pgmap v608: 129 pgs: 129 active+clean; 570 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-18T12:35:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:36 smithi186 ceph-mon[135253]: pgmap v608: 129 pgs: 129 active+clean; 570 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 217 op/s 2023-10-18T12:35:37.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1089856 11.59 MB/sec execute 366 sec latency 160.905 ms 2023-10-18T12:35:38.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1091384 11.58 MB/sec execute 367 sec latency 190.744 ms 2023-10-18T12:35:39.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:38 smithi070 ceph-mon[98639]: pgmap v609: 129 pgs: 129 active+clean; 581 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 228 op/s 2023-10-18T12:35:39.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:38 smithi152 ceph-mon[134767]: pgmap v609: 129 pgs: 129 active+clean; 581 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 228 op/s 2023-10-18T12:35:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:38 smithi186 ceph-mon[135253]: pgmap v609: 129 pgs: 129 active+clean; 581 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 228 op/s 2023-10-18T12:35:39.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:35:39.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1093696 11.58 MB/sec execute 368 sec latency 38.067 ms 2023-10-18T12:35:39.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.894+0000 7f0a42dff700 1 -- 172.21.15.70:0/1061724698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 msgr2=0x7f0a3c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.894+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1061724698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 0x7f0a3c068120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009990 tx=0x7f0a34009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 -- 172.21.15.70:0/1061724698 shutdown_connections 2023-10-18T12:35:39.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1061724698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105b30 0x7f0a3c10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1061724698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 0x7f0a3c068120 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1061724698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105170 0x7f0a3c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 -- 172.21.15.70:0/1061724698 >> 172.21.15.70:0/1061724698 conn(0x7f0a3c0767f0 msgr2=0x7f0a3c076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:39.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 -- 172.21.15.70:0/1061724698 shutdown_connections 2023-10-18T12:35:39.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.895+0000 7f0a42dff700 1 -- 172.21.15.70:0/1061724698 wait complete. 2023-10-18T12:35:39.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.896+0000 7f0a42dff700 1 Processor -- start 2023-10-18T12:35:39.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 -- start start 2023-10-18T12:35:39.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 0x7f0a3c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 0x7f0a3c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105b30 0x7f0a3c100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c1a4fd0 con 0x7f0a3c105b30 2023-10-18T12:35:39.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c1a5140 con 0x7f0a3c067d40 2023-10-18T12:35:39.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.897+0000 7f0a42dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c1a5300 con 0x7f0a3c105170 2023-10-18T12:35:39.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 0x7f0a3c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 0x7f0a3c0ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36462/0 (socket says 172.21.15.70:36462) 2023-10-18T12:35:39.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1971513039 learned_addr learned my addr 172.21.15.70:0/1971513039 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:39.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a4139c700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105b30 0x7f0a3c100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 0x7f0a3c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1971513039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 msgr2=0x7f0a3c0ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 0x7f0a3c0ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105b30 msgr2=0x7f0a3c100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105b30 0x7f0a3c100150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1971513039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340095f0 con 0x7f0a3c105170 2023-10-18T12:35:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a4139c700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105b30 0x7f0a3c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 0x7f0a3c0ffbf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400e9a0 tx=0x7f0a2400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a39ffb700 1 -- 172.21.15.70:0/1971513039 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24009790 con 0x7f0a3c105170 2023-10-18T12:35:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c1a55e0 con 0x7f0a3c105170 2023-10-18T12:35:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.898+0000 7f0a39ffb700 1 -- 172.21.15.70:0/1971513039 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2400cbd0 con 0x7f0a3c105170 2023-10-18T12:35:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.899+0000 7f0a39ffb700 1 -- 172.21.15.70:0/1971513039 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24018680 con 0x7f0a3c105170 2023-10-18T12:35:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.899+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c1a5af0 con 0x7f0a3c105170 2023-10-18T12:35:39.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.902+0000 7f0a39ffb700 1 -- 172.21.15.70:0/1971513039 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0a2400cd40 con 0x7f0a3c105170 2023-10-18T12:35:39.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.903+0000 7f0a39ffb700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a280a15f0 0x7f0a280a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.903+0000 7f0a5678bb80 1 Processor -- start 2023-10-18T12:35:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.903+0000 7f0a39ffb700 1 -- 172.21.15.70:0/1971513039 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0a24014070 con 0x7f0a3c105170 2023-10-18T12:35:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a280a15f0 0x7f0a280a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 -- start start 2023-10-18T12:35:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a280a15f0 0x7f0a280a3ab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0a340098e0 tx=0x7f0a34005960 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d006082fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d00607d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d00607da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005ff06e0 con 0x55d006083500 2023-10-18T12:35:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005fd6c60 con 0x55d00607d5d0 2023-10-18T12:35:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.904+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005fd4580 con 0x55d006082be0 2023-10-18T12:35:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d00607d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d00607d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50708/0 (socket says 172.21.15.70:50708) 2023-10-18T12:35:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d00607da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a3bfff700 1 -- 172.21.15.70:0/2922435018 learned_addr learned my addr 172.21.15.70:0/2922435018 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d006082fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 -- 172.21.15.70:0/2922435018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 msgr2=0x55d006082fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d006082fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 -- 172.21.15.70:0/2922435018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 msgr2=0x55d00607d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d00607d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 -- 172.21.15.70:0/2922435018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d005f8d3f0 con 0x55d00607d5d0 2023-10-18T12:35:39.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d00607d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:39.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a4139c700 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d00607da50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c100690 tx=0x7f0a2c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583b0891b674914e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:39.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.905+0000 7f0a32ffd700 1 -- 172.21.15.70:0/2922435018 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00e070 con 0x55d00607d5d0 2023-10-18T12:35:39.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a32ffd700 1 -- 172.21.15.70:0/2922435018 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2c00f040 con 0x55d00607d5d0 2023-10-18T12:35:39.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a32ffd700 1 -- 172.21.15.70:0/2922435018 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c004860 con 0x55d00607d5d0 2023-10-18T12:35:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 -- 172.21.15.70:0/2922435018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 msgr2=0x55d00607da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d00607da50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c100690 tx=0x7f0a2c009920 comp rx=0 tx=0).stop 2023-10-18T12:35:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 -- 172.21.15.70:0/2922435018 shutdown_connections 2023-10-18T12:35:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d00607da50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d006082fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/2922435018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d00607d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 -- 172.21.15.70:0/2922435018 >> 172.21.15.70:0/2922435018 conn(0x55d005ff1720 msgr2=0x55d006077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.906+0000 7f0a5678bb80 1 -- 172.21.15.70:0/2922435018 shutdown_connections 2023-10-18T12:35:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.907+0000 7f0a5678bb80 1 -- 172.21.15.70:0/2922435018 wait complete. 2023-10-18T12:35:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.907+0000 7f0a5678bb80 1 -- start start 2023-10-18T12:35:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.907+0000 7f0a5678bb80 1 Processor -- start 2023-10-18T12:35:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d0060893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d0060866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d006086c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005fcfe70 con 0x55d006083500 2023-10-18T12:35:39.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005f6c710 con 0x55d00607d5d0 2023-10-18T12:35:39.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005f6c3b0 con 0x55d006082be0 2023-10-18T12:35:39.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a5678bb80 10 client.0 _ll_register_callbacks cb 0x55d005eb6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:39.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a4139c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d006086c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a4139c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d006086c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50720/0 (socket says 172.21.15.70:50720) 2023-10-18T12:35:39.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a40b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d0060893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a4139c700 1 -- 172.21.15.70:0/3847137973 learned_addr learned my addr 172.21.15.70:0/3847137973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:39.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.908+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d0060866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a4139c700 1 -- 172.21.15.70:0/3847137973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 msgr2=0x55d0060866d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a4139c700 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d0060866d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a4139c700 1 -- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 msgr2=0x55d0060893e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a4139c700 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d0060893e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a4139c700 1 -- 172.21.15.70:0/3847137973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c1a5af0 con 0x55d006083500 2023-10-18T12:35:39.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a4139c700 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d006086c10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0094f0 tx=0x7f0a2c003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa365f0e065e5fd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:39.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:35:39.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3847137973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00e030 con 0x55d006083500 2023-10-18T12:35:39.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d005fd4ef0 con 0x55d006083500 2023-10-18T12:35:39.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.909+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3847137973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0a2c004ce0 con 0x55d006083500 2023-10-18T12:35:39.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3847137973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2c0095e0 con 0x55d006083500 2023-10-18T12:35:39.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 10 client.16851 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:39.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f0a2c009050 con 0x55d006083500 2023-10-18T12:35:39.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3847137973 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d005fd4ef0 con 0x55d006083500 2023-10-18T12:35:39.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3847137973 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a2c024020 con 0x55d006083500 2023-10-18T12:35:39.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 10 client.16851 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:39.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:39.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 20 client.16851 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:39.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d00609f440 0x55d0060a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 4 client.16851 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:39.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.910+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d00604c8b0 con 0x55d00609f440 2023-10-18T12:35:39.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.911+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d00609f440 0x55d0060a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.911+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d00609f440 0x55d0060a1860 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a2a22161fdd47 server_cookie=66ac49bfd78f154 in_seq=0 out_seq=0 2023-10-18T12:35:39.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.911+0000 7f0a1bfff700 10 client.16851 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:39.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3847137973 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d00604c8b0 con 0x55d00609f440 2023-10-18T12:35:39.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a1bfff700 10 client.16851 handle_command_reply: tid=0 2023-10-18T12:35:39.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 client.16851 shutdown 2023-10-18T12:35:39.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 msgr2=0x55d006086c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d006086c10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0094f0 tx=0x7f0a2c003cb0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 shutdown_connections 2023-10-18T12:35:39.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d00609f440 0x55d0060a1860 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006083500 0x55d006086c10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d00607d5d0 0x55d0060893e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3847137973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006082be0 0x55d0060866d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a1bfff700 0 client.16851 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:39.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 >> 172.21.15.70:0/3847137973 conn(0x55d005ff1720 msgr2=0x55d00607c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:39.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 shutdown_connections 2023-10-18T12:35:39.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.912+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3847137973 wait complete. 2023-10-18T12:35:39.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.913+0000 7f0a5678bb80 20 client.16851 trim_cache size 0 max 16384 2023-10-18T12:35:39.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 Processor -- start 2023-10-18T12:35:39.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 -- start start 2023-10-18T12:35:39.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d00607d5d0 0x55d006096740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 0x55d006077810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006083500 0x55d006086690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d005e50920 con 0x55d00607d5d0 2023-10-18T12:35:39.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006078a50 con 0x55d006082be0 2023-10-18T12:35:39.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.936+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006078650 con 0x55d006083500 2023-10-18T12:35:39.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 0x55d006077810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a40b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d00607d5d0 0x55d006096740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 0x55d006077810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54648/0 (socket says 172.21.15.70:54648) 2023-10-18T12:35:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1195127409 learned_addr learned my addr 172.21.15.70:0/1195127409 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a4139c700 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006083500 0x55d006086690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1195127409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006083500 msgr2=0x55d006086690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006083500 0x55d006086690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1195127409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d00607d5d0 msgr2=0x55d006096740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d00607d5d0 0x55d006096740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 -- 172.21.15.70:0/1195127409 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d005dfade0 con 0x55d006082be0 2023-10-18T12:35:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d00607d5d0 0x55d006096740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 0x55d006077810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0a2409a090 tx=0x7f0a2401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b810faae65328bf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.937+0000 7f0a337fe700 1 -- 172.21.15.70:0/1195127409 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2409a390 con 0x55d006082be0 2023-10-18T12:35:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a337fe700 1 -- 172.21.15.70:0/1195127409 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2409e4e0 con 0x55d006082be0 2023-10-18T12:35:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a337fe700 1 -- 172.21.15.70:0/1195127409 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a240cd570 con 0x55d006082be0 2023-10-18T12:35:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 -- 172.21.15.70:0/1195127409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 msgr2=0x55d006077810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 0x55d006077810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0a2409a090 tx=0x7f0a2401a040 comp rx=0 tx=0).stop 2023-10-18T12:35:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 -- 172.21.15.70:0/1195127409 shutdown_connections 2023-10-18T12:35:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d006083500 0x55d006086690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d00607d5d0 0x55d006096740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/1195127409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006082be0 0x55d006077810 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 -- 172.21.15.70:0/1195127409 >> 172.21.15.70:0/1195127409 conn(0x55d005ff1720 msgr2=0x55d00607c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 -- 172.21.15.70:0/1195127409 shutdown_connections 2023-10-18T12:35:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.938+0000 7f0a5678bb80 1 -- 172.21.15.70:0/1195127409 wait complete. 2023-10-18T12:35:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 -- start start 2023-10-18T12:35:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 Processor -- start 2023-10-18T12:35:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d00607d5d0 0x55d006094e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006082be0 0x55d0060953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 0x55d0060958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006096c90 con 0x55d006082be0 2023-10-18T12:35:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0060f32a0 con 0x55d006083500 2023-10-18T12:35:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0060f33e0 con 0x55d00607d5d0 2023-10-18T12:35:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.939+0000 7f0a5678bb80 10 client.0 _ll_register_callbacks cb 0x55d005eb6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 0x55d0060958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a3bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006082be0 0x55d0060953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 0x55d0060958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54654/0 (socket says 172.21.15.70:54654) 2023-10-18T12:35:39.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 -- 172.21.15.70:0/3413967167 learned_addr learned my addr 172.21.15.70:0/3413967167 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:39.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d00607d5d0 0x55d006094e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 -- 172.21.15.70:0/3413967167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d00607d5d0 msgr2=0x55d006094e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d00607d5d0 0x55d006094e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 -- 172.21.15.70:0/3413967167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006082be0 msgr2=0x55d0060953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:39.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006082be0 0x55d0060953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 -- 172.21.15.70:0/3413967167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c10de70 con 0x55d006083500 2023-10-18T12:35:39.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a4139c700 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 0x55d0060958e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009f50 tx=0x7f0a2c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78dff50b8b6107b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:39.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:39.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3413967167 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00e070 con 0x55d006083500 2023-10-18T12:35:39.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.940+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a2c00f040 con 0x55d006083500 2023-10-18T12:35:39.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3413967167 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0a2c004ce0 con 0x55d006083500 2023-10-18T12:35:39.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3413967167 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2c004ea0 con 0x55d006083500 2023-10-18T12:35:39.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 10 client.26903 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:39.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f0a3c1a55e0 con 0x55d006083500 2023-10-18T12:35:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3413967167 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2c00f040 con 0x55d006083500 2023-10-18T12:35:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3413967167 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a2c0077e0 con 0x55d006083500 2023-10-18T12:35:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 10 client.26903 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 10 client.26903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 20 client.26903 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d0060f46f0 0x55d0060f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 4 client.26903 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:35:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.941+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d005fc8e50 con 0x55d0060f46f0 2023-10-18T12:35:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.942+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d0060f46f0 0x55d0060f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.942+0000 7f0a40b9b700 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d0060f46f0 0x55d0060f6b10 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eedc2c4fb67388 server_cookie=ad529767dd5cbb0a in_seq=0 out_seq=0 2023-10-18T12:35:39.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.942+0000 7f0a1bfff700 10 client.26903 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a1bfff700 1 -- 172.21.15.70:0/3413967167 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d005fc8e50 con 0x55d0060f46f0 2023-10-18T12:35:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a1bfff700 10 client.26903 handle_command_reply: tid=0 2023-10-18T12:35:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 client.26903 shutdown 2023-10-18T12:35:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 msgr2=0x55d0060958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 0x55d0060958e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009f50 tx=0x7f0a2c009920 comp rx=0 tx=0).stop 2023-10-18T12:35:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 shutdown_connections 2023-10-18T12:35:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55d0060f46f0 0x55d0060f6b10 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006083500 0x55d0060958e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d00607d5d0 0x55d006094e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 --2- 172.21.15.70:0/3413967167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d006082be0 0x55d0060953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a1bfff700 0 client.26903 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:39.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.943+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 >> 172.21.15.70:0/3413967167 conn(0x55d005ff1720 msgr2=0x55d006084ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:39.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 shutdown_connections 2023-10-18T12:35:39.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a5678bb80 1 -- 172.21.15.70:0/3413967167 wait complete. 2023-10-18T12:35:39.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a5678bb80 20 client.26903 trim_cache size 0 max 16384 2023-10-18T12:35:39.961 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:35:39.961 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:35:39.961 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:35:39.961 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:35:39.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a280a15f0 msgr2=0x7f0a280a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a280a15f0 0x7f0a280a3ab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0a340098e0 tx=0x7f0a34005960 comp rx=0 tx=0).stop 2023-10-18T12:35:39.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 msgr2=0x7f0a3c0ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:39.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 0x7f0a3c0ffbf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400e9a0 tx=0x7f0a2400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 shutdown_connections 2023-10-18T12:35:39.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a3c105b30 0x7f0a3c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a280a15f0 0x7f0a280a3ab0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a3c067d40 0x7f0a3c0ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 --2- 172.21.15.70:0/1971513039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a3c105170 0x7f0a3c0ffbf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.944+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 >> 172.21.15.70:0/1971513039 conn(0x7f0a3c0767f0 msgr2=0x7f0a3c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.945+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 shutdown_connections 2023-10-18T12:35:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:39.945+0000 7f0a42dff700 1 -- 172.21.15.70:0/1971513039 wait complete. 2023-10-18T12:35:39.965 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:35:39.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:35:40.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 -- 172.21.15.70:0/3781317450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 msgr2=0x7f03f0106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 --2- 172.21.15.70:0/3781317450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 0x7f03f0106a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0098f0 tx=0x7f03ec009c00 comp rx=0 tx=0).stop 2023-10-18T12:35:40.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 -- 172.21.15.70:0/3781317450 shutdown_connections 2023-10-18T12:35:40.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 --2- 172.21.15.70:0/3781317450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0101060 0x7f03f010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 --2- 172.21.15.70:0/3781317450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 0x7f03f0106a90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 --2- 172.21.15.70:0/3781317450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f01006a0 0x7f03f0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 -- 172.21.15.70:0/3781317450 >> 172.21.15.70:0/3781317450 conn(0x7f03f00fc170 msgr2=0x7f03f00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 -- 172.21.15.70:0/3781317450 shutdown_connections 2023-10-18T12:35:40.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.172+0000 7f03f81ff700 1 -- 172.21.15.70:0/3781317450 wait complete. 2023-10-18T12:35:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.173+0000 7f03f81ff700 1 Processor -- start 2023-10-18T12:35:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.173+0000 7f03f81ff700 1 -- start start 2023-10-18T12:35:40.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f81ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f01006a0 0x7f03f0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f81ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 0x7f03f01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f81ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 0x7f03f019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f81ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f019dc90 con 0x7f03f0101060 2023-10-18T12:35:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f81ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f019de00 con 0x7f03f01066b0 2023-10-18T12:35:40.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f81ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f019dfc0 con 0x7f03f01006a0 2023-10-18T12:35:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 0x7f03f01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 0x7f03f01992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50778/0 (socket says 172.21.15.70:50778) 2023-10-18T12:35:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 -- 172.21.15.70:0/132473080 learned_addr learned my addr 172.21.15.70:0/132473080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f679c700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 0x7f03f019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f5f9b700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f01006a0 0x7f03f0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 -- 172.21.15.70:0/132473080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f01006a0 msgr2=0x7f03f0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f01006a0 0x7f03f0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 -- 172.21.15.70:0/132473080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 msgr2=0x7f03f019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 0x7f03f019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.174+0000 7f03f579a700 1 -- 172.21.15.70:0/132473080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00095f0 con 0x7f03f0101060 2023-10-18T12:35:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.175+0000 7f03f579a700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 0x7f03f01992b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03e000ec10 tx=0x7f03e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.175+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e000cc80 con 0x7f03f0101060 2023-10-18T12:35:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.175+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03e00044e0 con 0x7f03f0101060 2023-10-18T12:35:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.175+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0010430 con 0x7f03f0101060 2023-10-18T12:35:40.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.175+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e8009c40 con 0x7f03f0101060 2023-10-18T12:35:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.175+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec009520 con 0x7f03f0101060 2023-10-18T12:35:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.176+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0101de0 con 0x7f03f0101060 2023-10-18T12:35:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.178+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f03e0003f70 con 0x7f03f0101060 2023-10-18T12:35:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.179+0000 7f03e6ffd700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03dc0a1670 0x7f03dc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.180+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f03e0014070 con 0x7f03f0101060 2023-10-18T12:35:40.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.180+0000 7f03f5f9b700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03dc0a1670 0x7f03dc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.180+0000 7f03f5f9b700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03dc0a1670 0x7f03dc0a3b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0098c0 tx=0x7f03ec019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.182+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f03e00969b0 con 0x7f03f0101060 2023-10-18T12:35:40.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.507+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03f0067530 con 0x7f03f0101060 2023-10-18T12:35:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.508+0000 7f03e6ffd700 1 -- 172.21.15.70:0/132473080 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f03e001f340 con 0x7f03f0101060 2023-10-18T12:35:40.512 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:35:40.512 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:35:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.511+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03dc0a1670 msgr2=0x7f03dc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.511+0000 7f03f81ff700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03dc0a1670 0x7f03dc0a3b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0098c0 tx=0x7f03ec019040 comp rx=0 tx=0).stop 2023-10-18T12:35:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.511+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 msgr2=0x7f03f01992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.511+0000 7f03f81ff700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 0x7f03f01992b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03e000ec10 tx=0x7f03e000ef20 comp rx=0 tx=0).stop 2023-10-18T12:35:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 shutdown_connections 2023-10-18T12:35:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f03dc0a1670 0x7f03dc0a3b30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f01066b0 0x7f03f019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f01006a0 0x7f03f0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 --2- 172.21.15.70:0/132473080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03f0101060 0x7f03f01992b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 >> 172.21.15.70:0/132473080 conn(0x7f03f00fc170 msgr2=0x7f03f010bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 shutdown_connections 2023-10-18T12:35:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.512+0000 7f03f81ff700 1 -- 172.21.15.70:0/132473080 wait complete. 2023-10-18T12:35:40.518 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:35:40.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:35:40.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:40 smithi070 ceph-mon[98639]: pgmap v610: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2023-10-18T12:35:40.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:40 smithi152 ceph-mon[134767]: pgmap v610: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2023-10-18T12:35:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:40 smithi186 ceph-mon[135253]: pgmap v610: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2023-10-18T12:35:40.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 -- 172.21.15.70:0/3424201731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c100720 msgr2=0x7fd13c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 --2- 172.21.15.70:0/3424201731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c100720 0x7fd13c100ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd130009990 tx=0x7fd130009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 -- 172.21.15.70:0/3424201731 shutdown_connections 2023-10-18T12:35:40.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 --2- 172.21.15.70:0/3424201731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c1010e0 0x7fd13c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 --2- 172.21.15.70:0/3424201731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd13c106730 0x7fd13c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 --2- 172.21.15.70:0/3424201731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c100720 0x7fd13c100ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.768+0000 7fd1415ff700 1 -- 172.21.15.70:0/3424201731 >> 172.21.15.70:0/3424201731 conn(0x7fd13c0fc1b0 msgr2=0x7fd13c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.769+0000 7fd1415ff700 1 -- 172.21.15.70:0/3424201731 shutdown_connections 2023-10-18T12:35:40.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.769+0000 7fd1415ff700 1 -- 172.21.15.70:0/3424201731 wait complete. 2023-10-18T12:35:40.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.770+0000 7fd1415ff700 1 Processor -- start 2023-10-18T12:35:40.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.770+0000 7fd1415ff700 1 -- start start 2023-10-18T12:35:40.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd1415ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c100720 0x7fd13c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd1415ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd13c1010e0 0x7fd13c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd1415ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c106730 0x7fd13c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd1415ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c19dc90 con 0x7fd13c1010e0 2023-10-18T12:35:40.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd1415ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c19de00 con 0x7fd13c106730 2023-10-18T12:35:40.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd1415ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c19dfc0 con 0x7fd13c100720 2023-10-18T12:35:40.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd13affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c100720 0x7fd13c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd13affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c100720 0x7fd13c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36548/0 (socket says 172.21.15.70:36548) 2023-10-18T12:35:40.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd13affd700 1 -- 172.21.15.70:0/1935713750 learned_addr learned my addr 172.21.15.70:0/1935713750 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:40.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c106730 0x7fd13c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.771+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd13c1010e0 0x7fd13c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd13b7fe700 1 -- 172.21.15.70:0/1935713750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c100720 msgr2=0x7fd13c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c100720 0x7fd13c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd13b7fe700 1 -- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd13c1010e0 msgr2=0x7fd13c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd13c1010e0 0x7fd13c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd13b7fe700 1 -- 172.21.15.70:0/1935713750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1300095f0 con 0x7fd13c106730 2023-10-18T12:35:40.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c106730 0x7fd13c19d650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd13400efc0 tx=0x7fd13400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd123fff700 1 -- 172.21.15.70:0/1935713750 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134018070 con 0x7fd13c106730 2023-10-18T12:35:40.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd123fff700 1 -- 172.21.15.70:0/1935713750 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd13400f040 con 0x7fd13c106730 2023-10-18T12:35:40.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c19e240 con 0x7fd13c106730 2023-10-18T12:35:40.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd123fff700 1 -- 172.21.15.70:0/1935713750 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13400cd40 con 0x7fd13c106730 2023-10-18T12:35:40.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.772+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c1a5ae0 con 0x7fd13c106730 2023-10-18T12:35:40.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.775+0000 7fd123fff700 1 -- 172.21.15.70:0/1935713750 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd134003a00 con 0x7fd13c106730 2023-10-18T12:35:40.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.777+0000 7fd154e6cb80 1 Processor -- start 2023-10-18T12:35:40.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.777+0000 7fd123fff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1240a15f0 0x7fd1240a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.777+0000 7fd123fff700 1 -- 172.21.15.70:0/1935713750 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fd1340cd0b0 con 0x7fd13c106730 2023-10-18T12:35:40.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.777+0000 7fd154e6cb80 1 -- start start 2023-10-18T12:35:40.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.777+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1240a15f0 0x7fd1240a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.777+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf191fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf18c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf18ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf0ff6e0 con 0x562bdf192500 2023-10-18T12:35:40.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf0e5c60 con 0x562bdf18c5d0 2023-10-18T12:35:40.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf0e3580 con 0x562bdf191be0 2023-10-18T12:35:40.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf191fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf18ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf191fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36552/0 (socket says 172.21.15.70:36552) 2023-10-18T12:35:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13affd700 1 -- 172.21.15.70:0/3683736960 learned_addr learned my addr 172.21.15.70:0/3683736960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1240a15f0 0x7fd1240a3ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd13c19a2c0 tx=0x7fd130005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf18c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13b7fe700 1 -- 172.21.15.70:0/3683736960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 msgr2=0x562bdf191fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf191fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13b7fe700 1 -- 172.21.15.70:0/3683736960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 msgr2=0x562bdf18c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf18c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13b7fe700 1 -- 172.21.15.70:0/3683736960 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bdf09c3f0 con 0x562bdf18c5d0 2023-10-18T12:35:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf18c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.778+0000 7fd13affd700 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf191fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.779+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf18ca50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd1340ca440 tx=0x7fd134095aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8631d0bfc83aab0c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.779+0000 7fd1217fa700 1 -- 172.21.15.70:0/3683736960 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13409d020 con 0x562bdf18c5d0 2023-10-18T12:35:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.779+0000 7fd1217fa700 1 -- 172.21.15.70:0/3683736960 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd134004ad0 con 0x562bdf18c5d0 2023-10-18T12:35:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.779+0000 7fd1217fa700 1 -- 172.21.15.70:0/3683736960 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340981d0 con 0x562bdf18c5d0 2023-10-18T12:35:40.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.779+0000 7fd154e6cb80 1 -- 172.21.15.70:0/3683736960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 msgr2=0x562bdf18ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.779+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf18ca50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd1340ca440 tx=0x7fd134095aa0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 -- 172.21.15.70:0/3683736960 shutdown_connections 2023-10-18T12:35:40.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf18ca50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf191fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/3683736960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf18c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 -- 172.21.15.70:0/3683736960 >> 172.21.15.70:0/3683736960 conn(0x562bdf100720 msgr2=0x562bdf186b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 -- 172.21.15.70:0/3683736960 shutdown_connections 2023-10-18T12:35:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 -- 172.21.15.70:0/3683736960 wait complete. 2023-10-18T12:35:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 -- start start 2023-10-18T12:35:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.780+0000 7fd154e6cb80 1 Processor -- start 2023-10-18T12:35:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf198530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf195820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf195f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf0dee70 con 0x562bdf18c5d0 2023-10-18T12:35:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf07b710 con 0x562bdf191be0 2023-10-18T12:35:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf07b3b0 con 0x562bdf192500 2023-10-18T12:35:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd154e6cb80 10 client.0 _ll_register_callbacks cb 0x562bdefc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd13affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd13b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd13a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd13affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50806/0 (socket says 172.21.15.70:50806) 2023-10-18T12:35:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.781+0000 7fd13affd700 1 -- 172.21.15.70:0/2595174719 learned_addr learned my addr 172.21.15.70:0/2595174719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13affd700 1 -- 172.21.15.70:0/2595174719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 msgr2=0x562bdf195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13affd700 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13affd700 1 -- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 msgr2=0x562bdf195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13affd700 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13affd700 1 -- 172.21.15.70:0/2595174719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c009a30 con 0x562bdf18c5d0 2023-10-18T12:35:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd13affd700 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf198530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd12c011d10 tx=0x7fd12c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a53350b7032f6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd11eff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:35:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd11eff5700 1 -- 172.21.15.70:0/2595174719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c016070 con 0x562bdf18c5d0 2023-10-18T12:35:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bdf0e3ef0 con 0x562bdf18c5d0 2023-10-18T12:35:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd11eff5700 1 -- 172.21.15.70:0/2595174719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fd12c00fd90 con 0x562bdf18c5d0 2023-10-18T12:35:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.782+0000 7fd11eff5700 1 -- 172.21.15.70:0/2595174719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd12c009e70 con 0x562bdf18c5d0 2023-10-18T12:35:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 10 client.16875 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fd13c1a5ae0 con 0x562bdf18c5d0 2023-10-18T12:35:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd11eff5700 1 -- 172.21.15.70:0/2595174719 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bdf0e3ef0 con 0x562bdf18c5d0 2023-10-18T12:35:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd11eff5700 1 -- 172.21.15.70:0/2595174719 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd12c003cb0 con 0x562bdf18c5d0 2023-10-18T12:35:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 10 client.16875 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 20 client.16875 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1a5fb0 0x562bdf1a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 4 client.16875 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bdf15b8b0 con 0x562bdf1a5fb0 2023-10-18T12:35:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.783+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1a5fb0 0x562bdf1a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.784+0000 7fd11eff5700 10 client.16875 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.784+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1a5fb0 0x562bdf1a83d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52ff779cb8472d8 server_cookie=29e9923e235aae36 in_seq=0 out_seq=0 2023-10-18T12:35:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.784+0000 7fd11eff5700 1 -- 172.21.15.70:0/2595174719 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562bdf15b8b0 con 0x562bdf1a5fb0 2023-10-18T12:35:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.784+0000 7fd11eff5700 10 client.16875 handle_command_reply: tid=0 2023-10-18T12:35:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 client.16875 shutdown 2023-10-18T12:35:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 msgr2=0x562bdf198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf198530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd12c011d10 tx=0x7fd12c00f6a0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 shutdown_connections 2023-10-18T12:35:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1a5fb0 0x562bdf1a83d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf198530 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2595174719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd11eff5700 0 client.16875 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:40.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 >> 172.21.15.70:0/2595174719 conn(0x562bdf100720 msgr2=0x562bdf18b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 shutdown_connections 2023-10-18T12:35:40.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2595174719 wait complete. 2023-10-18T12:35:40.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.785+0000 7fd154e6cb80 20 client.16875 trim_cache size 0 max 16384 2023-10-18T12:35:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.799+0000 7fd154e6cb80 1 Processor -- start 2023-10-18T12:35:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.799+0000 7fd154e6cb80 1 -- start start 2023-10-18T12:35:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf18c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf18b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdef5f920 con 0x562bdf18c5d0 2023-10-18T12:35:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf196b70 con 0x562bdf191be0 2023-10-18T12:35:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf19d3f0 con 0x562bdf192500 2023-10-18T12:35:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf18c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54708/0 (socket says 172.21.15.70:54708) 2023-10-18T12:35:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 -- 172.21.15.70:0/2413611235 learned_addr learned my addr 172.21.15.70:0/2413611235 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf18b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 -- 172.21.15.70:0/2413611235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 msgr2=0x562bdf18b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf18b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 -- 172.21.15.70:0/2413611235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 msgr2=0x562bdf18c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf18c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.800+0000 7fd13a7fc700 1 -- 172.21.15.70:0/2413611235 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bdef09de0 con 0x562bdf191be0 2023-10-18T12:35:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.801+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf19ceb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd130009990 tx=0x7fd130020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96445570677db6f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.801+0000 7fd121ffb700 1 -- 172.21.15.70:0/2413611235 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130033ad0 con 0x562bdf191be0 2023-10-18T12:35:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.801+0000 7fd121ffb700 1 -- 172.21.15.70:0/2413611235 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd13001e040 con 0x562bdf191be0 2023-10-18T12:35:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.801+0000 7fd121ffb700 1 -- 172.21.15.70:0/2413611235 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130004920 con 0x562bdf191be0 2023-10-18T12:35:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.801+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2413611235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 msgr2=0x562bdf19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf19ceb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd130009990 tx=0x7fd130020000 comp rx=0 tx=0).stop 2023-10-18T12:35:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2413611235 shutdown_connections 2023-10-18T12:35:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf192500 0x562bdf18b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf18c5d0 0x562bdf18c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2413611235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf191be0 0x562bdf19ceb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2413611235 >> 172.21.15.70:0/2413611235 conn(0x562bdf100720 msgr2=0x562bdf1acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2413611235 shutdown_connections 2023-10-18T12:35:40.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2413611235 wait complete. 2023-10-18T12:35:40.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 -- start start 2023-10-18T12:35:40.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.802+0000 7fd154e6cb80 1 Processor -- start 2023-10-18T12:35:40.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf19c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf1f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf1f9300 con 0x562bdf192500 2023-10-18T12:35:40.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf1f9440 con 0x562bdf18c5d0 2023-10-18T12:35:40.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bdf1f9600 con 0x562bdf191be0 2023-10-18T12:35:40.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd154e6cb80 10 client.0 _ll_register_callbacks cb 0x562bdefc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:40.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd13affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd13a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd13affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54714/0 (socket says 172.21.15.70:54714) 2023-10-18T12:35:40.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd13affd700 1 -- 172.21.15.70:0/2863987612 learned_addr learned my addr 172.21.15.70:0/2863987612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:40.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.803+0000 7fd13b7fe700 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf1f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd13affd700 1 -- 172.21.15.70:0/2863987612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 msgr2=0x562bdf19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd13affd700 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd13affd700 1 -- 172.21.15.70:0/2863987612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 msgr2=0x562bdf1f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:40.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd13affd700 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf1f8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd13affd700 1 -- 172.21.15.70:0/2863987612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c10def0 con 0x562bdf18c5d0 2023-10-18T12:35:40.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd13affd700 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf19c2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd12c011d10 tx=0x7fd12c009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800064e5b3c0e9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:40.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd11eff5700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:40.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd11eff5700 1 -- 172.21.15.70:0/2863987612 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c016070 con 0x562bdf18c5d0 2023-10-18T12:35:40.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd134004ad0 con 0x562bdf18c5d0 2023-10-18T12:35:40.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd11eff5700 1 -- 172.21.15.70:0/2863987612 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fd12c000e40 con 0x562bdf18c5d0 2023-10-18T12:35:40.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd11eff5700 1 -- 172.21.15.70:0/2863987612 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd12c005110 con 0x562bdf18c5d0 2023-10-18T12:35:40.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd154e6cb80 10 client.26927 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:40.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.804+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562bdeef2c60 con 0x562bdf18c5d0 2023-10-18T12:35:40.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd11eff5700 1 -- 172.21.15.70:0/2863987612 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd134004ad0 con 0x562bdf18c5d0 2023-10-18T12:35:40.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd11eff5700 1 -- 172.21.15.70:0/2863987612 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd12c00f3c0 con 0x562bdf18c5d0 2023-10-18T12:35:40.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd154e6cb80 10 client.26927 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:40.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd154e6cb80 10 client.26927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:40.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd154e6cb80 20 client.26927 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:40.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1fae20 0x562bdf1fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:40.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd154e6cb80 4 client.26927 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:35:40.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562bdf0d7e50 con 0x562bdf1fae20 2023-10-18T12:35:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.805+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1fae20 0x562bdf1fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.806+0000 7fd13a7fc700 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1fae20 0x562bdf1fd240 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85937a8e145a6b9b server_cookie=7ff39d9aa613ab4c in_seq=0 out_seq=0 2023-10-18T12:35:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.806+0000 7fd11eff5700 10 client.26927 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd11eff5700 1 -- 172.21.15.70:0/2863987612 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562bdf0d7e50 con 0x562bdf1fae20 2023-10-18T12:35:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd11eff5700 10 client.26927 handle_command_reply: tid=0 2023-10-18T12:35:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 client.26927 shutdown 2023-10-18T12:35:40.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 msgr2=0x562bdf19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf19c2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd12c011d10 tx=0x7fd12c009960 comp rx=0 tx=0).stop 2023-10-18T12:35:40.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 shutdown_connections 2023-10-18T12:35:40.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562bdf1fae20 0x562bdf1fd240 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bdf192500 0x562bdf1f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562bdf18c5d0 0x562bdf19c2c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd11eff5700 0 client.26927 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:40.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 --2- 172.21.15.70:0/2863987612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562bdf191be0 0x562bdf19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 >> 172.21.15.70:0/2863987612 conn(0x562bdf100720 msgr2=0x562bdf194050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 shutdown_connections 2023-10-18T12:35:40.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 1 -- 172.21.15.70:0/2863987612 wait complete. 2023-10-18T12:35:40.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.807+0000 7fd154e6cb80 20 client.26927 trim_cache size 0 max 16384 2023-10-18T12:35:40.823 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:35:40.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1240a15f0 msgr2=0x7fd1240a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1240a15f0 0x7fd1240a3ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd13c19a2c0 tx=0x7fd130005fd0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c106730 msgr2=0x7fd13c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:40.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c106730 0x7fd13c19d650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd13400efc0 tx=0x7fd13400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 shutdown_connections 2023-10-18T12:35:40.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd1240a15f0 0x7fd1240a3ab0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd13c106730 0x7fd13c19d650 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd13c100720 0x7fd13c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 --2- 172.21.15.70:0/1935713750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd13c1010e0 0x7fd13c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:40.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 >> 172.21.15.70:0/1935713750 conn(0x7fd13c0fc1b0 msgr2=0x7fd13c10bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:40.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 shutdown_connections 2023-10-18T12:35:40.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:40.808+0000 7fd1415ff700 1 -- 172.21.15.70:0/1935713750 wait complete. 2023-10-18T12:35:40.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1095486 11.58 MB/sec execute 369 sec latency 18.466 ms 2023-10-18T12:35:41.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:35:41.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:41 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/132473080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:35:41.678 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:35:41.679 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:41 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/132473080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:35:41.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:35:41.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:41 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/132473080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:35:41.831 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:35:41.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:35:41.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1097926 11.59 MB/sec execute 370 sec latency 13.982 ms 2023-10-18T12:35:42.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 -- 172.21.15.70:0/1040960170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d50100710 msgr2=0x7f2d50100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 --2- 172.21.15.70:0/1040960170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d50100710 0x7f2d50100b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009990 tx=0x7f2d40009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 -- 172.21.15.70:0/1040960170 shutdown_connections 2023-10-18T12:35:42.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 --2- 172.21.15.70:0/1040960170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d501010d0 0x7f2d5010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 --2- 172.21.15.70:0/1040960170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 0x7f2d50106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 --2- 172.21.15.70:0/1040960170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d50100710 0x7f2d50100b90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 -- 172.21.15.70:0/1040960170 >> 172.21.15.70:0/1040960170 conn(0x7f2d500fc200 msgr2=0x7f2d500fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 -- 172.21.15.70:0/1040960170 shutdown_connections 2023-10-18T12:35:42.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.086+0000 7f2d583ff700 1 -- 172.21.15.70:0/1040960170 wait complete. 2023-10-18T12:35:42.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 Processor -- start 2023-10-18T12:35:42.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 -- start start 2023-10-18T12:35:42.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d50100710 0x7f2d50072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 0x7f2d50073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 0x7f2d50073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501a4ef0 con 0x7f2d50100710 2023-10-18T12:35:42.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501a5060 con 0x7f2d501010d0 2023-10-18T12:35:42.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.087+0000 7f2d583ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501a5220 con 0x7f2d50106720 2023-10-18T12:35:42.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 0x7f2d50073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d50100710 0x7f2d50072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 0x7f2d50073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54740/0 (socket says 172.21.15.70:54740) 2023-10-18T12:35:42.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 -- 172.21.15.70:0/287410456 learned_addr learned my addr 172.21.15.70:0/287410456 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5699c700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 0x7f2d50073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 -- 172.21.15.70:0/287410456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 msgr2=0x7f2d50073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 0x7f2d50073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 -- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d50100710 msgr2=0x7f2d50072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d50100710 0x7f2d50072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 -- 172.21.15.70:0/287410456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400095f0 con 0x7f2d501010d0 2023-10-18T12:35:42.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5699c700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 0x7f2d50073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:42.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5619b700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d50100710 0x7f2d50072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:42.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.088+0000 7f2d5599a700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 0x7f2d50073030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000c00 tx=0x7f2d40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.089+0000 7f2d477fe700 1 -- 172.21.15.70:0/287410456 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4001d070 con 0x7f2d501010d0 2023-10-18T12:35:42.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.089+0000 7f2d477fe700 1 -- 172.21.15.70:0/287410456 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d40022470 con 0x7f2d501010d0 2023-10-18T12:35:42.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.089+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d501a54a0 con 0x7f2d501010d0 2023-10-18T12:35:42.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.089+0000 7f2d477fe700 1 -- 172.21.15.70:0/287410456 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40017650 con 0x7f2d501010d0 2023-10-18T12:35:42.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.089+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d501a59d0 con 0x7f2d501010d0 2023-10-18T12:35:42.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.092+0000 7f2d477fe700 1 -- 172.21.15.70:0/287410456 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2d400225e0 con 0x7f2d501010d0 2023-10-18T12:35:42.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 Processor -- start 2023-10-18T12:35:42.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 -- start start 2023-10-18T12:35:42.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d477fe700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d3c0a1670 0x7f2d3c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bd7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bd1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd1650 0x555bb9bd1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9b44760 con 0x555bb9bd7580 2023-10-18T12:35:42.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9b2ace0 con 0x555bb9bd6c60 2023-10-18T12:35:42.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9b28600 con 0x555bb9bd1650 2023-10-18T12:35:42.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.094+0000 7f2d477fe700 1 -- 172.21.15.70:0/287410456 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f2d400ce740 con 0x7f2d501010d0 2023-10-18T12:35:42.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bd7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5699c700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d3c0a1670 0x7f2d3c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bd7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54744/0 (socket says 172.21.15.70:54744) 2023-10-18T12:35:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 -- 172.21.15.70:0/2683345695 learned_addr learned my addr 172.21.15.70:0/2683345695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5599a700 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bd1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5699c700 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd1650 0x555bb9bd1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 -- 172.21.15.70:0/2683345695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd1650 msgr2=0x555bb9bd1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd1650 0x555bb9bd1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 -- 172.21.15.70:0/2683345695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 msgr2=0x555bb9bd1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bd1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 -- 172.21.15.70:0/2683345695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bb9ae1470 con 0x555bb9bd6c60 2023-10-18T12:35:42.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5599a700 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:42.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5699c700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d3c0a1670 0x7f2d3c0a3b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d48015e20 tx=0x7f2d48015420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.095+0000 7f2d5619b700 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bd7040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0115f0 tx=0x7f2d4c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c668273ea64d961 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d44bfe700 1 -- 172.21.15.70:0/2683345695 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c019630 con 0x555bb9bd6c60 2023-10-18T12:35:42.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d44bfe700 1 -- 172.21.15.70:0/2683345695 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d4c019c50 con 0x555bb9bd6c60 2023-10-18T12:35:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d44bfe700 1 -- 172.21.15.70:0/2683345695 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00ebc0 con 0x555bb9bd6c60 2023-10-18T12:35:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/2683345695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 msgr2=0x555bb9bd7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bd7040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0115f0 tx=0x7f2d4c011900 comp rx=0 tx=0).stop 2023-10-18T12:35:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/2683345695 shutdown_connections 2023-10-18T12:35:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd1650 0x555bb9bd1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bd7040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/2683345695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/2683345695 >> 172.21.15.70:0/2683345695 conn(0x555bb9b457a0 msgr2=0x555bb9bcbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/2683345695 shutdown_connections 2023-10-18T12:35:42.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.096+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/2683345695 wait complete. 2023-10-18T12:35:42.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.097+0000 7f2d6bde9b80 1 -- start start 2023-10-18T12:35:42.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.097+0000 7f2d6bde9b80 1 Processor -- start 2023-10-18T12:35:42.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9bdd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bda800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd7580 0x555bb9bdae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9b23ef0 con 0x555bb9bd1650 2023-10-18T12:35:42.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9ac0790 con 0x555bb9bd6c60 2023-10-18T12:35:42.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9ac0430 con 0x555bb9bd7580 2023-10-18T12:35:42.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d6bde9b80 10 client.0 _ll_register_callbacks cb 0x555bb9a0ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:42.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9bdd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bda800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bda800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54756/0 (socket says 172.21.15.70:54756) 2023-10-18T12:35:42.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 -- 172.21.15.70:0/1892905355 learned_addr learned my addr 172.21.15.70:0/1892905355 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5699c700 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd7580 0x555bb9bdae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 -- 172.21.15.70:0/1892905355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd7580 msgr2=0x555bb9bdae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd7580 0x555bb9bdae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 -- 172.21.15.70:0/1892905355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 msgr2=0x555bb9bdd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9bdd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.098+0000 7f2d5599a700 1 -- 172.21.15.70:0/1892905355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d501a59d0 con 0x555bb9bd6c60 2023-10-18T12:35:42.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d5599a700 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bda800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2d4009b380 tx=0x7f2d40022bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6490bd8ea2d86617 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:42.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1892905355 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400ce540 con 0x555bb9bd6c60 2023-10-18T12:35:42.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bb9b28f70 con 0x555bb9bd6c60 2023-10-18T12:35:42.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1892905355 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f2d4009baf0 con 0x555bb9bd6c60 2023-10-18T12:35:42.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1892905355 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d4001c990 con 0x555bb9bd6c60 2023-10-18T12:35:42.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 10 client.26939 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:42.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f2d4c009d50 con 0x555bb9bd6c60 2023-10-18T12:35:42.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1892905355 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bb9b28f70 con 0x555bb9bd6c60 2023-10-18T12:35:42.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1892905355 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d400b9450 con 0x555bb9bd6c60 2023-10-18T12:35:42.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 10 client.26939 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:42.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 10 client.26939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:42.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 20 client.26939 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:42.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9beab20 0x555bb9becf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 4 client.26939 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:42.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.099+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bb9ba0930 con 0x555bb9beab20 2023-10-18T12:35:42.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.100+0000 7f2d5619b700 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9beab20 0x555bb9becf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.100+0000 7f2d5619b700 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9beab20 0x555bb9becf10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db32eb866b4159b6 server_cookie=7fab3dc8081f40a9 in_seq=0 out_seq=0 2023-10-18T12:35:42.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.100+0000 7f2d39ffb700 10 client.26939 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1892905355 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555bb9ba0930 con 0x555bb9beab20 2023-10-18T12:35:42.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d39ffb700 10 client.26939 handle_command_reply: tid=0 2023-10-18T12:35:42.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 client.26939 shutdown 2023-10-18T12:35:42.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 msgr2=0x555bb9bda800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bda800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2d4009b380 tx=0x7f2d40022bc0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 shutdown_connections 2023-10-18T12:35:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9beab20 0x555bb9becf10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd7580 0x555bb9bdae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9bdd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1892905355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd6c60 0x555bb9bda800 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 >> 172.21.15.70:0/1892905355 conn(0x555bb9b457a0 msgr2=0x555bb9bcbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d39ffb700 0 client.26939 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.104+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 shutdown_connections 2023-10-18T12:35:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.105+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1892905355 wait complete. 2023-10-18T12:35:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.105+0000 7f2d6bde9b80 20 client.26939 trim_cache size 0 max 16384 2023-10-18T12:35:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 Processor -- start 2023-10-18T12:35:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 -- start start 2023-10-18T12:35:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 0x555bb9bd1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9be6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bc5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9bcc1b0 con 0x555bb9bd7580 2023-10-18T12:35:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb99a49a0 con 0x555bb9bd1650 2023-10-18T12:35:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9bcd660 con 0x555bb9bd6c60 2023-10-18T12:35:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 0x555bb9bd1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 0x555bb9bd1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54764/0 (socket says 172.21.15.70:54764) 2023-10-18T12:35:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d5619b700 1 -- 172.21.15.70:0/296928498 learned_addr learned my addr 172.21.15.70:0/296928498 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d5699c700 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bc5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.123+0000 7f2d5599a700 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9be6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d5619b700 1 -- 172.21.15.70:0/296928498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 msgr2=0x555bb9be6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d5619b700 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9be6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d5619b700 1 -- 172.21.15.70:0/296928498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 msgr2=0x555bb9bc5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d5619b700 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bc5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d5619b700 1 -- 172.21.15.70:0/296928498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bb9bf1b40 con 0x555bb9bd1650 2023-10-18T12:35:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d5619b700 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 0x555bb9bd1110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0115f0 tx=0x7f2d4c00e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a0b08c66cc4a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d453ff700 1 -- 172.21.15.70:0/296928498 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0052e0 con 0x555bb9bd1650 2023-10-18T12:35:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d453ff700 1 -- 172.21.15.70:0/296928498 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d4c0198f0 con 0x555bb9bd1650 2023-10-18T12:35:42.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.124+0000 7f2d453ff700 1 -- 172.21.15.70:0/296928498 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c003b10 con 0x555bb9bd1650 2023-10-18T12:35:42.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/296928498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 msgr2=0x555bb9bd1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 0x555bb9bd1110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0115f0 tx=0x7f2d4c00e3b0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/296928498 shutdown_connections 2023-10-18T12:35:42.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd7580 0x555bb9bc5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd1650 0x555bb9bd1110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/296928498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9be6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/296928498 >> 172.21.15.70:0/296928498 conn(0x555bb9b457a0 msgr2=0x555bb9be6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/296928498 shutdown_connections 2023-10-18T12:35:42.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/296928498 wait complete. 2023-10-18T12:35:42.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.125+0000 7f2d6bde9b80 1 -- start start 2023-10-18T12:35:42.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 Processor -- start 2023-10-18T12:35:42.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9be1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9bdff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd7580 0x555bb9be0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9be09d0 con 0x555bb9bd1650 2023-10-18T12:35:42.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9be0b40 con 0x555bb9bd7580 2023-10-18T12:35:42.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb9be0d00 con 0x555bb9bd6c60 2023-10-18T12:35:42.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.126+0000 7f2d6bde9b80 10 client.0 _ll_register_callbacks cb 0x555bb9a0ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:42.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5619b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9be1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9bdff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9bdff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36630/0 (socket says 172.21.15.70:36630) 2023-10-18T12:35:42.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 -- 172.21.15.70:0/1909041314 learned_addr learned my addr 172.21.15.70:0/1909041314 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5699c700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd7580 0x555bb9be0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 -- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd7580 msgr2=0x555bb9be0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd7580 0x555bb9be0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 -- 172.21.15.70:0/1909041314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 msgr2=0x555bb9be1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9be1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 -- 172.21.15.70:0/1909041314 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bb9ae1470 con 0x555bb9bd6c60 2023-10-18T12:35:42.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5619b700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9be1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:42.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d5599a700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9bdff10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d4009b0f0 tx=0x7f2d40097130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa28b34323b3e80 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:35:42.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1909041314 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4009baf0 con 0x555bb9bd6c60 2023-10-18T12:35:42.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d4c019c50 con 0x555bb9bd6c60 2023-10-18T12:35:42.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.127+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1909041314 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f2d400a36f0 con 0x555bb9bd6c60 2023-10-18T12:35:42.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1909041314 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d4009b3f0 con 0x555bb9bd6c60 2023-10-18T12:35:42.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 10 client.26929 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:42.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555bb994ee60 con 0x555bb9bd6c60 2023-10-18T12:35:42.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1909041314 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d40080080 con 0x555bb9bd6c60 2023-10-18T12:35:42.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1909041314 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d4009b650 con 0x555bb9bd6c60 2023-10-18T12:35:42.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 10 client.26929 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:42.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:42.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 20 client.26929 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:42.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9c40b60 0x555bb9c42f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 4 client.26929 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:35:42.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555bb9b1ced0 con 0x555bb9c40b60 2023-10-18T12:35:42.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.128+0000 7f2d5619b700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9c40b60 0x555bb9c42f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.129+0000 7f2d39ffb700 10 client.26929 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.129+0000 7f2d5619b700 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9c40b60 0x555bb9c42f80 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f789e964a04fb499 server_cookie=b240669f7b240ca3 in_seq=0 out_seq=0 2023-10-18T12:35:42.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d39ffb700 1 -- 172.21.15.70:0/1909041314 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555bb9b1ced0 con 0x555bb9c40b60 2023-10-18T12:35:42.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d39ffb700 10 client.26929 handle_command_reply: tid=0 2023-10-18T12:35:42.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 client.26929 shutdown 2023-10-18T12:35:42.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 msgr2=0x555bb9bdff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9bdff10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d4009b0f0 tx=0x7f2d40097130 comp rx=0 tx=0).stop 2023-10-18T12:35:42.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 shutdown_connections 2023-10-18T12:35:42.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555bb9c40b60 0x555bb9c42f80 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555bb9bd7580 0x555bb9be0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555bb9bd1650 0x555bb9be1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d39ffb700 0 client.26929 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 --2- 172.21.15.70:0/1909041314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bb9bd6c60 0x555bb9bdff10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 >> 172.21.15.70:0/1909041314 conn(0x555bb9b457a0 msgr2=0x555bb9be6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 shutdown_connections 2023-10-18T12:35:42.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 1 -- 172.21.15.70:0/1909041314 wait complete. 2023-10-18T12:35:42.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.148+0000 7f2d6bde9b80 20 client.26929 trim_cache size 0 max 16384 2023-10-18T12:35:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:35:42.155 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:35:42.156 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "804e285d-9075-4a0f-9879-f34c943e2730", 2023-10-18T12:35:42.156 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:35:42.156 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:35:42.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d3c0a1670 msgr2=0x7f2d3c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d3c0a1670 0x7f2d3c0a3b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d48015e20 tx=0x7f2d48015420 comp rx=0 tx=0).stop 2023-10-18T12:35:42.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 msgr2=0x7f2d50073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 0x7f2d50073030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000c00 tx=0x7f2d40004730 comp rx=0 tx=0).stop 2023-10-18T12:35:42.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 shutdown_connections 2023-10-18T12:35:42.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d50106720 0x7f2d50073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d50100710 0x7f2d50072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2d3c0a1670 0x7f2d3c0a3b30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 --2- 172.21.15.70:0/287410456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d501010d0 0x7f2d50073030 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 >> 172.21.15.70:0/287410456 conn(0x7f2d500fc200 msgr2=0x7f2d500fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 shutdown_connections 2023-10-18T12:35:42.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.149+0000 7f2d583ff700 1 -- 172.21.15.70:0/287410456 wait complete. 2023-10-18T12:35:42.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:35:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 -- 172.21.15.70:0/2873976538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 msgr2=0x7fe8c8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/2873976538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8106b10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009990 tx=0x7fe8c4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 -- 172.21.15.70:0/2873976538 shutdown_connections 2023-10-18T12:35:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/2873976538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c81010e0 0x7fe8c810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/2873976538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8106b10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/2873976538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c8100720 0x7fe8c8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.427+0000 7fe8d03ff700 1 -- 172.21.15.70:0/2873976538 >> 172.21.15.70:0/2873976538 conn(0x7fe8c80fc1b0 msgr2=0x7fe8c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.428+0000 7fe8d03ff700 1 -- 172.21.15.70:0/2873976538 shutdown_connections 2023-10-18T12:35:42.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.428+0000 7fe8d03ff700 1 -- 172.21.15.70:0/2873976538 wait complete. 2023-10-18T12:35:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 Processor -- start 2023-10-18T12:35:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 -- start start 2023-10-18T12:35:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 0x7fe8c819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c81010e0 0x7fe8c819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81991f0 con 0x7fe8c81010e0 2023-10-18T12:35:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8199360 con 0x7fe8c8106730 2023-10-18T12:35:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.429+0000 7fe8d03ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8199520 con 0x7fe8c8100720 2023-10-18T12:35:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 0x7fe8c819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8cd99a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c81010e0 0x7fe8c819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 0x7fe8c819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36634/0 (socket says 172.21.15.70:36634) 2023-10-18T12:35:42.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 -- 172.21.15.70:0/3161344410 learned_addr learned my addr 172.21.15.70:0/3161344410 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 -- 172.21.15.70:0/3161344410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 msgr2=0x7fe8c8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 -- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c81010e0 msgr2=0x7fe8c819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c81010e0 0x7fe8c819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 -- 172.21.15.70:0/3161344410 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009520 con 0x7fe8c8100720 2023-10-18T12:35:42.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:35:42.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.430+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 0x7fe8c819ea60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4005770 tx=0x7fe8c40057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.431+0000 7fe8bf7fe700 1 -- 172.21.15.70:0/3161344410 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c401d070 con 0x7fe8c8100720 2023-10-18T12:35:42.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.431+0000 7fe8bf7fe700 1 -- 172.21.15.70:0/3161344410 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c4021410 con 0x7fe8c8100720 2023-10-18T12:35:42.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.431+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40095f0 con 0x7fe8c8100720 2023-10-18T12:35:42.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.431+0000 7fe8bf7fe700 1 -- 172.21.15.70:0/3161344410 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4017650 con 0x7fe8c8100720 2023-10-18T12:35:42.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.431+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81999f0 con 0x7fe8c8100720 2023-10-18T12:35:42.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.434+0000 7fe8bf7fe700 1 -- 172.21.15.70:0/3161344410 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe8c40217f0 con 0x7fe8c8100720 2023-10-18T12:35:42.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.436+0000 7fe8bf7fe700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe8b40a1670 0x7fe8b40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.436+0000 7fe8e3d93b80 1 Processor -- start 2023-10-18T12:35:42.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.436+0000 7fe8bf7fe700 1 -- 172.21.15.70:0/3161344410 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe8c40ce720 con 0x7fe8c8100720 2023-10-18T12:35:42.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.436+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe8b40a1670 0x7fe8b40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.436+0000 7fe8e3d93b80 1 -- start start 2023-10-18T12:35:42.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x55601200dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012008090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012008a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011f7b6e0 con 0x5560120085d0 2023-10-18T12:35:42.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011f61c60 con 0x55601200dbe0 2023-10-18T12:35:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011f5f580 con 0x55601200e500 2023-10-18T12:35:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe8b40a1670 0x7fe8b40a3b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe8c819a230 tx=0x7fe8b8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012008090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8ce19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x55601200dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012008090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36644/0 (socket says 172.21.15.70:36644) 2023-10-18T12:35:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 -- 172.21.15.70:0/1975858038 learned_addr learned my addr 172.21.15.70:0/1975858038 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012008a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 -- 172.21.15.70:0/1975858038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 msgr2=0x55601200dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x55601200dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 -- 172.21.15.70:0/1975858038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 msgr2=0x556012008a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012008a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 -- 172.21.15.70:0/1975858038 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556011f183f0 con 0x55601200e500 2023-10-18T12:35:42.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.437+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012008090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800e5c0 tx=0x7fe8b8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c410ce81468b0de4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.438+0000 7fe8bcbfe700 1 -- 172.21.15.70:0/1975858038 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8004220 con 0x55601200e500 2023-10-18T12:35:42.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.438+0000 7fe8bcbfe700 1 -- 172.21.15.70:0/1975858038 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b8021070 con 0x55601200e500 2023-10-18T12:35:42.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.438+0000 7fe8bcbfe700 1 -- 172.21.15.70:0/1975858038 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b801d670 con 0x55601200e500 2023-10-18T12:35:42.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/1975858038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 msgr2=0x556012008090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012008090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800e5c0 tx=0x7fe8b8014000 comp rx=0 tx=0).stop 2023-10-18T12:35:42.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/1975858038 shutdown_connections 2023-10-18T12:35:42.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012008a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x55601200dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/1975858038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012008090 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/1975858038 >> 172.21.15.70:0/1975858038 conn(0x556011f7c720 msgr2=0x556012002b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/1975858038 shutdown_connections 2023-10-18T12:35:42.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.439+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/1975858038 wait complete. 2023-10-18T12:35:42.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 -- start start 2023-10-18T12:35:42.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 Processor -- start 2023-10-18T12:35:42.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560120085d0 0x556012014450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55601200dbe0 0x556012011740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012011d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011f5ae70 con 0x55601200dbe0 2023-10-18T12:35:42.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011ef7710 con 0x55601200e500 2023-10-18T12:35:42.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011ef73b0 con 0x5560120085d0 2023-10-18T12:35:42.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.440+0000 7fe8e3d93b80 10 client.0 _ll_register_callbacks cb 0x556011e41a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:42.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560120085d0 0x556012014450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560120085d0 0x556012014450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:36652/0 (socket says 172.21.15.70:36652) 2023-10-18T12:35:42.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce19b700 1 -- 172.21.15.70:0/3466443005 learned_addr learned my addr 172.21.15.70:0/3466443005 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012011d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55601200dbe0 0x556012011740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3466443005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560120085d0 msgr2=0x556012014450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560120085d0 0x556012014450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3466443005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55601200dbe0 msgr2=0x556012011740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55601200dbe0 0x556012011740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.441+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3466443005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b8017070 con 0x55601200e500 2023-10-18T12:35:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012011d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007b60 tx=0x7fe8c0007b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1398a4d8c6a5578 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3466443005 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000e030 con 0x55601200e500 2023-10-18T12:35:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3466443005 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe8c0003c00 con 0x55601200e500 2023-10-18T12:35:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556011f5fef0 con 0x55601200e500 2023-10-18T12:35:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3466443005 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8c0003dc0 con 0x55601200e500 2023-10-18T12:35:42.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8e3d93b80 10 client.26957 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:42.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.442+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556011d85de0 con 0x55601200e500 2023-10-18T12:35:42.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3466443005 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556011f5fef0 con 0x55601200e500 2023-10-18T12:35:42.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3466443005 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8c0004400 con 0x55601200e500 2023-10-18T12:35:42.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8e3d93b80 10 client.26957 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:42.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8e3d93b80 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:42.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8e3d93b80 20 client.26957 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:42.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012021f90 0x5560120243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8e3d93b80 4 client.26957 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:35:42.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556011fd78b0 con 0x556012021f90 2023-10-18T12:35:42.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.443+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012021f90 0x5560120243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.444+0000 7fe8b1ffb700 10 client.26957 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.444+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012021f90 0x5560120243b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be72152e6dd3bb7b server_cookie=131d705a1700dd46 in_seq=0 out_seq=0 2023-10-18T12:35:42.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.445+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3466443005 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556011fd78b0 con 0x556012021f90 2023-10-18T12:35:42.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.445+0000 7fe8b1ffb700 10 client.26957 handle_command_reply: tid=0 2023-10-18T12:35:42.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.445+0000 7fe8e3d93b80 1 client.26957 shutdown 2023-10-18T12:35:42.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 msgr2=0x556012011d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012011d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007b60 tx=0x7fe8c0007b90 comp rx=0 tx=0).stop 2023-10-18T12:35:42.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 shutdown_connections 2023-10-18T12:35:42.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012021f90 0x5560120243b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012011d70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8b1ffb700 0 client.26957 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5560120085d0 0x556012014450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3466443005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55601200dbe0 0x556012011740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 >> 172.21.15.70:0/3466443005 conn(0x556011f7c720 msgr2=0x556012007080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 shutdown_connections 2023-10-18T12:35:42.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3466443005 wait complete. 2023-10-18T12:35:42.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.446+0000 7fe8e3d93b80 20 client.26957 trim_cache size 0 max 16384 2023-10-18T12:35:42.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 Processor -- start 2023-10-18T12:35:42.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 -- start start 2023-10-18T12:35:42.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012011c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x556012018f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012010010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556011ddb920 con 0x5560120085d0 2023-10-18T12:35:42.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556012008410 con 0x55601200dbe0 2023-10-18T12:35:42.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.465+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556012014c90 con 0x55601200e500 2023-10-18T12:35:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012010010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8cd99a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x556012018f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012011c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8cd99a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x556012018f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54822/0 (socket says 172.21.15.70:54822) 2023-10-18T12:35:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8cd99a700 1 -- 172.21.15.70:0/707948789 learned_addr learned my addr 172.21.15.70:0/707948789 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 -- 172.21.15.70:0/707948789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 msgr2=0x556012018f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x556012018f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 -- 172.21.15.70:0/707948789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 msgr2=0x556012011c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012011c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 -- 172.21.15.70:0/707948789 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556011d6ec60 con 0x55601200e500 2023-10-18T12:35:42.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012011c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:35:42.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012010010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007b30 tx=0x7fe8c0009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=464ec0d19f703fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.466+0000 7fe8bd3ff700 1 -- 172.21.15.70:0/707948789 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000e030 con 0x55601200e500 2023-10-18T12:35:42.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8bd3ff700 1 -- 172.21.15.70:0/707948789 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c000f040 con 0x55601200e500 2023-10-18T12:35:42.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8bd3ff700 1 -- 172.21.15.70:0/707948789 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0015c80 con 0x55601200e500 2023-10-18T12:35:42.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/707948789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 msgr2=0x556012010010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012010010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007b30 tx=0x7fe8c0009990 comp rx=0 tx=0).stop 2023-10-18T12:35:42.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/707948789 shutdown_connections 2023-10-18T12:35:42.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200e500 0x556012010010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x556012011c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/707948789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200dbe0 0x556012018f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/707948789 >> 172.21.15.70:0/707948789 conn(0x556011f7c720 msgr2=0x5560120078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/707948789 shutdown_connections 2023-10-18T12:35:42.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.467+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/707948789 wait complete. 2023-10-18T12:35:42.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 -- start start 2023-10-18T12:35:42.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 Processor -- start 2023-10-18T12:35:42.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x5560120178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200dbe0 0x556012017df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012018330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556012075eb0 con 0x5560120085d0 2023-10-18T12:35:42.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.468+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556012076020 con 0x55601200e500 2023-10-18T12:35:42.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8e3d93b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560120761e0 con 0x55601200dbe0 2023-10-18T12:35:42.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8e3d93b80 10 client.0 _ll_register_callbacks cb 0x556011e41a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:35:42.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012018330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012018330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:54828/0 (socket says 172.21.15.70:54828) 2023-10-18T12:35:42.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3313809138 learned_addr learned my addr 172.21.15.70:0/3313809138 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:35:42.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x5560120178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8cd99a700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200dbe0 0x556012017df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3313809138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200dbe0 msgr2=0x556012017df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200dbe0 0x556012017df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3313809138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 msgr2=0x5560120178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:35:42.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x5560120178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 -- 172.21.15.70:0/3313809138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81999f0 con 0x55601200e500 2023-10-18T12:35:42.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8ce99c700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012018330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009960 tx=0x7fe8c0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba689c20ea24d0f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:35:42.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.469+0000 7fe8b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:35:42.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3313809138 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000e040 con 0x55601200e500 2023-10-18T12:35:42.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8b8021070 con 0x55601200e500 2023-10-18T12:35:42.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3313809138 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe8c0015c20 con 0x55601200e500 2023-10-18T12:35:42.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3313809138 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8c0015de0 con 0x55601200e500 2023-10-18T12:35:42.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 10 client.26963 fetch_fsmap learned FSMap version 20 2023-10-18T12:35:42.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556011d85de0 con 0x55601200e500 2023-10-18T12:35:42.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3313809138 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b8021070 con 0x55601200e500 2023-10-18T12:35:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3313809138 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8c0032610 con 0x55601200e500 2023-10-18T12:35:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 10 client.26963 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:35:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:35:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 20 client.26963 populate_metadata read hostname 'smithi070' 2023-10-18T12:35:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012077560 0x556012079980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:35:42.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 4 client.26963 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:35:42.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.470+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556011f53e50 con 0x556012077560 2023-10-18T12:35:42.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.471+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012077560 0x556012079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:35:42.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.471+0000 7fe8b1ffb700 10 client.26963 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.471+0000 7fe8ce19b700 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012077560 0x556012079980 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10dcad71ccb598a server_cookie=a940ce0d162e6e5 in_seq=0 out_seq=0 2023-10-18T12:35:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8b1ffb700 1 -- 172.21.15.70:0/3313809138 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556011f53e50 con 0x556012077560 2023-10-18T12:35:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8b1ffb700 10 client.26963 handle_command_reply: tid=0 2023-10-18T12:35:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 client.26963 shutdown 2023-10-18T12:35:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 msgr2=0x556012018330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012018330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009960 tx=0x7fe8c0009560 comp rx=0 tx=0).stop 2023-10-18T12:35:42.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 shutdown_connections 2023-10-18T12:35:42.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x556012077560 0x556012079980 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55601200e500 0x556012018330 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8b1ffb700 0 client.26963 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:35:42.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560120085d0 0x5560120178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 --2- 172.21.15.70:0/3313809138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55601200dbe0 0x556012017df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 >> 172.21.15.70:0/3313809138 conn(0x556011f7c720 msgr2=0x55601200ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 shutdown_connections 2023-10-18T12:35:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 1 -- 172.21.15.70:0/3313809138 wait complete. 2023-10-18T12:35:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.472+0000 7fe8e3d93b80 20 client.26963 trim_cache size 0 max 16384 2023-10-18T12:35:42.489 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:35:42.489 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:35:42.489 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:35:42.489 INFO:teuthology.orchestra.run.smithi070.stdout: "804e285d-9075-4a0f-9879-f34c943e2730": { 2023-10-18T12:35:42.490 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:35:42.490 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "804e285d-9075-4a0f-9879-f34c943e2730", 2023-10-18T12:35:42.490 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:35:42.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:35:42.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:35:42.491 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:35:42.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe8b40a1670 msgr2=0x7fe8b40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe8b40a1670 0x7fe8b40a3b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe8c819a230 tx=0x7fe8b8009260 comp rx=0 tx=0).stop 2023-10-18T12:35:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 msgr2=0x7fe8c819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:35:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 0x7fe8c819ea60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4005770 tx=0x7fe8c40057a0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 shutdown_connections 2023-10-18T12:35:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe8b40a1670 0x7fe8b40a3b30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8c8106730 0x7fe8c8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8100720 0x7fe8c819ea60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 --2- 172.21.15.70:0/3161344410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8c81010e0 0x7fe8c819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:35:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 >> 172.21.15.70:0/3161344410 conn(0x7fe8c80fc1b0 msgr2=0x7fe8c810be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:35:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 shutdown_connections 2023-10-18T12:35:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:35:42.473+0000 7fe8d03ff700 1 -- 172.21.15.70:0/3161344410 wait complete. 2023-10-18T12:35:42.495 INFO:tasks.cephfs.filesystem:scrub status for tag:804e285d-9075-4a0f-9879-f34c943e2730 - {'path': '/', 'tag': '804e285d-9075-4a0f-9879-f34c943e2730', 'options': 'recursive,force'} 2023-10-18T12:35:42.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:35:42.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:35:42.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:35:42.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: pgmap v611: 129 pgs: 129 active+clean; 592 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-18T12:35:42.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:35:42.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:42.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:35:42.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:42 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:35:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:35:42.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:35:42.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: pgmap v611: 129 pgs: 129 active+clean; 592 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-18T12:35:42.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:35:42.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:42.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:35:42.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:42 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:42.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:35:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:35:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:35:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: pgmap v611: 129 pgs: 129 active+clean; 592 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2023-10-18T12:35:42.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:35:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:35:42.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:42 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:35:42.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1099492 11.57 MB/sec execute 371 sec latency 17.353 ms 2023-10-18T12:35:43.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1101578 11.58 MB/sec execute 372 sec latency 49.210 ms 2023-10-18T12:35:44.872 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1103512 11.56 MB/sec execute 373 sec latency 27.040 ms 2023-10-18T12:35:45.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:44 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:35:45.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:44 smithi070 ceph-mon[98639]: pgmap v612: 129 pgs: 129 active+clean; 593 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 197 op/s 2023-10-18T12:35:45.213 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:44 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:35:45.213 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:44 smithi152 ceph-mon[134767]: pgmap v612: 129 pgs: 129 active+clean; 593 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 197 op/s 2023-10-18T12:35:45.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:44 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:35:45.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:44 smithi186 ceph-mon[135253]: pgmap v612: 129 pgs: 129 active+clean; 593 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 197 op/s 2023-10-18T12:35:45.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1105433 11.57 MB/sec execute 374 sec latency 46.692 ms 2023-10-18T12:35:46.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1107954 11.57 MB/sec execute 375 sec latency 17.140 ms 2023-10-18T12:35:47.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:46 smithi070 ceph-mon[98639]: pgmap v613: 129 pgs: 129 active+clean; 593 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 188 op/s 2023-10-18T12:35:47.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:46 smithi186 ceph-mon[135253]: pgmap v613: 129 pgs: 129 active+clean; 593 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 188 op/s 2023-10-18T12:35:47.782 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:46 smithi152 ceph-mon[134767]: pgmap v613: 129 pgs: 129 active+clean; 593 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 188 op/s 2023-10-18T12:35:47.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1109765 11.57 MB/sec execute 376 sec latency 13.911 ms 2023-10-18T12:35:48.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1112005 11.57 MB/sec execute 377 sec latency 44.226 ms 2023-10-18T12:35:49.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:48 smithi070 ceph-mon[98639]: pgmap v614: 129 pgs: 129 active+clean; 604 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 241 op/s 2023-10-18T12:35:49.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:48 smithi152 ceph-mon[134767]: pgmap v614: 129 pgs: 129 active+clean; 604 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 241 op/s 2023-10-18T12:35:49.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:48 smithi186 ceph-mon[135253]: pgmap v614: 129 pgs: 129 active+clean; 604 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 241 op/s 2023-10-18T12:35:49.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1113476 11.56 MB/sec execute 378 sec latency 26.945 ms 2023-10-18T12:35:50.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:50 smithi070 ceph-mon[98639]: pgmap v615: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2023-10-18T12:35:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:50 smithi152 ceph-mon[134767]: pgmap v615: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2023-10-18T12:35:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:50 smithi186 ceph-mon[135253]: pgmap v615: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2023-10-18T12:35:50.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1115915 11.56 MB/sec execute 379 sec latency 22.946 ms 2023-10-18T12:35:51.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1118299 11.57 MB/sec execute 380 sec latency 25.763 ms 2023-10-18T12:35:52.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1120322 11.57 MB/sec execute 381 sec latency 13.680 ms 2023-10-18T12:35:53.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:52 smithi070 ceph-mon[98639]: pgmap v616: 129 pgs: 129 active+clean; 608 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 210 op/s 2023-10-18T12:35:53.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:52 smithi152 ceph-mon[134767]: pgmap v616: 129 pgs: 129 active+clean; 608 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 210 op/s 2023-10-18T12:35:53.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:52 smithi186 ceph-mon[135253]: pgmap v616: 129 pgs: 129 active+clean; 608 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 210 op/s 2023-10-18T12:35:53.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1122379 11.57 MB/sec execute 382 sec latency 126.786 ms 2023-10-18T12:35:54.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1123926 11.56 MB/sec execute 383 sec latency 25.135 ms 2023-10-18T12:35:55.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:54 smithi070 ceph-mon[98639]: pgmap v617: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2023-10-18T12:35:55.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:54 smithi152 ceph-mon[134767]: pgmap v617: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2023-10-18T12:35:55.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:54 smithi186 ceph-mon[135253]: pgmap v617: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 197 op/s 2023-10-18T12:35:55.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1126397 11.56 MB/sec execute 384 sec latency 21.068 ms 2023-10-18T12:35:56.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1128445 11.55 MB/sec execute 385 sec latency 22.187 ms 2023-10-18T12:35:57.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:56 smithi070 ceph-mon[98639]: pgmap v618: 129 pgs: 129 active+clean; 603 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 204 op/s 2023-10-18T12:35:57.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:56 smithi152 ceph-mon[134767]: pgmap v618: 129 pgs: 129 active+clean; 603 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 204 op/s 2023-10-18T12:35:57.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:56 smithi186 ceph-mon[135253]: pgmap v618: 129 pgs: 129 active+clean; 603 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 204 op/s 2023-10-18T12:35:57.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1130297 11.55 MB/sec execute 386 sec latency 49.649 ms 2023-10-18T12:35:58.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1132329 11.55 MB/sec execute 387 sec latency 91.533 ms 2023-10-18T12:35:59.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:35:58 smithi070 ceph-mon[98639]: pgmap v619: 129 pgs: 129 active+clean; 582 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2023-10-18T12:35:59.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:35:58 smithi152 ceph-mon[134767]: pgmap v619: 129 pgs: 129 active+clean; 582 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2023-10-18T12:35:59.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:35:58 smithi186 ceph-mon[135253]: pgmap v619: 129 pgs: 129 active+clean; 582 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2023-10-18T12:35:59.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1134089 11.54 MB/sec execute 388 sec latency 37.024 ms 2023-10-18T12:36:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:00 smithi070 ceph-mon[98639]: pgmap v620: 129 pgs: 129 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 195 op/s 2023-10-18T12:36:00.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:00 smithi152 ceph-mon[134767]: pgmap v620: 129 pgs: 129 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 195 op/s 2023-10-18T12:36:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:00 smithi186 ceph-mon[135253]: pgmap v620: 129 pgs: 129 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 195 op/s 2023-10-18T12:36:00.873 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1136350 11.55 MB/sec execute 389 sec latency 59.371 ms 2023-10-18T12:36:01.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1136980 11.53 MB/sec execute 390 sec latency 70.568 ms 2023-10-18T12:36:02.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1138698 11.52 MB/sec execute 391 sec latency 28.382 ms 2023-10-18T12:36:03.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:02 smithi070 ceph-mon[98639]: pgmap v621: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 237 op/s 2023-10-18T12:36:03.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:02 smithi152 ceph-mon[134767]: pgmap v621: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 237 op/s 2023-10-18T12:36:03.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:02 smithi186 ceph-mon[135253]: pgmap v621: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 237 op/s 2023-10-18T12:36:03.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1140673 11.52 MB/sec execute 392 sec latency 75.438 ms 2023-10-18T12:36:04.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1142571 11.51 MB/sec execute 393 sec latency 88.207 ms 2023-10-18T12:36:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:04 smithi152 ceph-mon[134767]: pgmap v622: 129 pgs: 129 active+clean; 579 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 238 op/s 2023-10-18T12:36:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:04 smithi152 ceph-mon[134767]: osdmap e100: 12 total, 12 up, 12 in 2023-10-18T12:36:05.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:04 smithi186 ceph-mon[135253]: pgmap v622: 129 pgs: 129 active+clean; 579 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 238 op/s 2023-10-18T12:36:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:04 smithi186 ceph-mon[135253]: osdmap e100: 12 total, 12 up, 12 in 2023-10-18T12:36:05.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:04 smithi070 ceph-mon[98639]: pgmap v622: 129 pgs: 129 active+clean; 579 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 238 op/s 2023-10-18T12:36:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:04 smithi070 ceph-mon[98639]: osdmap e100: 12 total, 12 up, 12 in 2023-10-18T12:36:05.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1144410 11.51 MB/sec execute 394 sec latency 37.690 ms 2023-10-18T12:36:06.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1146730 11.51 MB/sec execute 395 sec latency 16.738 ms 2023-10-18T12:36:07.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:06 smithi152 ceph-mon[134767]: pgmap v624: 129 pgs: 129 active+clean; 576 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 245 op/s 2023-10-18T12:36:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:06 smithi186 ceph-mon[135253]: pgmap v624: 129 pgs: 129 active+clean; 576 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 245 op/s 2023-10-18T12:36:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:06 smithi070 ceph-mon[98639]: pgmap v624: 129 pgs: 129 active+clean; 576 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 245 op/s 2023-10-18T12:36:07.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1148418 11.51 MB/sec execute 396 sec latency 29.689 ms 2023-10-18T12:36:08.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1151047 11.52 MB/sec execute 397 sec latency 17.900 ms 2023-10-18T12:36:09.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:08 smithi152 ceph-mon[134767]: pgmap v625: 129 pgs: 129 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 237 op/s 2023-10-18T12:36:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:08 smithi186 ceph-mon[135253]: pgmap v625: 129 pgs: 129 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 237 op/s 2023-10-18T12:36:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:08 smithi070 ceph-mon[98639]: pgmap v625: 129 pgs: 129 active+clean; 572 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 237 op/s 2023-10-18T12:36:09.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1152341 11.50 MB/sec execute 398 sec latency 84.270 ms 2023-10-18T12:36:10.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:10 smithi070 ceph-mon[98639]: pgmap v626: 129 pgs: 129 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2023-10-18T12:36:10.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:10 smithi152 ceph-mon[134767]: pgmap v626: 129 pgs: 129 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2023-10-18T12:36:10.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:10 smithi186 ceph-mon[135253]: pgmap v626: 129 pgs: 129 active+clean; 570 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2023-10-18T12:36:10.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1154745 11.51 MB/sec execute 399 sec latency 57.163 ms 2023-10-18T12:36:11.874 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1156773 11.50 MB/sec execute 400 sec latency 15.547 ms 2023-10-18T12:36:12.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:36:12.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 -- 172.21.15.70:0/2456410049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 msgr2=0x7f74d0106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 --2- 172.21.15.70:0/2456410049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0106b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009990 tx=0x7f74cc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 -- 172.21.15.70:0/2456410049 shutdown_connections 2023-10-18T12:36:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 --2- 172.21.15.70:0/2456410049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d01010e0 0x7f74d010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 --2- 172.21.15.70:0/2456410049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0106b10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 --2- 172.21.15.70:0/2456410049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74d0100720 0x7f74d0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 -- 172.21.15.70:0/2456410049 >> 172.21.15.70:0/2456410049 conn(0x7f74d00fc1b0 msgr2=0x7f74d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 -- 172.21.15.70:0/2456410049 shutdown_connections 2023-10-18T12:36:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.728+0000 7f74d79ff700 1 -- 172.21.15.70:0/2456410049 wait complete. 2023-10-18T12:36:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.729+0000 7f74d79ff700 1 Processor -- start 2023-10-18T12:36:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 -- start start 2023-10-18T12:36:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 0x7f74d0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74d01010e0 0x7f74d0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01a4f50 con 0x7f74d01010e0 2023-10-18T12:36:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01a50c0 con 0x7f74d0106730 2023-10-18T12:36:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d79ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01a5280 con 0x7f74d0100720 2023-10-18T12:36:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d579b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 0x7f74d0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d579b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 0x7f74d0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46142/0 (socket says 172.21.15.70:46142) 2023-10-18T12:36:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d579b700 1 -- 172.21.15.70:0/1618499033 learned_addr learned my addr 172.21.15.70:0/1618499033 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.730+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74d01010e0 0x7f74d0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d579b700 1 -- 172.21.15.70:0/1618499033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 msgr2=0x7f74d0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d579b700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d579b700 1 -- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74d01010e0 msgr2=0x7f74d0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d579b700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74d01010e0 0x7f74d0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d579b700 1 -- 172.21.15.70:0/1618499033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009520 con 0x7f74d0100720 2023-10-18T12:36:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d579b700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 0x7f74d0072af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74cc000c00 tx=0x7f74cc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74c67fc700 1 -- 172.21.15.70:0/1618499033 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc01d070 con 0x7f74d0100720 2023-10-18T12:36:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc0095f0 con 0x7f74d0100720 2023-10-18T12:36:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74c67fc700 1 -- 172.21.15.70:0/1618499033 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74cc022470 con 0x7f74d0100720 2023-10-18T12:36:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74c67fc700 1 -- 172.21.15.70:0/1618499033 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc017650 con 0x7f74d0100720 2023-10-18T12:36:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.731+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d01a5850 con 0x7f74d0100720 2023-10-18T12:36:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.734+0000 7f74c67fc700 1 -- 172.21.15.70:0/1618499033 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f74cc0225e0 con 0x7f74d0100720 2023-10-18T12:36:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74c67fc700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f74bc0a15f0 0x7f74bc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74c67fc700 1 -- 172.21.15.70:0/1618499033 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f74cc0ce810 con 0x7f74d0100720 2023-10-18T12:36:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74eb38eb80 1 Processor -- start 2023-10-18T12:36:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f74bc0a15f0 0x7f74bc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74eb38eb80 1 -- start start 2023-10-18T12:36:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a10fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a11500 0x55c9b5a0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.736+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f74bc0a15f0 0x7f74bc0a3ab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f74d0074220 tx=0x7f74c0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a0ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b597e6e0 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5964c60 con 0x55c9b5a10be0 2023-10-18T12:36:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5962580 con 0x55c9b5a11500 2023-10-18T12:36:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a10fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a10fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45170/0 (socket says 172.21.15.70:45170) 2023-10-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 -- 172.21.15.70:0/514081782 learned_addr learned my addr 172.21.15.70:0/514081782 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a11500 0x55c9b5a0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a0ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 -- 172.21.15.70:0/514081782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a11500 msgr2=0x55c9b5a0b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a11500 0x55c9b5a0b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 -- 172.21.15.70:0/514081782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a0b5d0 msgr2=0x55c9b5a0ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a0ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.737+0000 7f74d579b700 1 -- 172.21.15.70:0/514081782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b591b3f0 con 0x55c9b5a10be0 2023-10-18T12:36:12.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.738+0000 7f74d579b700 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a10fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74cc09b400 tx=0x7f74cc097830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1cf94ee9f64af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:12.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.738+0000 7f74bb7fe700 1 -- 172.21.15.70:0/514081782 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc09add0 con 0x55c9b5a10be0 2023-10-18T12:36:12.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.738+0000 7f74bb7fe700 1 -- 172.21.15.70:0/514081782 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74cc09bb70 con 0x55c9b5a10be0 2023-10-18T12:36:12.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.738+0000 7f74bb7fe700 1 -- 172.21.15.70:0/514081782 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0ce3e0 con 0x55c9b5a10be0 2023-10-18T12:36:12.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.738+0000 7f74eb38eb80 1 -- 172.21.15.70:0/514081782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 msgr2=0x55c9b5a10fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.738+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a10fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74cc09b400 tx=0x7f74cc097830 comp rx=0 tx=0).stop 2023-10-18T12:36:12.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 -- 172.21.15.70:0/514081782 shutdown_connections 2023-10-18T12:36:12.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a0ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a10fc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/514081782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a11500 0x55c9b5a0b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 -- 172.21.15.70:0/514081782 >> 172.21.15.70:0/514081782 conn(0x55c9b597f720 msgr2=0x55c9b5a05b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:12.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 -- 172.21.15.70:0/514081782 shutdown_connections 2023-10-18T12:36:12.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 -- 172.21.15.70:0/514081782 wait complete. 2023-10-18T12:36:12.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.739+0000 7f74eb38eb80 1 -- start start 2023-10-18T12:36:12.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 Processor -- start 2023-10-18T12:36:12.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a17400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b595de70 con 0x55c9b5a11500 2023-10-18T12:36:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b58fa710 con 0x55c9b5a10be0 2023-10-18T12:36:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b58fa3b0 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74eb38eb80 10 client.0 _ll_register_callbacks cb 0x55c9b5844a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74d4f9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74d4f9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45174/0 (socket says 172.21.15.70:45174) 2023-10-18T12:36:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74d4f9a700 1 -- 172.21.15.70:0/3698522903 learned_addr learned my addr 172.21.15.70:0/3698522903 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74d579b700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a17400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.740+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d4f9a700 1 -- 172.21.15.70:0/3698522903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 msgr2=0x55c9b5a17400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a17400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d4f9a700 1 -- 172.21.15.70:0/3698522903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 msgr2=0x55c9b5a14cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a14cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d4f9a700 1 -- 172.21.15.70:0/3698522903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d01a5850 con 0x55c9b5a10be0 2023-10-18T12:36:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d579b700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a17400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a146f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f74c000ec80 tx=0x7f74c00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763309abc8b59135 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.741+0000 7f74b8ff9700 1 -- 172.21.15.70:0/3698522903 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0022070 con 0x55c9b5a10be0 2023-10-18T12:36:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9b5962ef0 con 0x55c9b5a10be0 2023-10-18T12:36:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74b8ff9700 1 -- 172.21.15.70:0/3698522903 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f74c001daf0 con 0x55c9b5a10be0 2023-10-18T12:36:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74b8ff9700 1 -- 172.21.15.70:0/3698522903 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c001dcb0 con 0x55c9b5a10be0 2023-10-18T12:36:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 10 client.26975 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c9b5788de0 con 0x55c9b5a10be0 2023-10-18T12:36:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74b8ff9700 1 -- 172.21.15.70:0/3698522903 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9b5962ef0 con 0x55c9b5a10be0 2023-10-18T12:36:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74b8ff9700 1 -- 172.21.15.70:0/3698522903 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74c0003c10 con 0x55c9b5a10be0 2023-10-18T12:36:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 10 client.26975 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 20 client.26975 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a24b70 0x55c9b5a26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 4 client.26975 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.742+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9b59da8b0 con 0x55c9b5a24b70 2023-10-18T12:36:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.743+0000 7f74d579b700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a24b70 0x55c9b5a26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.743+0000 7f74b8ff9700 10 client.26975 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.743+0000 7f74d579b700 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a24b70 0x55c9b5a26f90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afddd28a58de7d1f server_cookie=4cfb4e5259ba30e9 in_seq=0 out_seq=0 2023-10-18T12:36:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74b8ff9700 1 -- 172.21.15.70:0/3698522903 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9b59da8b0 con 0x55c9b5a24b70 2023-10-18T12:36:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74b8ff9700 10 client.26975 handle_command_reply: tid=0 2023-10-18T12:36:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 client.26975 shutdown 2023-10-18T12:36:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 msgr2=0x55c9b5a146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a146f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f74c000ec80 tx=0x7f74c00047c0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 shutdown_connections 2023-10-18T12:36:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a24b70 0x55c9b5a26f90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a14cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a17400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74b8ff9700 0 client.26975 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/3698522903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a146f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 >> 172.21.15.70:0/3698522903 conn(0x55c9b597f720 msgr2=0x55c9b5a0a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 shutdown_connections 2023-10-18T12:36:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 1 -- 172.21.15.70:0/3698522903 wait complete. 2023-10-18T12:36:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.747+0000 7f74eb38eb80 20 client.26975 trim_cache size 0 max 16384 2023-10-18T12:36:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.768+0000 7f74eb38eb80 1 Processor -- start 2023-10-18T12:36:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.768+0000 7f74eb38eb80 1 -- start start 2023-10-18T12:36:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.768+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a1bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a10be0 0x55c9b5a0a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a0e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b57de920 con 0x55c9b5a11500 2023-10-18T12:36:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5a15da0 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5a02550 con 0x55c9b5a10be0 2023-10-18T12:36:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d4f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a10be0 0x55c9b5a0a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d579b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a1bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d4f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a10be0 0x55c9b5a0a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46178/0 (socket says 172.21.15.70:46178) 2023-10-18T12:36:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d4f9a700 1 -- 172.21.15.70:0/590146585 learned_addr learned my addr 172.21.15.70:0/590146585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a0e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d579b700 1 -- 172.21.15.70:0/590146585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a10be0 msgr2=0x55c9b5a0a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d579b700 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a10be0 0x55c9b5a0a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d579b700 1 -- 172.21.15.70:0/590146585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 msgr2=0x55c9b5a0e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d579b700 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a0e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.769+0000 7f74d579b700 1 -- 172.21.15.70:0/590146585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b5771c60 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.770+0000 7f74d579b700 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a1bc30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02d000 tx=0x7f74cc09e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b672c95c7a3b5293 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.771+0000 7f74bbfff700 1 -- 172.21.15.70:0/590146585 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0b9040 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.771+0000 7f74bbfff700 1 -- 172.21.15.70:0/590146585 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74cc09f5c0 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.771+0000 7f74bbfff700 1 -- 172.21.15.70:0/590146585 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0b3ea0 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 -- 172.21.15.70:0/590146585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a0b5d0 msgr2=0x55c9b5a1bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a1bc30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02d000 tx=0x7f74cc09e040 comp rx=0 tx=0).stop 2023-10-18T12:36:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 -- 172.21.15.70:0/590146585 shutdown_connections 2023-10-18T12:36:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a0e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a1bc30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/590146585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a10be0 0x55c9b5a0a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 -- 172.21.15.70:0/590146585 >> 172.21.15.70:0/590146585 conn(0x55c9b597f720 msgr2=0x55c9b5a0b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 -- 172.21.15.70:0/590146585 shutdown_connections 2023-10-18T12:36:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 -- 172.21.15.70:0/590146585 wait complete. 2023-10-18T12:36:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.772+0000 7f74eb38eb80 1 -- start start 2023-10-18T12:36:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 Processor -- start 2023-10-18T12:36:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a13480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a11e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5a12530 con 0x55c9b5a11500 2023-10-18T12:36:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5a126a0 con 0x55c9b5a10be0 2023-10-18T12:36:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5a811e0 con 0x55c9b5a0b5d0 2023-10-18T12:36:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.773+0000 7f74eb38eb80 10 client.0 _ll_register_callbacks cb 0x55c9b5844a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d579b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d579b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a13480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46194/0 (socket says 172.21.15.70:46194) 2023-10-18T12:36:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d579b700 1 -- 172.21.15.70:0/4177221513 learned_addr learned my addr 172.21.15.70:0/4177221513 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a11e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d4f9a700 1 -- 172.21.15.70:0/4177221513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 msgr2=0x55c9b5a13480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a13480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.774+0000 7f74d4f9a700 1 -- 172.21.15.70:0/4177221513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 msgr2=0x55c9b5a11e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a11e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74d4f9a700 1 -- 172.21.15.70:0/4177221513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9b591b3f0 con 0x55c9b5a10be0 2023-10-18T12:36:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74d5f9c700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a11e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74d4f9a700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a118e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55c9b5a21560 tx=0x7f74c0009c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e0c2fcacc14235 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74b8ff9700 1 -- 172.21.15.70:0/4177221513 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0022040 con 0x55c9b5a10be0 2023-10-18T12:36:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74cc09bb70 con 0x55c9b5a10be0 2023-10-18T12:36:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74b8ff9700 1 -- 172.21.15.70:0/4177221513 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f74c0026c20 con 0x55c9b5a10be0 2023-10-18T12:36:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.775+0000 7f74b8ff9700 1 -- 172.21.15.70:0/4177221513 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c001dea0 con 0x55c9b5a10be0 2023-10-18T12:36:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 10 client.26981 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c9b57af250 con 0x55c9b5a10be0 2023-10-18T12:36:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74b8ff9700 1 -- 172.21.15.70:0/4177221513 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74cc09bb70 con 0x55c9b5a10be0 2023-10-18T12:36:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74b8ff9700 1 -- 172.21.15.70:0/4177221513 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74c0003680 con 0x55c9b5a10be0 2023-10-18T12:36:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 10 client.26981 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 20 client.26981 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a82900 0x55c9b5a84d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 4 client.26981 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:36:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c9b5956e50 con 0x55c9b5a82900 2023-10-18T12:36:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.776+0000 7f74d579b700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a82900 0x55c9b5a84d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74d579b700 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a82900 0x55c9b5a84d20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592649245c803418 server_cookie=55cc1694bc8e24ad in_seq=0 out_seq=0 2023-10-18T12:36:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74b8ff9700 10 client.26981 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74b8ff9700 1 -- 172.21.15.70:0/4177221513 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c9b5956e50 con 0x55c9b5a82900 2023-10-18T12:36:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74b8ff9700 10 client.26981 handle_command_reply: tid=0 2023-10-18T12:36:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 client.26981 shutdown 2023-10-18T12:36:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 msgr2=0x55c9b5a118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a118e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55c9b5a21560 tx=0x7f74c0009c70 comp rx=0 tx=0).stop 2023-10-18T12:36:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 shutdown_connections 2023-10-18T12:36:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c9b5a82900 0x55c9b5a84d20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c9b5a11500 0x55c9b5a11e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9b5a0b5d0 0x55c9b5a13480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 --2- 172.21.15.70:0/4177221513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c9b5a10be0 0x55c9b5a118e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 >> 172.21.15.70:0/4177221513 conn(0x55c9b597f720 msgr2=0x55c9b5a0e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.777+0000 7f74b8ff9700 0 client.26981 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.778+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 shutdown_connections 2023-10-18T12:36:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.778+0000 7f74eb38eb80 1 -- 172.21.15.70:0/4177221513 wait complete. 2023-10-18T12:36:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.778+0000 7f74eb38eb80 20 client.26981 trim_cache size 0 max 16384 2023-10-18T12:36:12.793 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:36:12.793 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:36:12.793 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:36:12.793 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:36:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f74bc0a15f0 msgr2=0x7f74bc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f74bc0a15f0 0x7f74bc0a3ab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f74d0074220 tx=0x7f74c0009260 comp rx=0 tx=0).stop 2023-10-18T12:36:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 msgr2=0x7f74d0072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 0x7f74d0072af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74cc000c00 tx=0x7f74cc005740 comp rx=0 tx=0).stop 2023-10-18T12:36:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 shutdown_connections 2023-10-18T12:36:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f74bc0a15f0 0x7f74bc0a3ab0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d0106730 0x7f74d0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d0100720 0x7f74d0072af0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 --2- 172.21.15.70:0/1618499033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74d01010e0 0x7f74d0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 >> 172.21.15.70:0/1618499033 conn(0x7f74d00fc1b0 msgr2=0x7f74d00fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 shutdown_connections 2023-10-18T12:36:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:12.779+0000 7f74d79ff700 1 -- 172.21.15.70:0/1618499033 wait complete. 2023-10-18T12:36:12.803 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:36:12.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:36:12.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1158734 11.50 MB/sec execute 401 sec latency 12.031 ms 2023-10-18T12:36:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 -- 172.21.15.70:0/941465120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd8106700 msgr2=0x7f5fd8106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/941465120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd8106700 0x7f5fd8106ae0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009990 tx=0x7f5fd4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 -- 172.21.15.70:0/941465120 shutdown_connections 2023-10-18T12:36:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/941465120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fd81010b0 0x7f5fd810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/941465120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd8106700 0x7f5fd8106ae0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/941465120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81006f0 0x7f5fd8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.073+0000 7f5fe05ff700 1 -- 172.21.15.70:0/941465120 >> 172.21.15.70:0/941465120 conn(0x7f5fd80fc1a0 msgr2=0x7f5fd80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.074+0000 7f5fe05ff700 1 -- 172.21.15.70:0/941465120 shutdown_connections 2023-10-18T12:36:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.074+0000 7f5fe05ff700 1 -- 172.21.15.70:0/941465120 wait complete. 2023-10-18T12:36:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.075+0000 7f5fe05ff700 1 Processor -- start 2023-10-18T12:36:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.075+0000 7f5fe05ff700 1 -- start start 2023-10-18T12:36:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.075+0000 7f5fe05ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd81006f0 0x7f5fd8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.075+0000 7f5fe05ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 0x7f5fd8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.075+0000 7f5fe05ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fd8106700 0x7f5fd819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fe05ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd819cb50 con 0x7f5fd81006f0 2023-10-18T12:36:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fe05ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd819ccc0 con 0x7f5fd81010b0 2023-10-18T12:36:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fe05ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd819ce80 con 0x7f5fd8106700 2023-10-18T12:36:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 0x7f5fd8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fde39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd81006f0 0x7f5fd8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 0x7f5fd8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45206/0 (socket says 172.21.15.70:45206) 2023-10-18T12:36:13.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 -- 172.21.15.70:0/1095466710 learned_addr learned my addr 172.21.15.70:0/1095466710 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:13.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fdeb9c700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fd8106700 0x7f5fd819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 -- 172.21.15.70:0/1095466710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fd8106700 msgr2=0x7f5fd819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fd8106700 0x7f5fd819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 -- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd81006f0 msgr2=0x7f5fd8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd81006f0 0x7f5fd8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fddb9a700 1 -- 172.21.15.70:0/1095466710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40095f0 con 0x7f5fd81010b0 2023-10-18T12:36:13.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.076+0000 7f5fde39b700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd81006f0 0x7f5fd8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:13.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.077+0000 7f5fddb9a700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 0x7f5fd8199260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800d980 tx=0x7f5fc800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.077+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004220 con 0x7f5fd81010b0 2023-10-18T12:36:13.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.077+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd819d160 con 0x7f5fd81010b0 2023-10-18T12:36:13.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.077+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc8007640 con 0x7f5fd81010b0 2023-10-18T12:36:13.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.077+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8005690 con 0x7f5fd81010b0 2023-10-18T12:36:13.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.077+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd81a5ac0 con 0x7f5fd81010b0 2023-10-18T12:36:13.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.078+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd8101e30 con 0x7f5fd81010b0 2023-10-18T12:36:13.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.080+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5fc80447c0 con 0x7f5fd81010b0 2023-10-18T12:36:13.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.081+0000 7f5fcf7fe700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5fc40a15f0 0x7f5fc40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.081+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f5fc80cd5a0 con 0x7f5fd81010b0 2023-10-18T12:36:13.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.081+0000 7f5fde39b700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5fc40a15f0 0x7f5fc40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.082+0000 7f5fde39b700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5fc40a15f0 0x7f5fc40a3ab0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006010 tx=0x7f5fd400b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.084+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5fc80964b0 con 0x7f5fd81010b0 2023-10-18T12:36:13.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:12 smithi070 ceph-mon[98639]: pgmap v627: 129 pgs: 129 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2023-10-18T12:36:13.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:12 smithi152 ceph-mon[134767]: pgmap v627: 129 pgs: 129 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2023-10-18T12:36:13.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:12 smithi186 ceph-mon[135253]: pgmap v627: 129 pgs: 129 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2023-10-18T12:36:13.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.380+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd819a050 con 0x7f5fd81010b0 2023-10-18T12:36:13.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.381+0000 7f5fcf7fe700 1 -- 172.21.15.70:0/1095466710 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f5fd819a050 con 0x7f5fd81010b0 2023-10-18T12:36:13.384 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:36:13.384 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:36:13.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5fc40a15f0 msgr2=0x7f5fc40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5fc40a15f0 0x7f5fc40a3ab0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006010 tx=0x7f5fd400b540 comp rx=0 tx=0).stop 2023-10-18T12:36:13.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 msgr2=0x7f5fd8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 0x7f5fd8199260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800d980 tx=0x7f5fc800dc90 comp rx=0 tx=0).stop 2023-10-18T12:36:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 shutdown_connections 2023-10-18T12:36:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fd8106700 0x7f5fd819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd81006f0 0x7f5fd8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5fc40a15f0 0x7f5fc40a3ab0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 --2- 172.21.15.70:0/1095466710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fd81010b0 0x7f5fd8199260 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 >> 172.21.15.70:0/1095466710 conn(0x7f5fd80fc1a0 msgr2=0x7f5fd810bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 shutdown_connections 2023-10-18T12:36:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.384+0000 7f5fe05ff700 1 -- 172.21.15.70:0/1095466710 wait complete. 2023-10-18T12:36:13.390 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:36:13.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:36:13.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 -- 172.21.15.70:0/3459507369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd0100720 msgr2=0x7f6fd0100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/3459507369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd0100720 0x7f6fd0100ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009990 tx=0x7f6fbc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 -- 172.21.15.70:0/3459507369 shutdown_connections 2023-10-18T12:36:13.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/3459507369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd01010e0 0x7f6fd010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/3459507369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fd0106730 0x7f6fd0106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/3459507369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd0100720 0x7f6fd0100ba0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 -- 172.21.15.70:0/3459507369 >> 172.21.15.70:0/3459507369 conn(0x7f6fd00fc1b0 msgr2=0x7f6fd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 -- 172.21.15.70:0/3459507369 shutdown_connections 2023-10-18T12:36:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.615+0000 7f6fd6dff700 1 -- 172.21.15.70:0/3459507369 wait complete. 2023-10-18T12:36:13.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.616+0000 7f6fd6dff700 1 Processor -- start 2023-10-18T12:36:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 -- start start 2023-10-18T12:36:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fd0100720 0x7f6fd0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 0x7f6fd01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd0106730 0x7f6fd019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd019dc90 con 0x7f6fd0100720 2023-10-18T12:36:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd019de00 con 0x7f6fd01010e0 2023-10-18T12:36:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.617+0000 7f6fd6dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd019dfc0 con 0x7f6fd0106730 2023-10-18T12:36:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 0x7f6fd01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fd0100720 0x7f6fd0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 0x7f6fd01992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45210/0 (socket says 172.21.15.70:45210) 2023-10-18T12:36:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 -- 172.21.15.70:0/4032283156 learned_addr learned my addr 172.21.15.70:0/4032283156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fd539c700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd0106730 0x7f6fd019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 -- 172.21.15.70:0/4032283156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd0106730 msgr2=0x7f6fd019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd0106730 0x7f6fd019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 -- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fd0100720 msgr2=0x7f6fd0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fd0100720 0x7f6fd0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 -- 172.21.15.70:0/4032283156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0095f0 con 0x7f6fd01010e0 2023-10-18T12:36:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.618+0000 7f6fcffff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 0x7f6fd01992b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c00 tx=0x7f6fbc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.619+0000 7f6fcdffb700 1 -- 172.21.15.70:0/4032283156 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc01d070 con 0x7f6fd01010e0 2023-10-18T12:36:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.619+0000 7f6fcdffb700 1 -- 172.21.15.70:0/4032283156 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fbc022470 con 0x7f6fd01010e0 2023-10-18T12:36:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.619+0000 7f6fcdffb700 1 -- 172.21.15.70:0/4032283156 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc017650 con 0x7f6fd01010e0 2023-10-18T12:36:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.619+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd019e240 con 0x7f6fd01010e0 2023-10-18T12:36:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.619+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd01a5ae0 con 0x7f6fd01010e0 2023-10-18T12:36:13.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.622+0000 7f6fcdffb700 1 -- 172.21.15.70:0/4032283156 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6fbc0225e0 con 0x7f6fd01010e0 2023-10-18T12:36:13.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.623+0000 7f6fcdffb700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6fc00a1670 0x7f6fc00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 Processor -- start 2023-10-18T12:36:13.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fcdffb700 1 -- 172.21.15.70:0/4032283156 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f6fbc0ce6c0 con 0x7f6fd01010e0 2023-10-18T12:36:13.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 -- start start 2023-10-18T12:36:13.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6fc00a1670 0x7f6fc00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec5666e0 con 0x5598ec5f9500 2023-10-18T12:36:13.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec54cc60 con 0x5598ec5f35d0 2023-10-18T12:36:13.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.624+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec54a580 con 0x5598ec5f8be0 2023-10-18T12:36:13.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd539c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd539c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45212/0 (socket says 172.21.15.70:45212) 2023-10-18T12:36:13.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46248/0 (socket says 172.21.15.70:46248) 2023-10-18T12:36:13.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd539c700 1 -- 172.21.15.70:0/37775790 learned_addr learned my addr 172.21.15.70:0/37775790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:13.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6fc00a1670 0x7f6fc00a3b30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009c30 tx=0x7f6fc4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/37775790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 msgr2=0x5598ec5f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/37775790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 msgr2=0x5598ec5f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/37775790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ec5033f0 con 0x5598ec5f8be0 2023-10-18T12:36:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5f8fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4012c40 tx=0x7f6fc4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7506f652e4116d47 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fbaffd700 1 -- 172.21.15.70:0/37775790 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011d00 con 0x5598ec5f8be0 2023-10-18T12:36:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fbaffd700 1 -- 172.21.15.70:0/37775790 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc4011e60 con 0x5598ec5f8be0 2023-10-18T12:36:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.625+0000 7f6fbaffd700 1 -- 172.21.15.70:0/37775790 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4005020 con 0x5598ec5f8be0 2023-10-18T12:36:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 -- 172.21.15.70:0/37775790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 msgr2=0x5598ec5f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5f8fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4012c40 tx=0x7f6fc4014000 comp rx=0 tx=0).stop 2023-10-18T12:36:13.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 -- 172.21.15.70:0/37775790 shutdown_connections 2023-10-18T12:36:13.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5f8fc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/37775790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 -- 172.21.15.70:0/37775790 >> 172.21.15.70:0/37775790 conn(0x5598ec567720 msgr2=0x5598ec5edb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 -- 172.21.15.70:0/37775790 shutdown_connections 2023-10-18T12:36:13.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.626+0000 7f6fea66eb80 1 -- 172.21.15.70:0/37775790 wait complete. 2023-10-18T12:36:13.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.627+0000 7f6fea66eb80 1 -- start start 2023-10-18T12:36:13.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.627+0000 7f6fea66eb80 1 Processor -- start 2023-10-18T12:36:13.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f35d0 0x5598ec5ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f9500 0x5598ec5fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec545e70 con 0x5598ec5f35d0 2023-10-18T12:36:13.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec4e2710 con 0x5598ec5f9500 2023-10-18T12:36:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec4e23b0 con 0x5598ec5f8be0 2023-10-18T12:36:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fea66eb80 10 client.0 _ll_register_callbacks cb 0x5598ec42ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f35d0 0x5598ec5ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46262/0 (socket says 172.21.15.70:46262) 2023-10-18T12:36:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fcffff700 1 -- 172.21.15.70:0/945945032 learned_addr learned my addr 172.21.15.70:0/945945032 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd539c700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f9500 0x5598ec5fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/945945032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 msgr2=0x5598ec5fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f9500 msgr2=0x5598ec5fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f9500 0x5598ec5fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/945945032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc4017050 con 0x5598ec5f35d0 2023-10-18T12:36:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fcffff700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f35d0 0x5598ec5ff410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4004b60 tx=0x7f6fc4011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d60c088e082afb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.628+0000 7f6faffff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:36:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6faffff700 1 -- 172.21.15.70:0/945945032 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402b070 con 0x5598ec5f35d0 2023-10-18T12:36:13.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598ec54aef0 con 0x5598ec5f35d0 2023-10-18T12:36:13.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6faffff700 1 -- 172.21.15.70:0/945945032 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f6fc401c960 con 0x5598ec5f35d0 2023-10-18T12:36:13.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6faffff700 1 -- 172.21.15.70:0/945945032 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc401cb60 con 0x5598ec5f35d0 2023-10-18T12:36:13.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6fea66eb80 10 client.16953 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:13.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5598ec370de0 con 0x5598ec5f35d0 2023-10-18T12:36:13.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6faffff700 1 -- 172.21.15.70:0/945945032 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598ec54aef0 con 0x5598ec5f35d0 2023-10-18T12:36:13.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6faffff700 1 -- 172.21.15.70:0/945945032 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6fc4024430 con 0x5598ec5f35d0 2023-10-18T12:36:13.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.629+0000 7f6fea66eb80 10 client.16953 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:13.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fea66eb80 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:13.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fea66eb80 20 client.16953 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:13.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec60cd40 0x5598ec60f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fea66eb80 4 client.16953 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:13.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598ec5c28b0 con 0x5598ec60cd40 2023-10-18T12:36:13.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fcffff700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec60cd40 0x5598ec60f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6faffff700 10 client.16953 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:13.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.630+0000 7f6fcffff700 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec60cd40 0x5598ec60f160 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d05d69900851f2 server_cookie=9ff1c88ff588193b in_seq=0 out_seq=0 2023-10-18T12:36:13.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.631+0000 7f6faffff700 1 -- 172.21.15.70:0/945945032 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598ec5c28b0 con 0x5598ec60cd40 2023-10-18T12:36:13.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.631+0000 7f6faffff700 10 client.16953 handle_command_reply: tid=0 2023-10-18T12:36:13.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.631+0000 7f6fea66eb80 1 client.16953 shutdown 2023-10-18T12:36:13.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f35d0 msgr2=0x5598ec5ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f35d0 0x5598ec5ff410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4004b60 tx=0x7f6fc4011b80 comp rx=0 tx=0).stop 2023-10-18T12:36:13.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 shutdown_connections 2023-10-18T12:36:13.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec60cd40 0x5598ec60f160 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f9500 0x5598ec5fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f35d0 0x5598ec5ff410 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/945945032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec5fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6faffff700 0 client.16953 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:13.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 >> 172.21.15.70:0/945945032 conn(0x5598ec567720 msgr2=0x5598ec5f2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 shutdown_connections 2023-10-18T12:36:13.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 1 -- 172.21.15.70:0/945945032 wait complete. 2023-10-18T12:36:13.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.632+0000 7f6fea66eb80 20 client.16953 trim_cache size 0 max 16384 2023-10-18T12:36:13.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.645+0000 7f6fea66eb80 1 Processor -- start 2023-10-18T12:36:13.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.645+0000 7f6fea66eb80 1 -- start start 2023-10-18T12:36:13.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec603eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec3c6920 con 0x5598ec5f9500 2023-10-18T12:36:13.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec5ee0a0 con 0x5598ec5f35d0 2023-10-18T12:36:13.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec6043f0 con 0x5598ec5f8be0 2023-10-18T12:36:13.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec603eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec603eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46270/0 (socket says 172.21.15.70:46270) 2023-10-18T12:36:13.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45228/0 (socket says 172.21.15.70:45228) 2023-10-18T12:36:13.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 -- 172.21.15.70:0/1991889212 learned_addr learned my addr 172.21.15.70:0/1991889212 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:13.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fd539c700 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 -- 172.21.15.70:0/1991889212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 msgr2=0x5598ec5ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 -- 172.21.15.70:0/1991889212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 msgr2=0x5598ec5f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fcffff700 1 -- 172.21.15.70:0/1991889212 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ec359c60 con 0x5598ec5f8be0 2023-10-18T12:36:13.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fd539c700 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:13.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.646+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:13.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fcffff700 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec603eb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc006010 tx=0x7f6fbc01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eefd5284b69113f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fbb7fe700 1 -- 172.21.15.70:0/1991889212 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0b7070 con 0x5598ec5f8be0 2023-10-18T12:36:13.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fbb7fe700 1 -- 172.21.15.70:0/1991889212 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fbc097040 con 0x5598ec5f8be0 2023-10-18T12:36:13.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fbb7fe700 1 -- 172.21.15.70:0/1991889212 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0a3740 con 0x5598ec5f8be0 2023-10-18T12:36:13.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1991889212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 msgr2=0x5598ec603eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec603eb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc006010 tx=0x7f6fbc01a040 comp rx=0 tx=0).stop 2023-10-18T12:36:13.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1991889212 shutdown_connections 2023-10-18T12:36:13.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec5f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f35d0 0x5598ec5ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1991889212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f8be0 0x5598ec603eb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.647+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1991889212 >> 172.21.15.70:0/1991889212 conn(0x5598ec567720 msgr2=0x5598ec5f39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.648+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1991889212 shutdown_connections 2023-10-18T12:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.648+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1991889212 wait complete. 2023-10-18T12:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.648+0000 7f6fea66eb80 1 -- start start 2023-10-18T12:36:13.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.648+0000 7f6fea66eb80 1 Processor -- start 2023-10-18T12:36:13.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 0x5598ec5f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f8be0 0x5598ec5f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec65fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec5f8590 con 0x5598ec5f9500 2023-10-18T12:36:13.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec5f8700 con 0x5598ec5f8be0 2023-10-18T12:36:13.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec660150 con 0x5598ec5f35d0 2023-10-18T12:36:13.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fea66eb80 10 client.0 _ll_register_callbacks cb 0x5598ec42ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:13.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 0x5598ec5f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 0x5598ec5f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46274/0 (socket says 172.21.15.70:46274) 2023-10-18T12:36:13.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/1380420094 learned_addr learned my addr 172.21.15.70:0/1380420094 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:13.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fcffff700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f8be0 0x5598ec5f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd539c700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec65fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f8be0 msgr2=0x5598ec5f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f8be0 0x5598ec5f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/1380420094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 msgr2=0x5598ec65fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:13.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec65fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 -- 172.21.15.70:0/1380420094 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598ec370de0 con 0x5598ec5f35d0 2023-10-18T12:36:13.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd539c700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec65fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:13.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6fd4b9b700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 0x5598ec5f7850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4016040 tx=0x7f6fc40092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40001b920e24f673 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:13.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.649+0000 7f6faffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:36:13.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6faffff700 1 -- 172.21.15.70:0/1380420094 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402b040 con 0x5598ec5f35d0 2023-10-18T12:36:13.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6faffff700 1 -- 172.21.15.70:0/1380420094 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f6fc401cd70 con 0x5598ec5f35d0 2023-10-18T12:36:13.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fc4011e60 con 0x5598ec5f35d0 2023-10-18T12:36:13.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6faffff700 1 -- 172.21.15.70:0/1380420094 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc4011fd0 con 0x5598ec5f35d0 2023-10-18T12:36:13.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6fea66eb80 10 client.26989 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:13.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f6fbc0b3e70 con 0x5598ec5f35d0 2023-10-18T12:36:13.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.650+0000 7f6faffff700 1 -- 172.21.15.70:0/1380420094 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc4011e60 con 0x5598ec5f35d0 2023-10-18T12:36:13.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6faffff700 1 -- 172.21.15.70:0/1380420094 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6fc403e6f0 con 0x5598ec5f35d0 2023-10-18T12:36:13.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 10 client.26989 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:13.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:13.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 20 client.26989 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:13.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec6615f0 0x5598ec663a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:13.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 4 client.26989 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:36:13.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598ec53ee50 con 0x5598ec6615f0 2023-10-18T12:36:13.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fcffff700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec6615f0 0x5598ec663a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:13.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6faffff700 10 client.26989 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:13.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fcffff700 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec6615f0 0x5598ec663a10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3aa89c973e8b06 server_cookie=7101791c6b6853e6 in_seq=0 out_seq=0 2023-10-18T12:36:13.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6faffff700 1 -- 172.21.15.70:0/1380420094 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598ec53ee50 con 0x5598ec6615f0 2023-10-18T12:36:13.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6faffff700 10 client.26989 handle_command_reply: tid=0 2023-10-18T12:36:13.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.651+0000 7f6fea66eb80 1 client.26989 shutdown 2023-10-18T12:36:13.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 msgr2=0x5598ec5f7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 0x5598ec5f7850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4016040 tx=0x7f6fc40092d0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 shutdown_connections 2023-10-18T12:36:13.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5598ec6615f0 0x5598ec663a10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598ec5f9500 0x5598ec65fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5598ec5f35d0 0x5598ec5f7850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 --2- 172.21.15.70:0/1380420094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598ec5f8be0 0x5598ec5f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6faffff700 0 client.26989 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:13.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 >> 172.21.15.70:0/1380420094 conn(0x5598ec567720 msgr2=0x5598ec5fafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 shutdown_connections 2023-10-18T12:36:13.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 1 -- 172.21.15.70:0/1380420094 wait complete. 2023-10-18T12:36:13.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.652+0000 7f6fea66eb80 20 client.26989 trim_cache size 0 max 16384 2023-10-18T12:36:13.680 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:36:13.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6fc00a1670 msgr2=0x7f6fc00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6fc00a1670 0x7f6fc00a3b30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009c30 tx=0x7f6fc4009260 comp rx=0 tx=0).stop 2023-10-18T12:36:13.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 msgr2=0x7f6fd01992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:13.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 0x7f6fd01992b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c00 tx=0x7f6fbc005740 comp rx=0 tx=0).stop 2023-10-18T12:36:13.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 shutdown_connections 2023-10-18T12:36:13.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd0106730 0x7f6fd019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fd0100720 0x7f6fd0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6fc00a1670 0x7f6fc00a3b30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 --2- 172.21.15.70:0/4032283156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fd01010e0 0x7f6fd01992b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:13.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 >> 172.21.15.70:0/4032283156 conn(0x7f6fd00fc1b0 msgr2=0x7f6fd010bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:13.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.653+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 shutdown_connections 2023-10-18T12:36:13.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:13.654+0000 7f6fd6dff700 1 -- 172.21.15.70:0/4032283156 wait complete. 2023-10-18T12:36:13.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1160924 11.50 MB/sec execute 402 sec latency 44.196 ms 2023-10-18T12:36:14.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:13 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1095466710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:36:14.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:13 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1095466710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:36:14.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:13 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1095466710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:36:14.684 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:36:14.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:36:14.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1162275 11.49 MB/sec execute 403 sec latency 48.198 ms 2023-10-18T12:36:14.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 -- 172.21.15.70:0/609405942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 msgr2=0x7f0a78102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/609405942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78102db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009990 tx=0x7f0a6c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 -- 172.21.15.70:0/609405942 shutdown_connections 2023-10-18T12:36:14.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/609405942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a781032f0 0x7f0a7810f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/609405942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a78108940 0x7f0a78108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/609405942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78102db0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 -- 172.21.15.70:0/609405942 >> 172.21.15.70:0/609405942 conn(0x7f0a780fe3e0 msgr2=0x7f0a78100800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 -- 172.21.15.70:0/609405942 shutdown_connections 2023-10-18T12:36:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.907+0000 7f0a7ddff700 1 -- 172.21.15.70:0/609405942 wait complete. 2023-10-18T12:36:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.908+0000 7f0a7ddff700 1 Processor -- start 2023-10-18T12:36:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 -- start start 2023-10-18T12:36:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a781032f0 0x7f0a78199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a78108940 0x7f0a7819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819dce0 con 0x7f0a78108940 2023-10-18T12:36:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819de50 con 0x7f0a78102930 2023-10-18T12:36:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a7ddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819e010 con 0x7f0a781032f0 2023-10-18T12:36:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45258/0 (socket says 172.21.15.70:45258) 2023-10-18T12:36:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a777fe700 1 -- 172.21.15.70:0/3476444883 learned_addr learned my addr 172.21.15.70:0/3476444883 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a781032f0 0x7f0a78199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.909+0000 7f0a77fff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a78108940 0x7f0a7819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a777fe700 1 -- 172.21.15.70:0/3476444883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a781032f0 msgr2=0x7f0a78199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a777fe700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a781032f0 0x7f0a78199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a777fe700 1 -- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a78108940 msgr2=0x7f0a7819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a777fe700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a78108940 0x7f0a7819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a777fe700 1 -- 172.21.15.70:0/3476444883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0095f0 con 0x7f0a78102930 2023-10-18T12:36:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a777fe700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78198e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e9a0 tx=0x7f0a6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a74ff9700 1 -- 172.21.15.70:0/3476444883 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68009790 con 0x7f0a78102930 2023-10-18T12:36:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7819e2f0 con 0x7f0a78102930 2023-10-18T12:36:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a74ff9700 1 -- 172.21.15.70:0/3476444883 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68010430 con 0x7f0a78102930 2023-10-18T12:36:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a74ff9700 1 -- 172.21.15.70:0/3476444883 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68018680 con 0x7f0a78102930 2023-10-18T12:36:14.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.910+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a5b30 con 0x7f0a78102930 2023-10-18T12:36:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a74ff9700 1 -- 172.21.15.70:0/3476444883 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0a680105a0 con 0x7f0a78102930 2023-10-18T12:36:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 Processor -- start 2023-10-18T12:36:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 -- start start 2023-10-18T12:36:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a74ff9700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a600a15f0 0x7f0a600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119753760 con 0x55a1197e5c60 2023-10-18T12:36:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119739ce0 con 0x55a1197e0650 2023-10-18T12:36:14.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119737600 con 0x55a1197e6580 2023-10-18T12:36:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a74ff9700 1 -- 172.21.15.70:0/3476444883 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0a68014070 con 0x7f0a78102930 2023-10-18T12:36:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a77fff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a600a15f0 0x7f0a600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33058/0 (socket says 172.21.15.70:33058) 2023-10-18T12:36:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a777fe700 1 -- 172.21.15.70:0/1564476280 learned_addr learned my addr 172.21.15.70:0/1564476280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.915+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a77fff700 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a777fe700 1 -- 172.21.15.70:0/1564476280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 msgr2=0x55a1197e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a777fe700 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a777fe700 1 -- 172.21.15.70:0/1564476280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 msgr2=0x55a1197e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a777fe700 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a777fe700 1 -- 172.21.15.70:0/1564476280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1196f0470 con 0x55a1197e5c60 2023-10-18T12:36:14.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a77fff700 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:14.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a777fe700 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e6040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0a68096cb0 tx=0x7f0a68099ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea5a97ec1a40af server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:14.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a77fff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a600a15f0 0x7f0a600a3ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009ec0 tx=0x7f0a70009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a5dffb700 1 -- 172.21.15.70:0/1564476280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680be740 con 0x55a1197e5c60 2023-10-18T12:36:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a5dffb700 1 -- 172.21.15.70:0/1564476280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6809a940 con 0x55a1197e5c60 2023-10-18T12:36:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.916+0000 7f0a5dffb700 1 -- 172.21.15.70:0/1564476280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680cd200 con 0x55a1197e5c60 2023-10-18T12:36:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1564476280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 msgr2=0x55a1197e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e6040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0a68096cb0 tx=0x7f0a68099ba0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1564476280 shutdown_connections 2023-10-18T12:36:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e6040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1564476280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1564476280 >> 172.21.15.70:0/1564476280 conn(0x55a1197547a0 msgr2=0x55a1197dabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1564476280 shutdown_connections 2023-10-18T12:36:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.917+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1564476280 wait complete. 2023-10-18T12:36:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.918+0000 7f0a917c5b80 1 -- start start 2023-10-18T12:36:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.918+0000 7f0a917c5b80 1 Processor -- start 2023-10-18T12:36:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.918+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e0650 0x55a1197ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.918+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 0x55a1197e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119732ef0 con 0x55a1197e5c60 2023-10-18T12:36:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1196cf790 con 0x55a1197e6580 2023-10-18T12:36:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1196cf430 con 0x55a1197e0650 2023-10-18T12:36:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a917c5b80 10 client.0 _ll_register_callbacks cb 0x55a119619b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 0x55a1197e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e0650 0x55a1197ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 0x55a1197e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45278/0 (socket says 172.21.15.70:45278) 2023-10-18T12:36:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 -- 172.21.15.70:0/2800809326 learned_addr learned my addr 172.21.15.70:0/2800809326 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 -- 172.21.15.70:0/2800809326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e0650 msgr2=0x55a1197ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e0650 0x55a1197ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 -- 172.21.15.70:0/2800809326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 msgr2=0x55a1197e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 -- 172.21.15.70:0/2800809326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a5b30 con 0x55a1197e6580 2023-10-18T12:36:14.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a77fff700 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 0x55a1197e9f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0a70018a20 tx=0x7f0a7000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6624868237651898 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:14.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.919+0000 7f0a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:14.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a56ffd700 1 -- 172.21.15.70:0/2800809326 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000e040 con 0x55a1197e6580 2023-10-18T12:36:14.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a56ffd700 1 -- 172.21.15.70:0/2800809326 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0a70004db0 con 0x55a1197e6580 2023-10-18T12:36:14.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a119737f70 con 0x55a1197e6580 2023-10-18T12:36:14.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a56ffd700 1 -- 172.21.15.70:0/2800809326 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a700087e0 con 0x55a1197e6580 2023-10-18T12:36:14.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a917c5b80 10 client.27017 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:14.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a11955de60 con 0x55a1197e6580 2023-10-18T12:36:14.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a56ffd700 1 -- 172.21.15.70:0/2800809326 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a119737f70 con 0x55a1197e6580 2023-10-18T12:36:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.920+0000 7f0a56ffd700 1 -- 172.21.15.70:0/2800809326 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a7001a950 con 0x55a1197e6580 2023-10-18T12:36:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a917c5b80 10 client.27017 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a917c5b80 10 client.27017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a917c5b80 20 client.27017 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a1197f9fc0 0x55a1197fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a917c5b80 4 client.27017 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1197af930 con 0x55a1197f9fc0 2023-10-18T12:36:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a777fe700 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a1197f9fc0 0x55a1197fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a777fe700 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a1197f9fc0 0x55a1197fc3e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c75346bd3b488694 server_cookie=87595dfc5aef7538 in_seq=0 out_seq=0 2023-10-18T12:36:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.921+0000 7f0a56ffd700 10 client.27017 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a56ffd700 1 -- 172.21.15.70:0/2800809326 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1197af930 con 0x55a1197f9fc0 2023-10-18T12:36:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a56ffd700 10 client.27017 handle_command_reply: tid=0 2023-10-18T12:36:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 client.27017 shutdown 2023-10-18T12:36:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 msgr2=0x55a1197e9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 0x55a1197e9f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0a70018a20 tx=0x7f0a7000b040 comp rx=0 tx=0).stop 2023-10-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 shutdown_connections 2023-10-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a1197f9fc0 0x55a1197fc3e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e6580 0x55a1197e9f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e0650 0x55a1197ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/2800809326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a56ffd700 0 client.27017 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.922+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 >> 172.21.15.70:0/2800809326 conn(0x55a1197547a0 msgr2=0x55a1197df100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.923+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 shutdown_connections 2023-10-18T12:36:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.923+0000 7f0a917c5b80 1 -- 172.21.15.70:0/2800809326 wait complete. 2023-10-18T12:36:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.923+0000 7f0a917c5b80 20 client.27017 trim_cache size 0 max 16384 2023-10-18T12:36:14.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 Processor -- start 2023-10-18T12:36:14.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 -- start start 2023-10-18T12:36:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1195b39a0 con 0x55a1197e5c60 2023-10-18T12:36:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1197dbad0 con 0x55a1197e0650 2023-10-18T12:36:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1197f7620 con 0x55a1197e6580 2023-10-18T12:36:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45286/0 (socket says 172.21.15.70:45286) 2023-10-18T12:36:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.940+0000 7f0a777fe700 1 -- 172.21.15.70:0/4058414680 learned_addr learned my addr 172.21.15.70:0/4058414680 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a77fff700 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a76ffd700 1 -- 172.21.15.70:0/4058414680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 msgr2=0x55a1197f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a76ffd700 1 -- 172.21.15.70:0/4058414680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 msgr2=0x55a1197f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a76ffd700 1 -- 172.21.15.70:0/4058414680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a119546ce0 con 0x55a1197e5c60 2023-10-18T12:36:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a77fff700 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e3dd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c006010 tx=0x7f0a6c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4987a302a9a0b83e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a5e7fc700 1 -- 172.21.15.70:0/4058414680 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c01d030 con 0x55a1197e5c60 2023-10-18T12:36:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a5e7fc700 1 -- 172.21.15.70:0/4058414680 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c01e040 con 0x55a1197e5c60 2023-10-18T12:36:14.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.941+0000 7f0a5e7fc700 1 -- 172.21.15.70:0/4058414680 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c004500 con 0x55a1197e5c60 2023-10-18T12:36:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 -- 172.21.15.70:0/4058414680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 msgr2=0x55a1197e3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e3dd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c006010 tx=0x7f0a6c00b260 comp rx=0 tx=0).stop 2023-10-18T12:36:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 -- 172.21.15.70:0/4058414680 shutdown_connections 2023-10-18T12:36:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a1197f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/4058414680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197e3dd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 -- 172.21.15.70:0/4058414680 >> 172.21.15.70:0/4058414680 conn(0x55a1197547a0 msgr2=0x55a1197e36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 -- 172.21.15.70:0/4058414680 shutdown_connections 2023-10-18T12:36:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 -- 172.21.15.70:0/4058414680 wait complete. 2023-10-18T12:36:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.942+0000 7f0a917c5b80 1 -- start start 2023-10-18T12:36:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 Processor -- start 2023-10-18T12:36:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a119847dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119848310 con 0x55a1197e5c60 2023-10-18T12:36:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119848480 con 0x55a1197e0650 2023-10-18T12:36:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a119848640 con 0x55a1197e6580 2023-10-18T12:36:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a917c5b80 10 client.0 _ll_register_callbacks cb 0x55a119619b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45296/0 (socket says 172.21.15.70:45296) 2023-10-18T12:36:14.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a77fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a119847dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.943+0000 7f0a777fe700 1 -- 172.21.15.70:0/1177621719 learned_addr learned my addr 172.21.15.70:0/1177621719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:14.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a777fe700 1 -- 172.21.15.70:0/1177621719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 msgr2=0x55a119847dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a777fe700 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a119847dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a777fe700 1 -- 172.21.15.70:0/1177621719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 msgr2=0x55a1197f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:14.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a777fe700 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a777fe700 1 -- 172.21.15.70:0/1177621719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a78110100 con 0x55a1197e0650 2023-10-18T12:36:14.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a77fff700 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a119847dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:14.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a777fe700 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197efec0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0a680bfb80 tx=0x7f0a6809aab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa03c1091102802 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:14.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.944+0000 7f0a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:14.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a56ffd700 1 -- 172.21.15.70:0/1177621719 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6809e410 con 0x55a1197e0650 2023-10-18T12:36:14.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a56ffd700 1 -- 172.21.15.70:0/1177621719 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0a68010880 con 0x55a1197e0650 2023-10-18T12:36:14.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a6809a940 con 0x55a1197e0650 2023-10-18T12:36:14.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a56ffd700 1 -- 172.21.15.70:0/1177621719 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a680cd200 con 0x55a1197e0650 2023-10-18T12:36:14.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 10 client.27023 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:14.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a1195842d0 con 0x55a1197e0650 2023-10-18T12:36:14.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a56ffd700 1 -- 172.21.15.70:0/1177621719 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6809a940 con 0x55a1197e0650 2023-10-18T12:36:14.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a56ffd700 1 -- 172.21.15.70:0/1177621719 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a68099730 con 0x55a1197e0650 2023-10-18T12:36:14.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 10 client.27023 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:14.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 10 client.27023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:14.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 20 client.27023 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:14.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a11984fbb0 0x55a119851fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:14.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 4 client.27023 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:36:14.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.945+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a11972bed0 con 0x55a11984fbb0 2023-10-18T12:36:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.946+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a11984fbb0 0x55a119851fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.946+0000 7f0a56ffd700 10 client.27023 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.946+0000 7f0a76ffd700 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a11984fbb0 0x55a119851fd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52fdcc33dcdcfec2 server_cookie=2dd2169cc04d5109 in_seq=0 out_seq=0 2023-10-18T12:36:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a56ffd700 1 -- 172.21.15.70:0/1177621719 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a11972bed0 con 0x55a11984fbb0 2023-10-18T12:36:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a56ffd700 10 client.27023 handle_command_reply: tid=0 2023-10-18T12:36:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 client.27023 shutdown 2023-10-18T12:36:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 msgr2=0x55a1197efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197efec0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0a680bfb80 tx=0x7f0a6809aab0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 shutdown_connections 2023-10-18T12:36:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a11984fbb0 0x55a119851fd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a1197e6580 0x55a119847dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1197e0650 0x55a1197efec0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a56ffd700 0 client.27023 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 --2- 172.21.15.70:0/1177621719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a1197e5c60 0x55a1197f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 >> 172.21.15.70:0/1177621719 conn(0x55a1197547a0 msgr2=0x55a1197d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 shutdown_connections 2023-10-18T12:36:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 1 -- 172.21.15.70:0/1177621719 wait complete. 2023-10-18T12:36:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.948+0000 7f0a917c5b80 20 client.27023 trim_cache size 0 max 16384 2023-10-18T12:36:14.964 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:36:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:36:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "9fb1aab7-f91f-4fc0-baba-759bc64383e3", 2023-10-18T12:36:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:36:14.965 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:36:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a600a15f0 msgr2=0x7f0a600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a600a15f0 0x7f0a600a3ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009ec0 tx=0x7f0a70009920 comp rx=0 tx=0).stop 2023-10-18T12:36:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 msgr2=0x7f0a78198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78198e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e9a0 tx=0x7f0a6800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 shutdown_connections 2023-10-18T12:36:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a78108940 0x7f0a7819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f0a600a15f0 0x7f0a600a3ab0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a78102930 0x7f0a78198e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 --2- 172.21.15.70:0/3476444883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a781032f0 0x7f0a78199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:14.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 >> 172.21.15.70:0/3476444883 conn(0x7f0a780fe3e0 msgr2=0x7f0a780fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:14.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 shutdown_connections 2023-10-18T12:36:14.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:14.949+0000 7f0a7ddff700 1 -- 172.21.15.70:0/3476444883 wait complete. 2023-10-18T12:36:14.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:36:15.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:14 smithi070 ceph-mon[98639]: pgmap v628: 129 pgs: 129 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 218 op/s 2023-10-18T12:36:15.057 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:14 smithi070 ceph-mon[98639]: osdmap e101: 12 total, 12 up, 12 in 2023-10-18T12:36:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:14 smithi152 ceph-mon[134767]: pgmap v628: 129 pgs: 129 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 218 op/s 2023-10-18T12:36:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:14 smithi152 ceph-mon[134767]: osdmap e101: 12 total, 12 up, 12 in 2023-10-18T12:36:15.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:14 smithi186 ceph-mon[135253]: pgmap v628: 129 pgs: 129 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 218 op/s 2023-10-18T12:36:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:14 smithi186 ceph-mon[135253]: osdmap e101: 12 total, 12 up, 12 in 2023-10-18T12:36:15.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 -- 172.21.15.70:0/3727176814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290106720 msgr2=0x7f6290106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 --2- 172.21.15.70:0/3727176814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290106720 0x7f6290106b00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f62800098f0 tx=0x7f6280009c00 comp rx=0 tx=0).stop 2023-10-18T12:36:15.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 -- 172.21.15.70:0/3727176814 shutdown_connections 2023-10-18T12:36:15.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 --2- 172.21.15.70:0/3727176814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901010d0 0x7f629010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 --2- 172.21.15.70:0/3727176814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290106720 0x7f6290106b00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 --2- 172.21.15.70:0/3727176814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6290100710 0x7f6290100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 -- 172.21.15.70:0/3727176814 >> 172.21.15.70:0/3727176814 conn(0x7f62900fc200 msgr2=0x7f62900fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:15.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 -- 172.21.15.70:0/3727176814 shutdown_connections 2023-10-18T12:36:15.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.247+0000 7f62951ff700 1 -- 172.21.15.70:0/3727176814 wait complete. 2023-10-18T12:36:15.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 Processor -- start 2023-10-18T12:36:15.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 -- start start 2023-10-18T12:36:15.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 0x7f6290198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901010d0 0x7f6290199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6290106720 0x7f629019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019dca0 con 0x7f62901010d0 2023-10-18T12:36:15.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019de10 con 0x7f6290100710 2023-10-18T12:36:15.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.248+0000 7f62951ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019dfd0 con 0x7f6290106720 2023-10-18T12:36:15.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 0x7f6290198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 0x7f6290198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45306/0 (socket says 172.21.15.70:45306) 2023-10-18T12:36:15.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901010d0 0x7f6290199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 -- 172.21.15.70:0/710048885 learned_addr learned my addr 172.21.15.70:0/710048885 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:15.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628f59e700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6290106720 0x7f629019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 -- 172.21.15.70:0/710048885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6290106720 msgr2=0x7f629019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6290106720 0x7f629019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 -- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901010d0 msgr2=0x7f6290199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901010d0 0x7f6290199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 -- 172.21.15.70:0/710048885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009d00 con 0x7f6290100710 2023-10-18T12:36:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f628ed9d700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 0x7f6290198df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6280005950 tx=0x7f628000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f6277fff700 1 -- 172.21.15.70:0/710048885 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628001c070 con 0x7f6290100710 2023-10-18T12:36:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f6277fff700 1 -- 172.21.15.70:0/710048885 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f628000bb20 con 0x7f6290100710 2023-10-18T12:36:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6280009520 con 0x7f6290100710 2023-10-18T12:36:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.249+0000 7f6277fff700 1 -- 172.21.15.70:0/710048885 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800038e0 con 0x7f6290100710 2023-10-18T12:36:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.250+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901a5910 con 0x7f6290100710 2023-10-18T12:36:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.252+0000 7f6277fff700 1 -- 172.21.15.70:0/710048885 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6280020030 con 0x7f6290100710 2023-10-18T12:36:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f6277fff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f62780a1670 0x7f62780a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 Processor -- start 2023-10-18T12:36:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f6277fff700 1 -- 172.21.15.70:0/710048885 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f62800ce290 con 0x7f6290100710 2023-10-18T12:36:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 -- start start 2023-10-18T12:36:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f628e59c700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f62780a1670 0x7f62780a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a82a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e59f56e0 con 0x5559e5a87be0 2023-10-18T12:36:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e59dbc60 con 0x5559e5a825d0 2023-10-18T12:36:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.254+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e59d9580 con 0x5559e5a88500 2023-10-18T12:36:15.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a87fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33112/0 (socket says 172.21.15.70:33112) 2023-10-18T12:36:15.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 -- 172.21.15.70:0/468693735 learned_addr learned my addr 172.21.15.70:0/468693735 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:15.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628f59e700 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a82a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628e59c700 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 -- 172.21.15.70:0/468693735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 msgr2=0x5559e5a82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 -- 172.21.15.70:0/468693735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 msgr2=0x5559e5a82a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a82a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 -- 172.21.15.70:0/468693735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559e59923f0 con 0x5559e5a87be0 2023-10-18T12:36:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628e59c700 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628e59c700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f62780a1670 0x7f62780a3b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6290106070 tx=0x7f628400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f628ed9d700 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a87fc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f62800979b0 tx=0x7f6280005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fa20e1d924934e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f62757fa700 1 -- 172.21.15.70:0/468693735 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628009af40 con 0x5559e5a87be0 2023-10-18T12:36:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f62757fa700 1 -- 172.21.15.70:0/468693735 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f628009f090 con 0x5559e5a87be0 2023-10-18T12:36:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.255+0000 7f62757fa700 1 -- 172.21.15.70:0/468693735 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800977a0 con 0x5559e5a87be0 2023-10-18T12:36:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/468693735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 msgr2=0x5559e5a87fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a87fc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f62800979b0 tx=0x7f6280005570 comp rx=0 tx=0).stop 2023-10-18T12:36:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/468693735 shutdown_connections 2023-10-18T12:36:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a82a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a87fc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/468693735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/468693735 >> 172.21.15.70:0/468693735 conn(0x5559e59f6720 msgr2=0x5559e5a7cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/468693735 shutdown_connections 2023-10-18T12:36:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.256+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/468693735 wait complete. 2023-10-18T12:36:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.257+0000 7f62a8a3ab80 1 -- start start 2023-10-18T12:36:15.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.257+0000 7f62a8a3ab80 1 Processor -- start 2023-10-18T12:36:15.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a8bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e59d4e70 con 0x5559e5a87be0 2023-10-18T12:36:15.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e5971710 con 0x5559e5a825d0 2023-10-18T12:36:15.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e59713b0 con 0x5559e5a88500 2023-10-18T12:36:15.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f62a8a3ab80 10 client.0 _ll_register_callbacks cb 0x5559e58bba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:15.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45318/0 (socket says 172.21.15.70:45318) 2023-10-18T12:36:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628ed9d700 1 -- 172.21.15.70:0/288316602 learned_addr learned my addr 172.21.15.70:0/288316602 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628f59e700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a8bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628e59c700 1 -- 172.21.15.70:0/288316602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 msgr2=0x5559e5a8bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628e59c700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a8bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628e59c700 1 -- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 msgr2=0x5559e5a8e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628e59c700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628e59c700 1 -- 172.21.15.70:0/288316602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901a5910 con 0x5559e5a87be0 2023-10-18T12:36:15.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.258+0000 7f628ed9d700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:15.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f628e59c700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a8b740 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x5559e5a7e7d0 tx=0x7f62840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39657dfa06ce1670 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:15.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f6272ff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:36:15.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f6272ff5700 1 -- 172.21.15.70:0/288316602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284011640 con 0x5559e5a87be0 2023-10-18T12:36:15.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559e59d9ef0 con 0x5559e5a87be0 2023-10-18T12:36:15.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f6272ff5700 1 -- 172.21.15.70:0/288316602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f6284017070 con 0x5559e5a87be0 2023-10-18T12:36:15.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f6272ff5700 1 -- 172.21.15.70:0/288316602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f628401ee50 con 0x5559e5a87be0 2023-10-18T12:36:15.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f62a8a3ab80 10 client.16989 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:15.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5559e57ffde0 con 0x5559e5a87be0 2023-10-18T12:36:15.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f6272ff5700 1 -- 172.21.15.70:0/288316602 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559e59d9ef0 con 0x5559e5a87be0 2023-10-18T12:36:15.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.259+0000 7f6272ff5700 1 -- 172.21.15.70:0/288316602 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f62840267b0 con 0x5559e5a87be0 2023-10-18T12:36:15.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f62a8a3ab80 10 client.16989 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:15.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f62a8a3ab80 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:15.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f62a8a3ab80 20 client.16989 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:15.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5a9bb60 0x5559e5a9df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f62a8a3ab80 4 client.16989 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:15.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559e5a518b0 con 0x5559e5a9bb60 2023-10-18T12:36:15.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f628ed9d700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5a9bb60 0x5559e5a9df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f6272ff5700 10 client.16989 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:15.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.260+0000 7f628ed9d700 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5a9bb60 0x5559e5a9df80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b2971db4f57589 server_cookie=1b7cceb34fd9f83c in_seq=0 out_seq=0 2023-10-18T12:36:15.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f6272ff5700 1 -- 172.21.15.70:0/288316602 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559e5a518b0 con 0x5559e5a9bb60 2023-10-18T12:36:15.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f6272ff5700 10 client.16989 handle_command_reply: tid=0 2023-10-18T12:36:15.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 client.16989 shutdown 2023-10-18T12:36:15.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 msgr2=0x5559e5a8b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a8b740 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x5559e5a7e7d0 tx=0x7f62840047c0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 shutdown_connections 2023-10-18T12:36:15.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5a9bb60 0x5559e5a9df80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a8bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f6272ff5700 0 client.16989 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:15.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.261+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/288316602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a8b740 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.262+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 >> 172.21.15.70:0/288316602 conn(0x5559e59f6720 msgr2=0x5559e5a81080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:15.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.262+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 shutdown_connections 2023-10-18T12:36:15.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.262+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/288316602 wait complete. 2023-10-18T12:36:15.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.262+0000 7f62a8a3ab80 20 client.16989 trim_cache size 0 max 16384 2023-10-18T12:36:15.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.279+0000 7f62a8a3ab80 1 Processor -- start 2023-10-18T12:36:15.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.279+0000 7f62a8a3ab80 1 -- start start 2023-10-18T12:36:15.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a825d0 0x5559e5a92c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 0x5559e5a8a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a88500 0x5559e5a8a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e5855920 con 0x5559e5a88500 2023-10-18T12:36:15.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e59d9f20 con 0x5559e5a87be0 2023-10-18T12:36:15.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e5a83030 con 0x5559e5a825d0 2023-10-18T12:36:15.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 0x5559e5a8a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628f59e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a88500 0x5559e5a8a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 0x5559e5a8a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:45322/0 (socket says 172.21.15.70:45322) 2023-10-18T12:36:15.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 -- 172.21.15.70:0/3038163724 learned_addr learned my addr 172.21.15.70:0/3038163724 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:15.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628ed9d700 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a825d0 0x5559e5a92c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 -- 172.21.15.70:0/3038163724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a825d0 msgr2=0x5559e5a92c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a825d0 0x5559e5a92c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 -- 172.21.15.70:0/3038163724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a88500 msgr2=0x5559e5a8a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a88500 0x5559e5a8a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 -- 172.21.15.70:0/3038163724 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559e57e8c60 con 0x5559e5a87be0 2023-10-18T12:36:15.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628ed9d700 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a825d0 0x5559e5a92c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:15.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628f59e700 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a88500 0x5559e5a8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:15.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.280+0000 7f628e59c700 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 0x5559e5a8a010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6284026bc0 tx=0x7f6284005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b905707dc425c966 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:15.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f6275ffb700 1 -- 172.21.15.70:0/3038163724 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840265f0 con 0x5559e5a87be0 2023-10-18T12:36:15.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f6275ffb700 1 -- 172.21.15.70:0/3038163724 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f628401ea00 con 0x5559e5a87be0 2023-10-18T12:36:15.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f6275ffb700 1 -- 172.21.15.70:0/3038163724 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628401cea0 con 0x5559e5a87be0 2023-10-18T12:36:15.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/3038163724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 msgr2=0x5559e5a8a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 0x5559e5a8a010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6284026bc0 tx=0x7f6284005ee0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/3038163724 shutdown_connections 2023-10-18T12:36:15.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a88500 0x5559e5a8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a825d0 0x5559e5a92c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/3038163724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a87be0 0x5559e5a8a010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/3038163724 >> 172.21.15.70:0/3038163724 conn(0x5559e59f6720 msgr2=0x5559e5a81890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:15.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/3038163724 shutdown_connections 2023-10-18T12:36:15.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.281+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/3038163724 wait complete. 2023-10-18T12:36:15.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 -- start start 2023-10-18T12:36:15.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 Processor -- start 2023-10-18T12:36:15.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a91ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a92510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e5a92bb0 con 0x5559e5a87be0 2023-10-18T12:36:15.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e5a92d20 con 0x5559e5a825d0 2023-10-18T12:36:15.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.282+0000 7f62a8a3ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e5aef9c0 con 0x5559e5a88500 2023-10-18T12:36:15.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f62a8a3ab80 10 client.0 _ll_register_callbacks cb 0x5559e58bba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:15.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a92510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a91ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a92510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:46368/0 (socket says 172.21.15.70:46368) 2023-10-18T12:36:15.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 -- 172.21.15.70:0/2716662921 learned_addr learned my addr 172.21.15.70:0/2716662921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:15.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a91ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33134/0 (socket says 172.21.15.70:33134) 2023-10-18T12:36:15.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 -- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 msgr2=0x5559e5a8b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 -- 172.21.15.70:0/2716662921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 msgr2=0x5559e5a91ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:15.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a91ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 -- 172.21.15.70:0/2716662921 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6288003a00 con 0x5559e5a88500 2023-10-18T12:36:15.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628ed9d700 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:15.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f628f59e700 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a92510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f62880176c0 tx=0x7f62880179d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e354442fd546501 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:15.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.283+0000 7f6272ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:36:15.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f6272ff5700 1 -- 172.21.15.70:0/2716662921 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628801c070 con 0x5559e5a88500 2023-10-18T12:36:15.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559e59cde50 con 0x5559e5a88500 2023-10-18T12:36:15.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f6272ff5700 1 -- 172.21.15.70:0/2716662921 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f6288004bb0 con 0x5559e5a88500 2023-10-18T12:36:15.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f6272ff5700 1 -- 172.21.15.70:0/2716662921 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f628800e040 con 0x5559e5a88500 2023-10-18T12:36:15.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 10 client.27025 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:15.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5559e57ffde0 con 0x5559e5a88500 2023-10-18T12:36:15.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f6272ff5700 1 -- 172.21.15.70:0/2716662921 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559e59cde50 con 0x5559e5a88500 2023-10-18T12:36:15.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f6272ff5700 1 -- 172.21.15.70:0/2716662921 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f62880207e0 con 0x5559e5a88500 2023-10-18T12:36:15.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 10 client.27025 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:15.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 10 client.27025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:15.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 20 client.27025 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:15.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5af1270 0x5559e5af3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:15.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 4 client.27025 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:36:15.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.284+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5559e59cdb70 con 0x5559e5af1270 2023-10-18T12:36:15.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.285+0000 7f628ed9d700 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5af1270 0x5559e5af3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:15.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.285+0000 7f628ed9d700 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5af1270 0x5559e5af3690 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d0748d99e9d745 server_cookie=5c3c9e642e949bc3 in_seq=0 out_seq=0 2023-10-18T12:36:15.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.285+0000 7f6272ff5700 10 client.27025 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:15.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f6272ff5700 1 -- 172.21.15.70:0/2716662921 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5559e59cdb70 con 0x5559e5af1270 2023-10-18T12:36:15.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f6272ff5700 10 client.27025 handle_command_reply: tid=0 2023-10-18T12:36:15.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 client.27025 shutdown 2023-10-18T12:36:15.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 msgr2=0x5559e5a92510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a92510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f62880176c0 tx=0x7f62880179d0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 shutdown_connections 2023-10-18T12:36:15.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5559e5af1270 0x5559e5af3690 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5559e5a88500 0x5559e5a92510 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f6272ff5700 0 client.27025 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5559e5a825d0 0x5559e5a8b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 --2- 172.21.15.70:0/2716662921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559e5a87be0 0x5559e5a91ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 >> 172.21.15.70:0/2716662921 conn(0x5559e59f6720 msgr2=0x5559e5a89f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 shutdown_connections 2023-10-18T12:36:15.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 1 -- 172.21.15.70:0/2716662921 wait complete. 2023-10-18T12:36:15.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.286+0000 7f62a8a3ab80 20 client.27025 trim_cache size 0 max 16384 2023-10-18T12:36:15.304 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:36:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:36:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:36:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "9fb1aab7-f91f-4fc0-baba-759bc64383e3": { 2023-10-18T12:36:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:36:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "9fb1aab7-f91f-4fc0-baba-759bc64383e3", 2023-10-18T12:36:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:36:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:36:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:36:15.306 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:36:15.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f62780a1670 msgr2=0x7f62780a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f62780a1670 0x7f62780a3b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6290106070 tx=0x7f628400b410 comp rx=0 tx=0).stop 2023-10-18T12:36:15.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 msgr2=0x7f6290198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:15.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 0x7f6290198df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6280005950 tx=0x7f628000fa00 comp rx=0 tx=0).stop 2023-10-18T12:36:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 shutdown_connections 2023-10-18T12:36:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6290106720 0x7f629019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f62780a1670 0x7f62780a3b30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6290100710 0x7f6290198df0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 --2- 172.21.15.70:0/710048885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901010d0 0x7f6290199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:15.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 >> 172.21.15.70:0/710048885 conn(0x7f62900fc200 msgr2=0x7f629010bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:15.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 shutdown_connections 2023-10-18T12:36:15.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:15.287+0000 7f62951ff700 1 -- 172.21.15.70:0/710048885 wait complete. 2023-10-18T12:36:15.310 INFO:tasks.cephfs.filesystem:scrub status for tag:9fb1aab7-f91f-4fc0-baba-759bc64383e3 - {'path': '/', 'tag': '9fb1aab7-f91f-4fc0-baba-759bc64383e3', 'options': 'recursive,force'} 2023-10-18T12:36:15.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1164455 11.49 MB/sec execute 404 sec latency 39.829 ms 2023-10-18T12:36:16.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:15 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:36:16.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:15 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:16.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:15 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:36:16.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:15 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:16.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:15 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:36:16.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:15 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:16.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:15 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:36:16.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:15 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:15 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:36:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:15 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:15 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:36:16.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:15 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:16.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1166442 11.49 MB/sec execute 405 sec latency 12.273 ms 2023-10-18T12:36:17.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:16 smithi152 ceph-mon[134767]: pgmap v630: 129 pgs: 129 active+clean; 585 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2023-10-18T12:36:17.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:16 smithi186 ceph-mon[135253]: pgmap v630: 129 pgs: 129 active+clean; 585 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2023-10-18T12:36:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:16 smithi070 ceph-mon[98639]: pgmap v630: 129 pgs: 129 active+clean; 585 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2023-10-18T12:36:17.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1168818 11.50 MB/sec execute 406 sec latency 52.095 ms 2023-10-18T12:36:18.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1170462 11.49 MB/sec execute 407 sec latency 19.094 ms 2023-10-18T12:36:19.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:18 smithi152 ceph-mon[134767]: pgmap v631: 129 pgs: 129 active+clean; 593 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 256 op/s 2023-10-18T12:36:19.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:18 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:36:19.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:18 smithi186 ceph-mon[135253]: pgmap v631: 129 pgs: 129 active+clean; 593 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 256 op/s 2023-10-18T12:36:19.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:18 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:36:19.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:18 smithi070 ceph-mon[98639]: pgmap v631: 129 pgs: 129 active+clean; 593 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 256 op/s 2023-10-18T12:36:19.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:18 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:36:19.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1172583 11.49 MB/sec execute 408 sec latency 54.016 ms 2023-10-18T12:36:20.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:20 smithi070 ceph-mon[98639]: pgmap v632: 129 pgs: 129 active+clean; 596 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 234 op/s 2023-10-18T12:36:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:20 smithi152 ceph-mon[134767]: pgmap v632: 129 pgs: 129 active+clean; 596 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 234 op/s 2023-10-18T12:36:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:20 smithi186 ceph-mon[135253]: pgmap v632: 129 pgs: 129 active+clean; 596 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 234 op/s 2023-10-18T12:36:20.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1174445 11.48 MB/sec execute 409 sec latency 32.056 ms 2023-10-18T12:36:21.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1176470 11.48 MB/sec execute 410 sec latency 12.985 ms 2023-10-18T12:36:22.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1179005 11.49 MB/sec execute 411 sec latency 22.567 ms 2023-10-18T12:36:23.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:22 smithi070 ceph-mon[98639]: pgmap v633: 129 pgs: 129 active+clean; 601 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-18T12:36:23.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:22 smithi152 ceph-mon[134767]: pgmap v633: 129 pgs: 129 active+clean; 601 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-18T12:36:23.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:22 smithi186 ceph-mon[135253]: pgmap v633: 129 pgs: 129 active+clean; 601 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2023-10-18T12:36:23.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1180719 11.49 MB/sec execute 412 sec latency 19.803 ms 2023-10-18T12:36:24.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1182964 11.49 MB/sec execute 413 sec latency 114.874 ms 2023-10-18T12:36:25.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:24 smithi152 ceph-mon[134767]: pgmap v634: 129 pgs: 129 active+clean; 598 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 207 op/s 2023-10-18T12:36:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:24 smithi186 ceph-mon[135253]: pgmap v634: 129 pgs: 129 active+clean; 598 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 207 op/s 2023-10-18T12:36:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:24 smithi070 ceph-mon[98639]: pgmap v634: 129 pgs: 129 active+clean; 598 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 207 op/s 2023-10-18T12:36:25.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1184278 11.48 MB/sec execute 414 sec latency 73.374 ms 2023-10-18T12:36:26.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1186725 11.48 MB/sec execute 415 sec latency 37.363 ms 2023-10-18T12:36:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:26 smithi152 ceph-mon[134767]: pgmap v635: 129 pgs: 129 active+clean; 586 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 225 op/s 2023-10-18T12:36:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:26 smithi186 ceph-mon[135253]: pgmap v635: 129 pgs: 129 active+clean; 586 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 225 op/s 2023-10-18T12:36:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:26 smithi070 ceph-mon[98639]: pgmap v635: 129 pgs: 129 active+clean; 586 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 225 op/s 2023-10-18T12:36:27.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1189076 11.48 MB/sec execute 416 sec latency 12.499 ms 2023-10-18T12:36:28.875 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1191090 11.48 MB/sec execute 417 sec latency 11.375 ms 2023-10-18T12:36:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:28 smithi152 ceph-mon[134767]: pgmap v636: 129 pgs: 129 active+clean; 570 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2023-10-18T12:36:29.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:28 smithi186 ceph-mon[135253]: pgmap v636: 129 pgs: 129 active+clean; 570 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2023-10-18T12:36:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:28 smithi070 ceph-mon[98639]: pgmap v636: 129 pgs: 129 active+clean; 570 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2023-10-18T12:36:29.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1193195 11.48 MB/sec execute 418 sec latency 65.035 ms 2023-10-18T12:36:30.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:30 smithi070 ceph-mon[98639]: pgmap v637: 129 pgs: 129 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 216 op/s 2023-10-18T12:36:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:30 smithi152 ceph-mon[134767]: pgmap v637: 129 pgs: 129 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 216 op/s 2023-10-18T12:36:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:30 smithi186 ceph-mon[135253]: pgmap v637: 129 pgs: 129 active+clean; 567 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 216 op/s 2023-10-18T12:36:30.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1194766 11.48 MB/sec execute 419 sec latency 32.464 ms 2023-10-18T12:36:31.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1197214 11.48 MB/sec execute 420 sec latency 15.287 ms 2023-10-18T12:36:32.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1199021 11.47 MB/sec execute 421 sec latency 16.634 ms 2023-10-18T12:36:33.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:32 smithi152 ceph-mon[134767]: pgmap v638: 129 pgs: 129 active+clean; 568 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 201 op/s 2023-10-18T12:36:33.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:32 smithi186 ceph-mon[135253]: pgmap v638: 129 pgs: 129 active+clean; 568 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 201 op/s 2023-10-18T12:36:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:32 smithi070 ceph-mon[98639]: pgmap v638: 129 pgs: 129 active+clean; 568 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 201 op/s 2023-10-18T12:36:33.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1201151 11.47 MB/sec execute 422 sec latency 18.299 ms 2023-10-18T12:36:34.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1202922 11.46 MB/sec execute 423 sec latency 97.354 ms 2023-10-18T12:36:35.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:34 smithi152 ceph-mon[134767]: pgmap v639: 129 pgs: 129 active+clean; 576 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2023-10-18T12:36:35.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:34 smithi186 ceph-mon[135253]: pgmap v639: 129 pgs: 129 active+clean; 576 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2023-10-18T12:36:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:34 smithi070 ceph-mon[98639]: pgmap v639: 129 pgs: 129 active+clean; 576 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2023-10-18T12:36:35.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1204775 11.46 MB/sec execute 424 sec latency 23.464 ms 2023-10-18T12:36:36.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1207188 11.47 MB/sec execute 425 sec latency 39.568 ms 2023-10-18T12:36:37.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:36 smithi152 ceph-mon[134767]: pgmap v640: 129 pgs: 129 active+clean; 577 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 209 op/s 2023-10-18T12:36:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:36 smithi186 ceph-mon[135253]: pgmap v640: 129 pgs: 129 active+clean; 577 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 209 op/s 2023-10-18T12:36:37.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:36 smithi070 ceph-mon[98639]: pgmap v640: 129 pgs: 129 active+clean; 577 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 209 op/s 2023-10-18T12:36:37.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1209082 11.47 MB/sec execute 426 sec latency 14.554 ms 2023-10-18T12:36:38.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1211525 11.47 MB/sec execute 427 sec latency 16.793 ms 2023-10-18T12:36:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:38 smithi152 ceph-mon[134767]: pgmap v641: 129 pgs: 129 active+clean; 586 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2023-10-18T12:36:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:38 smithi186 ceph-mon[135253]: pgmap v641: 129 pgs: 129 active+clean; 586 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2023-10-18T12:36:39.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:38 smithi070 ceph-mon[98639]: pgmap v641: 129 pgs: 129 active+clean; 586 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2023-10-18T12:36:39.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1213467 11.46 MB/sec execute 428 sec latency 87.992 ms 2023-10-18T12:36:40.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:40 smithi070 ceph-mon[98639]: pgmap v642: 129 pgs: 129 active+clean; 590 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-18T12:36:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:40 smithi152 ceph-mon[134767]: pgmap v642: 129 pgs: 129 active+clean; 590 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-18T12:36:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:40 smithi186 ceph-mon[135253]: pgmap v642: 129 pgs: 129 active+clean; 590 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2023-10-18T12:36:40.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1215194 11.46 MB/sec execute 429 sec latency 41.722 ms 2023-10-18T12:36:41.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1217253 11.46 MB/sec execute 430 sec latency 16.295 ms 2023-10-18T12:36:41.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:41 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:36:42.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:41 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:36:42.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:41 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:36:42.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1219113 11.46 MB/sec execute 431 sec latency 12.693 ms 2023-10-18T12:36:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:42 smithi186 ceph-mon[135253]: pgmap v643: 129 pgs: 129 active+clean; 594 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 205 op/s 2023-10-18T12:36:43.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:42 smithi070 ceph-mon[98639]: pgmap v643: 129 pgs: 129 active+clean; 594 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 205 op/s 2023-10-18T12:36:43.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:42 smithi152 ceph-mon[134767]: pgmap v643: 129 pgs: 129 active+clean; 594 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 205 op/s 2023-10-18T12:36:43.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1221200 11.46 MB/sec execute 432 sec latency 54.465 ms 2023-10-18T12:36:44.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1222508 11.45 MB/sec execute 433 sec latency 105.607 ms 2023-10-18T12:36:45.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:44 smithi152 ceph-mon[134767]: pgmap v644: 129 pgs: 129 active+clean; 602 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 242 op/s 2023-10-18T12:36:45.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:45.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:44 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:45.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:44 smithi186 ceph-mon[135253]: pgmap v644: 129 pgs: 129 active+clean; 602 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 242 op/s 2023-10-18T12:36:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:45.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:44 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:45.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:36:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:44 smithi070 ceph-mon[98639]: pgmap v644: 129 pgs: 129 active+clean; 602 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 242 op/s 2023-10-18T12:36:45.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:45.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:44 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:45.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 -- 172.21.15.70:0/2710370991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 msgr2=0x7fcb2410f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 --2- 172.21.15.70:0/2710370991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 0x7fcb2410f3e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009f90 tx=0x7fcb1c00b330 comp rx=0 tx=0).stop 2023-10-18T12:36:45.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 -- 172.21.15.70:0/2710370991 shutdown_connections 2023-10-18T12:36:45.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 --2- 172.21.15.70:0/2710370991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 0x7fcb2410f3e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 --2- 172.21.15.70:0/2710370991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb24072ba0 0x7fcb24072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 --2- 172.21.15.70:0/2710370991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb240734c0 0x7fcb2410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 -- 172.21.15.70:0/2710370991 >> 172.21.15.70:0/2710370991 conn(0x7fcb240fc200 msgr2=0x7fcb240fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 -- 172.21.15.70:0/2710370991 shutdown_connections 2023-10-18T12:36:45.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.543+0000 7fcb2abff700 1 -- 172.21.15.70:0/2710370991 wait complete. 2023-10-18T12:36:45.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.544+0000 7fcb2abff700 1 Processor -- start 2023-10-18T12:36:45.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.544+0000 7fcb2abff700 1 -- start start 2023-10-18T12:36:45.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2abff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb24072ba0 0x7fcb24198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2abff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 0x7fcb24199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2abff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 0x7fcb2419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2abff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419de00 con 0x7fcb24072ba0 2023-10-18T12:36:45.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2abff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419df70 con 0x7fcb240734c0 2023-10-18T12:36:45.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2abff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419e130 con 0x7fcb2410cff0 2023-10-18T12:36:45.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 0x7fcb24199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb24072ba0 0x7fcb24198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 0x7fcb24199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39926/0 (socket says 172.21.15.70:39926) 2023-10-18T12:36:45.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 -- 172.21.15.70:0/3840138276 learned_addr learned my addr 172.21.15.70:0/3840138276 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:45.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb2919c700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 0x7fcb2419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 -- 172.21.15.70:0/3840138276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 msgr2=0x7fcb2419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 0x7fcb2419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 -- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb24072ba0 msgr2=0x7fcb24198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb24072ba0 0x7fcb24198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.545+0000 7fcb1bfff700 1 -- 172.21.15.70:0/3840138276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200095f0 con 0x7fcb240734c0 2023-10-18T12:36:45.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.546+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 0x7fcb24199420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9a0 tx=0x7fcb1400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.546+0000 7fcb19ffb700 1 -- 172.21.15.70:0/3840138276 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14009790 con 0x7fcb240734c0 2023-10-18T12:36:45.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.546+0000 7fcb19ffb700 1 -- 172.21.15.70:0/3840138276 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb14010430 con 0x7fcb240734c0 2023-10-18T12:36:45.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.546+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c009c40 con 0x7fcb240734c0 2023-10-18T12:36:45.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.546+0000 7fcb19ffb700 1 -- 172.21.15.70:0/3840138276 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018680 con 0x7fcb240734c0 2023-10-18T12:36:45.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.546+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2419e600 con 0x7fcb240734c0 2023-10-18T12:36:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.549+0000 7fcb19ffb700 1 -- 172.21.15.70:0/3840138276 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcb140105a0 con 0x7fcb240734c0 2023-10-18T12:36:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 Processor -- start 2023-10-18T12:36:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 -- start start 2023-10-18T12:36:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585542fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558553d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558553da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555854b06e0 con 0x555585543500 2023-10-18T12:36:45.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555585496c60 con 0x55558553d5d0 2023-10-18T12:36:45.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555585494580 con 0x555585542be0 2023-10-18T12:36:45.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb2919c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585542fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb2919c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585542fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49592/0 (socket says 172.21.15.70:49592) 2023-10-18T12:36:45.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb19ffb700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcb0c0a1670 0x7fcb0c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb2919c700 1 -- 172.21.15.70:0/1422253120 learned_addr learned my addr 172.21.15.70:0/1422253120 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:45.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.551+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558553da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558553d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb19ffb700 1 -- 172.21.15.70:0/3840138276 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fcb14014070 con 0x7fcb240734c0 2023-10-18T12:36:45.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb2919c700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcb0c0a1670 0x7fcb0c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1422253120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 msgr2=0x555585542fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585542fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1422253120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 msgr2=0x55558553d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558553d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1422253120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55558544d3f0 con 0x55558553d5d0 2023-10-18T12:36:45.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb2919c700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585542fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:45.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558553d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:45.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558553da50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcb140ca910 tx=0x7fcb14095ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cbaa6d49153fa2a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.552+0000 7fcb0affd700 1 -- 172.21.15.70:0/1422253120 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1409a8a0 con 0x55558553d5d0 2023-10-18T12:36:45.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb0affd700 1 -- 172.21.15.70:0/1422253120 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb1409a1a0 con 0x55558553d5d0 2023-10-18T12:36:45.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb0affd700 1 -- 172.21.15.70:0/1422253120 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1409a400 con 0x55558553d5d0 2023-10-18T12:36:45.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb2919c700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcb0c0a1670 0x7fcb0c0a3b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0060b0 tx=0x7fcb1c00b330 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1422253120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 msgr2=0x55558553da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558553da50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcb140ca910 tx=0x7fcb14095ee0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1422253120 shutdown_connections 2023-10-18T12:36:45.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558553da50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585542fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1422253120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558553d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1422253120 >> 172.21.15.70:0/1422253120 conn(0x5555854b1720 msgr2=0x555585537b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1422253120 shutdown_connections 2023-10-18T12:36:45.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.553+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1422253120 wait complete. 2023-10-18T12:36:45.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.554+0000 7fcb3e569b80 1 -- start start 2023-10-18T12:36:45.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.554+0000 7fcb3e569b80 1 Processor -- start 2023-10-18T12:36:45.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x555585546830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 0x555585546d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585543500 0x5555855472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558548fe70 con 0x55558553d5d0 2023-10-18T12:36:45.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558542c710 con 0x555585542be0 2023-10-18T12:36:45.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558542c3b0 con 0x555585543500 2023-10-18T12:36:45.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb3e569b80 10 client.0 _ll_register_callbacks cb 0x555585376a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:45.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb2899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x555585546830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 0x555585546d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 0x555585546d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39952/0 (socket says 172.21.15.70:39952) 2023-10-18T12:36:45.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1240609734 learned_addr learned my addr 172.21.15.70:0/1240609734 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:45.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb2919c700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585543500 0x5555855472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1240609734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585543500 msgr2=0x5555855472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585543500 0x5555855472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1240609734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 msgr2=0x555585546830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x555585546830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb1bfff700 1 -- 172.21.15.70:0/1240609734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb20009b80 con 0x555585542be0 2023-10-18T12:36:45.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.555+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x555585546830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:45.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 0x555585546d70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb140caf10 tx=0x7fcb14098750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c234a835154f1604 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb03fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:45.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb03fff700 1 -- 172.21.15.70:0/1240609734 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1409a600 con 0x555585542be0 2023-10-18T12:36:45.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb03fff700 1 -- 172.21.15.70:0/1240609734 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fcb140a20a0 con 0x555585542be0 2023-10-18T12:36:45.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555585494ef0 con 0x555585542be0 2023-10-18T12:36:45.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb03fff700 1 -- 172.21.15.70:0/1240609734 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb140a2260 con 0x555585542be0 2023-10-18T12:36:45.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb3e569b80 10 client.27053 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:45.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fcb2419e600 con 0x555585542be0 2023-10-18T12:36:45.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb03fff700 1 -- 172.21.15.70:0/1240609734 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555585494ef0 con 0x555585542be0 2023-10-18T12:36:45.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.556+0000 7fcb03fff700 1 -- 172.21.15.70:0/1240609734 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcb140cd1d0 con 0x555585542be0 2023-10-18T12:36:45.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb3e569b80 10 client.27053 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb3e569b80 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb3e569b80 20 client.27053 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555585556a30 0x555585558e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb3e569b80 4 client.27053 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55558550c8b0 con 0x555585556a30 2023-10-18T12:36:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555585556a30 0x555585558e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555585556a30 0x555585558e20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb82fc141ae4402 server_cookie=dac674e14b309e95 in_seq=0 out_seq=0 2023-10-18T12:36:45.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.557+0000 7fcb03fff700 10 client.27053 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:45.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.558+0000 7fcb03fff700 1 -- 172.21.15.70:0/1240609734 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55558550c8b0 con 0x555585556a30 2023-10-18T12:36:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.558+0000 7fcb03fff700 10 client.27053 handle_command_reply: tid=0 2023-10-18T12:36:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.558+0000 7fcb3e569b80 1 client.27053 shutdown 2023-10-18T12:36:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 msgr2=0x555585546d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 0x555585546d70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb140caf10 tx=0x7fcb14098750 comp rx=0 tx=0).stop 2023-10-18T12:36:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 shutdown_connections 2023-10-18T12:36:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x555585556a30 0x555585558e20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585543500 0x5555855472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x555585546830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb03fff700 0 client.27053 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1240609734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585542be0 0x555585546d70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 >> 172.21.15.70:0/1240609734 conn(0x5555854b1720 msgr2=0x55558553c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 shutdown_connections 2023-10-18T12:36:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1240609734 wait complete. 2023-10-18T12:36:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.559+0000 7fcb3e569b80 20 client.27053 trim_cache size 0 max 16384 2023-10-18T12:36:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 Processor -- start 2023-10-18T12:36:45.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 -- start start 2023-10-18T12:36:45.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x55558554deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585537810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 0x555585540720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555585310920 con 0x55558553d5d0 2023-10-18T12:36:45.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555585547e30 con 0x555585543500 2023-10-18T12:36:45.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.580+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555585534230 con 0x555585542be0 2023-10-18T12:36:45.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x55558554deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2919c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 0x555585540720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2919c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 0x555585540720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39956/0 (socket says 172.21.15.70:39956) 2023-10-18T12:36:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x55558554deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45404/0 (socket says 172.21.15.70:45404) 2023-10-18T12:36:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2919c700 1 -- 172.21.15.70:0/571560993 learned_addr learned my addr 172.21.15.70:0/571560993 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585537810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 -- 172.21.15.70:0/571560993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 msgr2=0x555585537810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585537810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 -- 172.21.15.70:0/571560993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 msgr2=0x555585540720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 0x555585540720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2899b700 1 -- 172.21.15.70:0/571560993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555852bade0 con 0x55558553d5d0 2023-10-18T12:36:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.581+0000 7fcb2919c700 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 0x555585540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.582+0000 7fcb2899b700 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x55558554deb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x555585559770 tx=0x7fcb20009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9574e77c067e896 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.582+0000 7fcb0b7fe700 1 -- 172.21.15.70:0/571560993 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20017040 con 0x55558553d5d0 2023-10-18T12:36:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.582+0000 7fcb0b7fe700 1 -- 172.21.15.70:0/571560993 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb20025020 con 0x55558553d5d0 2023-10-18T12:36:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.582+0000 7fcb0b7fe700 1 -- 172.21.15.70:0/571560993 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20004410 con 0x55558553d5d0 2023-10-18T12:36:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 -- 172.21.15.70:0/571560993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 msgr2=0x55558554deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x55558554deb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x555585559770 tx=0x7fcb20009310 comp rx=0 tx=0).stop 2023-10-18T12:36:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 -- 172.21.15.70:0/571560993 shutdown_connections 2023-10-18T12:36:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555585543500 0x555585540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55558553d5d0 0x55558554deb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/571560993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x555585537810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 -- 172.21.15.70:0/571560993 >> 172.21.15.70:0/571560993 conn(0x5555854b1720 msgr2=0x55558553c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 -- 172.21.15.70:0/571560993 shutdown_connections 2023-10-18T12:36:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 -- 172.21.15.70:0/571560993 wait complete. 2023-10-18T12:36:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.583+0000 7fcb3e569b80 1 -- start start 2023-10-18T12:36:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 Processor -- start 2023-10-18T12:36:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558554db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x55558554c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558554c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558554cb50 con 0x555585543500 2023-10-18T12:36:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558554ccc0 con 0x55558553d5d0 2023-10-18T12:36:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558554ce80 con 0x555585542be0 2023-10-18T12:36:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb3e569b80 10 client.0 _ll_register_callbacks cb 0x555585376a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb2899b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558554db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb2899b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558554db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39970/0 (socket says 172.21.15.70:39970) 2023-10-18T12:36:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb2899b700 1 -- 172.21.15.70:0/1008093751 learned_addr learned my addr 172.21.15.70:0/1008093751 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x55558554c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.584+0000 7fcb2919c700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558554c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2899b700 1 -- 172.21.15.70:0/1008093751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 msgr2=0x55558554c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x55558554c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2899b700 1 -- 172.21.15.70:0/1008093751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 msgr2=0x55558554c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:45.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558554c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2899b700 1 -- 172.21.15.70:0/1008093751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb241100a0 con 0x55558553d5d0 2023-10-18T12:36:45.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2919c700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558554c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:45.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x55558554c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:45.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb2899b700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558554db50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009860 tx=0x7fcb200057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec95a476766df291 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb03fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:45.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb03fff700 1 -- 172.21.15.70:0/1008093751 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20017040 con 0x55558553d5d0 2023-10-18T12:36:45.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb03fff700 1 -- 172.21.15.70:0/1008093751 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fcb20026030 con 0x55558553d5d0 2023-10-18T12:36:45.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.585+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb1409a1a0 con 0x55558553d5d0 2023-10-18T12:36:45.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb03fff700 1 -- 172.21.15.70:0/1008093751 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb20021500 con 0x55558553d5d0 2023-10-18T12:36:45.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 10 client.27059 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:45.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fcb20021a00 con 0x55558553d5d0 2023-10-18T12:36:45.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb03fff700 1 -- 172.21.15.70:0/1008093751 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb1409a1a0 con 0x55558553d5d0 2023-10-18T12:36:45.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb03fff700 1 -- 172.21.15.70:0/1008093751 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcb200105f0 con 0x55558553d5d0 2023-10-18T12:36:45.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 10 client.27059 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:45.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 10 client.27059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:45.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 20 client.27059 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:45.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555855abd80 0x5555855ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 4 client.27059 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:36:45.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555585488e50 con 0x5555855abd80 2023-10-18T12:36:45.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.586+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555855abd80 0x5555855ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.587+0000 7fcb03fff700 10 client.27059 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:45.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.587+0000 7fcb1bfff700 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555855abd80 0x5555855ae1a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da505e123b68828 server_cookie=eed83c323152c395 in_seq=0 out_seq=0 2023-10-18T12:36:45.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.587+0000 7fcb03fff700 1 -- 172.21.15.70:0/1008093751 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555585488e50 con 0x5555855abd80 2023-10-18T12:36:45.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.587+0000 7fcb03fff700 10 client.27059 handle_command_reply: tid=0 2023-10-18T12:36:45.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.587+0000 7fcb3e569b80 1 client.27059 shutdown 2023-10-18T12:36:45.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 msgr2=0x55558554db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558554db50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009860 tx=0x7fcb200057e0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 shutdown_connections 2023-10-18T12:36:45.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5555855abd80 0x5555855ae1a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555585543500 0x55558554c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55558553d5d0 0x55558554db50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 --2- 172.21.15.70:0/1008093751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555585542be0 0x55558554c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb03fff700 0 client.27059 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:45.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 >> 172.21.15.70:0/1008093751 conn(0x5555854b1720 msgr2=0x555585537810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 shutdown_connections 2023-10-18T12:36:45.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 1 -- 172.21.15.70:0/1008093751 wait complete. 2023-10-18T12:36:45.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.588+0000 7fcb3e569b80 20 client.27059 trim_cache size 0 max 16384 2023-10-18T12:36:45.606 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:36:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:36:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:36:45.607 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:36:45.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcb0c0a1670 msgr2=0x7fcb0c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcb0c0a1670 0x7fcb0c0a3b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0060b0 tx=0x7fcb1c00b330 comp rx=0 tx=0).stop 2023-10-18T12:36:45.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 msgr2=0x7fcb24199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 0x7fcb24199420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9a0 tx=0x7fcb1400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 shutdown_connections 2023-10-18T12:36:45.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2410cff0 0x7fcb2419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb24072ba0 0x7fcb24198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcb0c0a1670 0x7fcb0c0a3b30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 --2- 172.21.15.70:0/3840138276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240734c0 0x7fcb24199420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 >> 172.21.15.70:0/3840138276 conn(0x7fcb240fc200 msgr2=0x7fcb240fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 shutdown_connections 2023-10-18T12:36:45.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.589+0000 7fcb2abff700 1 -- 172.21.15.70:0/3840138276 wait complete. 2023-10-18T12:36:45.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:36:45.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:36:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 -- 172.21.15.70:0/2892596996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5208072c20 msgr2=0x7f5208073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2892596996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5208072c20 0x7f5208073000 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009990 tx=0x7f51f8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 -- 172.21.15.70:0/2892596996 shutdown_connections 2023-10-18T12:36:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2892596996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f520810ce00 0x7f520810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2892596996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5208072c20 0x7f5208073000 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2892596996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208073540 0x7f520810c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 -- 172.21.15.70:0/2892596996 >> 172.21.15.70:0/2892596996 conn(0x7f52080fc0b0 msgr2=0x7f52080fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:45.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 -- 172.21.15.70:0/2892596996 shutdown_connections 2023-10-18T12:36:45.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.861+0000 7f520e5ff700 1 -- 172.21.15.70:0/2892596996 wait complete. 2023-10-18T12:36:45.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.862+0000 7f520e5ff700 1 Processor -- start 2023-10-18T12:36:45.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.862+0000 7f520e5ff700 1 -- start start 2023-10-18T12:36:45.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520e5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 0x7f520819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520e5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5208073540 0x7f520819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520e5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520810ce00 0x7f5208198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520e5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5208199160 con 0x7f520810ce00 2023-10-18T12:36:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520e5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52081992d0 con 0x7f5208072c20 2023-10-18T12:36:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520e5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5208199490 con 0x7f5208073540 2023-10-18T12:36:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 0x7f520819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 0x7f520819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39982/0 (socket says 172.21.15.70:39982) 2023-10-18T12:36:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 -- 172.21.15.70:0/2261901771 learned_addr learned my addr 172.21.15.70:0/2261901771 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f520cb9c700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520810ce00 0x7f5208198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f52077fe700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5208073540 0x7f520819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 -- 172.21.15.70:0/2261901771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5208073540 msgr2=0x7f520819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5208073540 0x7f520819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 -- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520810ce00 msgr2=0x7f5208198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520810ce00 0x7f5208198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.863+0000 7f5207fff700 1 -- 172.21.15.70:0/2261901771 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009520 con 0x7f5208072c20 2023-10-18T12:36:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.864+0000 7f5207fff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 0x7f520819e980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f51f8004730 tx=0x7f51f8004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.864+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f801d070 con 0x7f5208072c20 2023-10-18T12:36:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.864+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f8022470 con 0x7f5208072c20 2023-10-18T12:36:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.864+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f80095f0 con 0x7f5208072c20 2023-10-18T12:36:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.864+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8017650 con 0x7f5208072c20 2023-10-18T12:36:45.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.864+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52081a5790 con 0x7f5208072c20 2023-10-18T12:36:45.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.865+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5208108010 con 0x7f5208072c20 2023-10-18T12:36:45.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.867+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f51f80225e0 con 0x7f5208072c20 2023-10-18T12:36:45.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.868+0000 7f52057fa700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f51f00a15f0 0x7f51f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:45.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.868+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f51f80ce660 con 0x7f5208072c20 2023-10-18T12:36:45.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.869+0000 7f52077fe700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f51f00a15f0 0x7f51f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.869+0000 7f52077fe700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f51f00a15f0 0x7f51f00a3ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5208199c70 tx=0x7f51fc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:45.872+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f51f805d100 con 0x7f5208072c20 2023-10-18T12:36:45.876 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1224312 11.44 MB/sec execute 434 sec latency 27.876 ms 2023-10-18T12:36:46.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.172+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5208199f70 con 0x7f5208072c20 2023-10-18T12:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.172+0000 7f52057fa700 1 -- 172.21.15.70:0/2261901771 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f51f8097650 con 0x7f5208072c20 2023-10-18T12:36:46.176 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:36:46.176 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:36:46.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f51f00a15f0 msgr2=0x7f51f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f51f00a15f0 0x7f51f00a3ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5208199c70 tx=0x7f51fc009260 comp rx=0 tx=0).stop 2023-10-18T12:36:46.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 msgr2=0x7f520819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 0x7f520819e980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f51f8004730 tx=0x7f51f8004760 comp rx=0 tx=0).stop 2023-10-18T12:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 shutdown_connections 2023-10-18T12:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520810ce00 0x7f5208198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f51f00a15f0 0x7f51f00a3ab0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5208072c20 0x7f520819e980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 --2- 172.21.15.70:0/2261901771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5208073540 0x7f520819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 >> 172.21.15.70:0/2261901771 conn(0x7f52080fc0b0 msgr2=0x7f52080fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 shutdown_connections 2023-10-18T12:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.176+0000 7f520e5ff700 1 -- 172.21.15.70:0/2261901771 wait complete. 2023-10-18T12:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:36:46.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:36:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 -- 172.21.15.70:0/3077530007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 msgr2=0x7f7a44106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 --2- 172.21.15.70:0/3077530007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44106b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009990 tx=0x7f7a34009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 -- 172.21.15.70:0/3077530007 shutdown_connections 2023-10-18T12:36:46.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 --2- 172.21.15.70:0/3077530007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 --2- 172.21.15.70:0/3077530007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44106b00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 --2- 172.21.15.70:0/3077530007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a44100710 0x7f7a44100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 -- 172.21.15.70:0/3077530007 >> 172.21.15.70:0/3077530007 conn(0x7f7a440fc200 msgr2=0x7f7a440fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 -- 172.21.15.70:0/3077530007 shutdown_connections 2023-10-18T12:36:46.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.406+0000 7f7a48dff700 1 -- 172.21.15.70:0/3077530007 wait complete. 2023-10-18T12:36:46.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 Processor -- start 2023-10-18T12:36:46.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 -- start start 2023-10-18T12:36:46.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a44100710 0x7f7a4419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44199200 con 0x7f7a44100710 2023-10-18T12:36:46.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44199370 con 0x7f7a44106720 2023-10-18T12:36:46.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.407+0000 7f7a48dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44199530 con 0x7f7a441010d0 2023-10-18T12:36:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a42d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a44100710 0x7f7a4419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49672/0 (socket says 172.21.15.70:49672) 2023-10-18T12:36:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 -- 172.21.15.70:0/509601955 learned_addr learned my addr 172.21.15.70:0/509601955 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a42d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:39998/0 (socket says 172.21.15.70:39998) 2023-10-18T12:36:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 -- 172.21.15.70:0/509601955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 msgr2=0x7f7a44198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 -- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a44100710 msgr2=0x7f7a4419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a44100710 0x7f7a4419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 -- 172.21.15.70:0/509601955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38009520 con 0x7f7a441010d0 2023-10-18T12:36:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a42d9d700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4419efb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800eb30 tx=0x7f7a3800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a337fe700 1 -- 172.21.15.70:0/509601955 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38009880 con 0x7f7a441010d0 2023-10-18T12:36:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.408+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a340095f0 con 0x7f7a441010d0 2023-10-18T12:36:46.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.409+0000 7f7a337fe700 1 -- 172.21.15.70:0/509601955 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a380044e0 con 0x7f7a441010d0 2023-10-18T12:36:46.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.409+0000 7f7a337fe700 1 -- 172.21.15.70:0/509601955 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38003d80 con 0x7f7a441010d0 2023-10-18T12:36:46.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.409+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a44199a00 con 0x7f7a441010d0 2023-10-18T12:36:46.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.412+0000 7f7a337fe700 1 -- 172.21.15.70:0/509601955 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7a38003fa0 con 0x7f7a441010d0 2023-10-18T12:36:46.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 Processor -- start 2023-10-18T12:36:46.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 -- start start 2023-10-18T12:36:46.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a337fe700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7a2c0a1670 0x7f7a2c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901920fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901921500 0x56490191b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a337fe700 1 -- 172.21.15.70:0/509601955 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f7a38014070 con 0x7f7a441010d0 2023-10-18T12:36:46.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x56490191ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56490188e6e0 con 0x564901921500 2023-10-18T12:36:46.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564901874c60 con 0x564901920be0 2023-10-18T12:36:46.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564901872580 con 0x56490191b5d0 2023-10-18T12:36:46.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.413+0000 7f7a42d9d700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7a2c0a1670 0x7f7a2c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901920fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901920fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40014/0 (socket says 172.21.15.70:40014) 2023-10-18T12:36:46.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 -- 172.21.15.70:0/797212526 learned_addr learned my addr 172.21.15.70:0/797212526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:46.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901921500 0x56490191b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 -- 172.21.15.70:0/797212526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 msgr2=0x56490191ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x56490191ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 -- 172.21.15.70:0/797212526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901921500 msgr2=0x56490191b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901921500 0x56490191b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 -- 172.21.15.70:0/797212526 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56490182b3f0 con 0x564901920be0 2023-10-18T12:36:46.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a42d9d700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7a2c0a1670 0x7f7a2c0a3b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c003e40 tx=0x7f7a3c009920 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:46.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.414+0000 7f7a4259c700 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901920fc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005ec0 tx=0x7f7a34005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d0b2947bb967a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:46.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a30ff9700 1 -- 172.21.15.70:0/797212526 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3401d070 con 0x564901920be0 2023-10-18T12:36:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a30ff9700 1 -- 172.21.15.70:0/797212526 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a3401e040 con 0x564901920be0 2023-10-18T12:36:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a30ff9700 1 -- 172.21.15.70:0/797212526 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34004880 con 0x564901920be0 2023-10-18T12:36:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/797212526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 msgr2=0x564901920fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901920fc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005ec0 tx=0x7f7a34005ef0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/797212526 shutdown_connections 2023-10-18T12:36:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x56490191ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901920fc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/797212526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901921500 0x56490191b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/797212526 >> 172.21.15.70:0/797212526 conn(0x56490188f720 msgr2=0x564901915b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/797212526 shutdown_connections 2023-10-18T12:36:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.415+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/797212526 wait complete. 2023-10-18T12:36:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.416+0000 7f7a5c6ddb80 1 -- start start 2023-10-18T12:36:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.416+0000 7f7a5c6ddb80 1 Processor -- start 2023-10-18T12:36:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x564901927410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x564901924700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901921500 0x564901924c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56490186de70 con 0x564901920be0 2023-10-18T12:36:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56490180a710 con 0x564901921500 2023-10-18T12:36:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56490180a3b0 con 0x56490191b5d0 2023-10-18T12:36:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a5c6ddb80 10 client.0 _ll_register_callbacks cb 0x564901754a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x564901927410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x564901927410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49688/0 (socket says 172.21.15.70:49688) 2023-10-18T12:36:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 -- 172.21.15.70:0/2121009035 learned_addr learned my addr 172.21.15.70:0/2121009035 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a42d9d700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901921500 0x564901924c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x564901924700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 -- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901921500 msgr2=0x564901924c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901921500 0x564901924c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 -- 172.21.15.70:0/2121009035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 msgr2=0x564901924700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x564901924700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 -- 172.21.15.70:0/2121009035 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4410dee0 con 0x56490191b5d0 2023-10-18T12:36:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x564901924700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:36:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a4259c700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x564901927410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009ce0 tx=0x7f7a3400b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e9542def879221 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.417+0000 7f7a29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:36:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2121009035 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34032070 con 0x56490191b5d0 2023-10-18T12:36:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2121009035 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f7a34004710 con 0x56490191b5d0 2023-10-18T12:36:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564901872ef0 con 0x56490191b5d0 2023-10-18T12:36:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2121009035 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a3401d070 con 0x56490191b5d0 2023-10-18T12:36:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a5c6ddb80 10 client.27058 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564901698de0 con 0x56490191b5d0 2023-10-18T12:36:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2121009035 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564901872ef0 con 0x56490191b5d0 2023-10-18T12:36:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2121009035 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a34035020 con 0x56490191b5d0 2023-10-18T12:36:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a5c6ddb80 10 client.27058 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a5c6ddb80 10 client.27058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.418+0000 7f7a5c6ddb80 20 client.27058 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.419+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564901934890 0x564901936c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.419+0000 7f7a5c6ddb80 4 client.27058 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.419+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649018ea8b0 con 0x564901934890 2023-10-18T12:36:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.419+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564901934890 0x564901936c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.419+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564901934890 0x564901936c80 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0c661abdd8867 server_cookie=de5bde394949e2b in_seq=0 out_seq=0 2023-10-18T12:36:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.419+0000 7f7a29ffb700 10 client.27058 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2121009035 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649018ea8b0 con 0x564901934890 2023-10-18T12:36:46.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a29ffb700 10 client.27058 handle_command_reply: tid=0 2023-10-18T12:36:46.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 client.27058 shutdown 2023-10-18T12:36:46.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 msgr2=0x564901927410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x564901927410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009ce0 tx=0x7f7a3400b980 comp rx=0 tx=0).stop 2023-10-18T12:36:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 shutdown_connections 2023-10-18T12:36:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564901934890 0x564901936c80 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901921500 0x564901924c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56490191b5d0 0x564901927410 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2121009035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x564901924700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a29ffb700 0 client.27058 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 >> 172.21.15.70:0/2121009035 conn(0x56490188f720 msgr2=0x564901915ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 shutdown_connections 2023-10-18T12:36:46.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.420+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2121009035 wait complete. 2023-10-18T12:36:46.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.421+0000 7f7a5c6ddb80 20 client.27058 trim_cache size 0 max 16384 2023-10-18T12:36:46.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.434+0000 7f7a5c6ddb80 1 Processor -- start 2023-10-18T12:36:46.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.434+0000 7f7a5c6ddb80 1 -- start start 2023-10-18T12:36:46.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.434+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56490191b5d0 0x56490192be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901926f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x564901923f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649016ee920 con 0x56490191b5d0 2023-10-18T12:36:46.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56490192c440 con 0x564901920be0 2023-10-18T12:36:46.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564901937330 con 0x564901921500 2023-10-18T12:36:46.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901926f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56490191b5d0 0x56490192be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901926f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40024/0 (socket says 172.21.15.70:40024) 2023-10-18T12:36:46.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 -- 172.21.15.70:0/2057488976 learned_addr learned my addr 172.21.15.70:0/2057488976 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:46.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a42d9d700 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x564901923f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 -- 172.21.15.70:0/2057488976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 msgr2=0x564901923f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x564901923f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 -- 172.21.15.70:0/2057488976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56490191b5d0 msgr2=0x56490192be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56490191b5d0 0x56490192be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.435+0000 7f7a41d9b700 1 -- 172.21.15.70:0/2057488976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564901681c60 con 0x564901920be0 2023-10-18T12:36:46.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901926f40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a3809a2e0 tx=0x7f7a3809a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c721ccb5db936a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:46.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a317fa700 1 -- 172.21.15.70:0/2057488976 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3809ace0 con 0x564901920be0 2023-10-18T12:36:46.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a317fa700 1 -- 172.21.15.70:0/2057488976 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a3809a5e0 con 0x564901920be0 2023-10-18T12:36:46.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a317fa700 1 -- 172.21.15.70:0/2057488976 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38098980 con 0x564901920be0 2023-10-18T12:36:46.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2057488976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 msgr2=0x564901926f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901926f40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a3809a2e0 tx=0x7f7a3809a440 comp rx=0 tx=0).stop 2023-10-18T12:36:46.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2057488976 shutdown_connections 2023-10-18T12:36:46.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x564901923f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56490191b5d0 0x56490192be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.436+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2057488976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564901920be0 0x564901926f40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.437+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2057488976 >> 172.21.15.70:0/2057488976 conn(0x56490188f720 msgr2=0x564901922ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.437+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2057488976 shutdown_connections 2023-10-18T12:36:46.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.437+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2057488976 wait complete. 2023-10-18T12:36:46.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.437+0000 7f7a5c6ddb80 1 -- start start 2023-10-18T12:36:46.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.437+0000 7f7a5c6ddb80 1 Processor -- start 2023-10-18T12:36:46.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 0x56490191d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x56490191d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x5649019885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56490191df20 con 0x564901920be0 2023-10-18T12:36:46.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564901988b30 con 0x56490191b5d0 2023-10-18T12:36:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564901988cf0 con 0x564901921500 2023-10-18T12:36:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a5c6ddb80 10 client.0 _ll_register_callbacks cb 0x564901754a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 0x56490191d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x56490191d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 0x56490191d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40028/0 (socket says 172.21.15.70:40028) 2023-10-18T12:36:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a42d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x5649019885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 -- 172.21.15.70:0/2339848907 learned_addr learned my addr 172.21.15.70:0/2339848907 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 -- 172.21.15.70:0/2339848907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 msgr2=0x5649019885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x5649019885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 -- 172.21.15.70:0/2339848907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 msgr2=0x56490191d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x56490191d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.438+0000 7f7a4259c700 1 -- 172.21.15.70:0/2339848907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a44199a00 con 0x56490191b5d0 2023-10-18T12:36:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a4259c700 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 0x56490191d290 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400f860 tx=0x7f7a3400b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c425a00ff3cbce50 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:36:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2339848907 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34032070 con 0x56490191b5d0 2023-10-18T12:36:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2339848907 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f7a34005990 con 0x56490191b5d0 2023-10-18T12:36:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56490191c040 con 0x56490191b5d0 2023-10-18T12:36:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2339848907 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a3401d040 con 0x56490191b5d0 2023-10-18T12:36:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a5c6ddb80 10 client.27083 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5649016bf250 con 0x56490191b5d0 2023-10-18T12:36:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.439+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2339848907 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56490191c040 con 0x56490191b5d0 2023-10-18T12:36:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2339848907 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a34035070 con 0x56490191b5d0 2023-10-18T12:36:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a5c6ddb80 10 client.27083 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a5c6ddb80 10 client.27083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a5c6ddb80 20 client.27083 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56490198a0e0 0x56490198c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a5c6ddb80 4 client.27083 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:36:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f7a3401e040 con 0x56490198a0e0 2023-10-18T12:36:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56490198a0e0 0x56490198c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a41d9b700 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56490198a0e0 0x56490198c500 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db33179981dbd034 server_cookie=abcf65e3cb199baf in_seq=0 out_seq=0 2023-10-18T12:36:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.440+0000 7f7a29ffb700 10 client.27083 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a29ffb700 1 -- 172.21.15.70:0/2339848907 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f7a3401e040 con 0x56490198a0e0 2023-10-18T12:36:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a29ffb700 10 client.27083 handle_command_reply: tid=0 2023-10-18T12:36:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 client.27083 shutdown 2023-10-18T12:36:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 msgr2=0x56490191d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 0x56490191d290 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400f860 tx=0x7f7a3400b410 comp rx=0 tx=0).stop 2023-10-18T12:36:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 shutdown_connections 2023-10-18T12:36:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56490198a0e0 0x56490198c500 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564901921500 0x5649019885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a29ffb700 0 client.27083 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56490191b5d0 0x56490191d290 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 --2- 172.21.15.70:0/2339848907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564901920be0 0x56490191d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 >> 172.21.15.70:0/2339848907 conn(0x56490188f720 msgr2=0x564901923f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 shutdown_connections 2023-10-18T12:36:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 1 -- 172.21.15.70:0/2339848907 wait complete. 2023-10-18T12:36:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.441+0000 7f7a5c6ddb80 20 client.27083 trim_cache size 0 max 16384 2023-10-18T12:36:46.467 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:36:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7a2c0a1670 msgr2=0x7f7a2c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7a2c0a1670 0x7f7a2c0a3b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c003e40 tx=0x7f7a3c009920 comp rx=0 tx=0).stop 2023-10-18T12:36:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 msgr2=0x7f7a4419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4419efb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800eb30 tx=0x7f7a3800ee40 comp rx=0 tx=0).stop 2023-10-18T12:36:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 shutdown_connections 2023-10-18T12:36:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f7a2c0a1670 0x7f7a2c0a3b30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a44106720 0x7f7a44198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a44100710 0x7f7a4419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 --2- 172.21.15.70:0/509601955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a441010d0 0x7f7a4419efb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 >> 172.21.15.70:0/509601955 conn(0x7f7a440fc200 msgr2=0x7f7a4410be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 shutdown_connections 2023-10-18T12:36:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:46.442+0000 7f7a48dff700 1 -- 172.21.15.70:0/509601955 wait complete. 2023-10-18T12:36:46.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:46.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:46 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:46.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:46 smithi070 ceph-mon[98639]: pgmap v645: 129 pgs: 129 active+clean; 602 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2023-10-18T12:36:46.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:46 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2261901771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:36:46.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1226175 11.44 MB/sec execute 435 sec latency 17.719 ms 2023-10-18T12:36:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:46 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:46.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:46 smithi152 ceph-mon[134767]: pgmap v645: 129 pgs: 129 active+clean; 602 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2023-10-18T12:36:46.923 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:46 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2261901771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:36:46.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:46 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:46 smithi186 ceph-mon[135253]: pgmap v645: 129 pgs: 129 active+clean; 602 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2023-10-18T12:36:46.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:46 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2261901771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:36:47.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:36:47.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:36:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 -- 172.21.15.70:0/1990045867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 msgr2=0x7f39ac100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 --2- 172.21.15.70:0/1990045867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 0x7f39ac100b90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009990 tx=0x7f39a0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 -- 172.21.15.70:0/1990045867 shutdown_connections 2023-10-18T12:36:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 --2- 172.21.15.70:0/1990045867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 --2- 172.21.15.70:0/1990045867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac106720 0x7f39ac106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 --2- 172.21.15.70:0/1990045867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 0x7f39ac100b90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 -- 172.21.15.70:0/1990045867 >> 172.21.15.70:0/1990045867 conn(0x7f39ac0fc200 msgr2=0x7f39ac0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 -- 172.21.15.70:0/1990045867 shutdown_connections 2023-10-18T12:36:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.696+0000 7f39b27ff700 1 -- 172.21.15.70:0/1990045867 wait complete. 2023-10-18T12:36:47.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.697+0000 7f39b27ff700 1 Processor -- start 2023-10-18T12:36:47.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.697+0000 7f39b27ff700 1 -- start start 2023-10-18T12:36:47.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b27ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 0x7f39ac198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b27ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b27ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac106720 0x7f39ac19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b27ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19dcc0 con 0x7f39ac100710 2023-10-18T12:36:47.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b27ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19de30 con 0x7f39ac1010d0 2023-10-18T12:36:47.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b27ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19dff0 con 0x7f39ac106720 2023-10-18T12:36:47.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 0x7f39ac198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40042/0 (socket says 172.21.15.70:40042) 2023-10-18T12:36:47.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39ab7fe700 1 -- 172.21.15.70:0/2160837998 learned_addr learned my addr 172.21.15.70:0/2160837998 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:47.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.698+0000 7f39b0d9c700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac106720 0x7f39ac19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39ab7fe700 1 -- 172.21.15.70:0/2160837998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac106720 msgr2=0x7f39ac19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac106720 0x7f39ac19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39ab7fe700 1 -- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 msgr2=0x7f39ac198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 0x7f39ac198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39ab7fe700 1 -- 172.21.15.70:0/2160837998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00095f0 con 0x7f39ac1010d0 2023-10-18T12:36:47.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac1992e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39a0005910 tx=0x7f39a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39a97fa700 1 -- 172.21.15.70:0/2160837998 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a001d070 con 0x7f39ac1010d0 2023-10-18T12:36:47.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac19e270 con 0x7f39ac1010d0 2023-10-18T12:36:47.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39a97fa700 1 -- 172.21.15.70:0/2160837998 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a0022470 con 0x7f39ac1010d0 2023-10-18T12:36:47.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39a97fa700 1 -- 172.21.15.70:0/2160837998 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0017650 con 0x7f39ac1010d0 2023-10-18T12:36:47.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.699+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac1a5b10 con 0x7f39ac1010d0 2023-10-18T12:36:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.704+0000 7f39a97fa700 1 -- 172.21.15.70:0/2160837998 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f39a00225e0 con 0x7f39ac1010d0 2023-10-18T12:36:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.705+0000 7f39c619bb80 1 Processor -- start 2023-10-18T12:36:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.705+0000 7f39c619bb80 1 -- start start 2023-10-18T12:36:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.705+0000 7f39a97fa700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f39940a15f0 0x7f39940a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.705+0000 7f39a97fa700 1 -- 172.21.15.70:0/2160837998 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f39a00ce680 con 0x7f39ac1010d0 2023-10-18T12:36:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.705+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.705+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e7ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39b0d9c700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f39940a15f0 0x7f39940a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39c619bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959df1760 con 0x558959e7e650 2023-10-18T12:36:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39c619bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959dd7ce0 con 0x558959e83c60 2023-10-18T12:36:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39c619bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959dd5600 con 0x558959e84580 2023-10-18T12:36:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39abfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39abfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e84040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40052/0 (socket says 172.21.15.70:40052) 2023-10-18T12:36:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39abfff700 1 -- 172.21.15.70:0/4067579272 learned_addr learned my addr 172.21.15.70:0/4067579272 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39b0d9c700 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e7ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 -- 172.21.15.70:0/4067579272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 msgr2=0x558959e84040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e84040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 -- 172.21.15.70:0/4067579272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 msgr2=0x558959e7ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e7ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 -- 172.21.15.70:0/4067579272 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558959d8e470 con 0x558959e84580 2023-10-18T12:36:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39b0d9c700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f39940a15f0 0x7f39940a3ab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f39a400f870 tx=0x7f39a400fc30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.706+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e7e110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39a00cba70 tx=0x7f39a0022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=928023f96b757470 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39927fc700 1 -- 172.21.15.70:0/4067579272 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a009ae10 con 0x558959e84580 2023-10-18T12:36:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39927fc700 1 -- 172.21.15.70:0/4067579272 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a009bbb0 con 0x558959e84580 2023-10-18T12:36:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39927fc700 1 -- 172.21.15.70:0/4067579272 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a00ce2d0 con 0x558959e84580 2023-10-18T12:36:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 -- 172.21.15.70:0/4067579272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 msgr2=0x558959e7e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e7e110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39a00cba70 tx=0x7f39a0022f90 comp rx=0 tx=0).stop 2023-10-18T12:36:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 -- 172.21.15.70:0/4067579272 shutdown_connections 2023-10-18T12:36:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e7ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e84040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 --2- 172.21.15.70:0/4067579272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e7e110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.707+0000 7f39c619bb80 1 -- 172.21.15.70:0/4067579272 >> 172.21.15.70:0/4067579272 conn(0x558959df27a0 msgr2=0x558959e78be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.708+0000 7f39c619bb80 1 -- 172.21.15.70:0/4067579272 shutdown_connections 2023-10-18T12:36:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.708+0000 7f39c619bb80 1 -- 172.21.15.70:0/4067579272 wait complete. 2023-10-18T12:36:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.708+0000 7f39c619bb80 1 -- start start 2023-10-18T12:36:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.708+0000 7f39c619bb80 1 Processor -- start 2023-10-18T12:36:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e87880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959dd0ef0 con 0x558959e7e650 2023-10-18T12:36:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959d6d790 con 0x558959e83c60 2023-10-18T12:36:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959d6d430 con 0x558959e84580 2023-10-18T12:36:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39c619bb80 10 client.0 _ll_register_callbacks cb 0x558959cb7b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39b0d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45524/0 (socket says 172.21.15.70:45524) 2023-10-18T12:36:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39b0d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e87f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49746/0 (socket says 172.21.15.70:49746) 2023-10-18T12:36:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 -- 172.21.15.70:0/2231920266 learned_addr learned my addr 172.21.15.70:0/2231920266 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e87880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 -- 172.21.15.70:0/2231920266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 msgr2=0x558959e87f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e87f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 -- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 msgr2=0x558959e87880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.709+0000 7f39abfff700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e87880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39abfff700 1 -- 172.21.15.70:0/2231920266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac1a5b10 con 0x558959e7e650 2023-10-18T12:36:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39b0d9c700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e87f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e87880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39abfff700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8a590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e610 tx=0x7f399c00e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b58c4ec54705cc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f398b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:36:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f398b7fe700 1 -- 172.21.15.70:0/2231920266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c01d070 con 0x558959e7e650 2023-10-18T12:36:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558959dd5f70 con 0x558959e7e650 2023-10-18T12:36:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f398b7fe700 1 -- 172.21.15.70:0/2231920266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f399c00d550 con 0x558959e7e650 2023-10-18T12:36:47.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f398b7fe700 1 -- 172.21.15.70:0/2231920266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f399c00eea0 con 0x558959e7e650 2023-10-18T12:36:47.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39c619bb80 10 client.17046 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:47.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558959bfbe60 con 0x558959e7e650 2023-10-18T12:36:47.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.710+0000 7f398b7fe700 1 -- 172.21.15.70:0/2231920266 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558959dd5f70 con 0x558959e7e650 2023-10-18T12:36:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f398b7fe700 1 -- 172.21.15.70:0/2231920266 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f399c021950 con 0x558959e7e650 2023-10-18T12:36:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39c619bb80 10 client.17046 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39c619bb80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39c619bb80 20 client.17046 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39c619bb80 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959e97c30 0x558959e9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39c619bb80 4 client.17046 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:47.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558959e4d930 con 0x558959e97c30 2023-10-18T12:36:47.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959e97c30 0x558959e9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f398b7fe700 10 client.17046 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:47.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.711+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959e97c30 0x558959e9a050 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c36fead6c66397 server_cookie=f41f12e977acc08a in_seq=0 out_seq=0 2023-10-18T12:36:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.712+0000 7f398b7fe700 1 -- 172.21.15.70:0/2231920266 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558959e4d930 con 0x558959e97c30 2023-10-18T12:36:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.712+0000 7f398b7fe700 10 client.17046 handle_command_reply: tid=0 2023-10-18T12:36:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.712+0000 7f39c619bb80 1 client.17046 shutdown 2023-10-18T12:36:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 msgr2=0x558959e8a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8a590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e610 tx=0x7f399c00e920 comp rx=0 tx=0).stop 2023-10-18T12:36:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 shutdown_connections 2023-10-18T12:36:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959e97c30 0x558959e9a050 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e84580 0x558959e87f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f398b7fe700 0 client.17046 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:47.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8a590 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 --2- 172.21.15.70:0/2231920266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e83c60 0x558959e87880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 >> 172.21.15.70:0/2231920266 conn(0x558959df27a0 msgr2=0x558959e7d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 shutdown_connections 2023-10-18T12:36:47.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 1 -- 172.21.15.70:0/2231920266 wait complete. 2023-10-18T12:36:47.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.713+0000 7f39c619bb80 20 client.17046 trim_cache size 0 max 16384 2023-10-18T12:36:47.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 Processor -- start 2023-10-18T12:36:47.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 -- start start 2023-10-18T12:36:47.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e81e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959e93400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959c519a0 con 0x558959e7e650 2023-10-18T12:36:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959e88eb0 con 0x558959e84580 2023-10-18T12:36:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.731+0000 7f39c619bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959e752b0 con 0x558959e83c60 2023-10-18T12:36:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e81e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e81e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45534/0 (socket says 172.21.15.70:45534) 2023-10-18T12:36:47.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 -- 172.21.15.70:0/1380299780 learned_addr learned my addr 172.21.15.70:0/1380299780 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:47.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39b0d9c700 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959e93400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 -- 172.21.15.70:0/1380299780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 msgr2=0x558959e8ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 -- 172.21.15.70:0/1380299780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 msgr2=0x558959e93400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959e93400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 -- 172.21.15.70:0/1380299780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558959e9ec80 con 0x558959e7e650 2023-10-18T12:36:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f39abfff700 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e81e40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f399c009340 tx=0x7f399c009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b1f83e4d7c841b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.732+0000 7f3992ffd700 1 -- 172.21.15.70:0/1380299780 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c01d070 con 0x558959e7e650 2023-10-18T12:36:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f3992ffd700 1 -- 172.21.15.70:0/1380299780 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f399c0055d0 con 0x558959e7e650 2023-10-18T12:36:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f3992ffd700 1 -- 172.21.15.70:0/1380299780 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0038e0 con 0x558959e7e650 2023-10-18T12:36:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 -- 172.21.15.70:0/1380299780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 msgr2=0x558959e81e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e81e40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f399c009340 tx=0x7f399c009de0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 -- 172.21.15.70:0/1380299780 shutdown_connections 2023-10-18T12:36:47.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959e93400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e81e40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 --2- 172.21.15.70:0/1380299780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 -- 172.21.15.70:0/1380299780 >> 172.21.15.70:0/1380299780 conn(0x558959df27a0 msgr2=0x558959e81730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 -- 172.21.15.70:0/1380299780 shutdown_connections 2023-10-18T12:36:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.733+0000 7f39c619bb80 1 -- 172.21.15.70:0/1380299780 wait complete. 2023-10-18T12:36:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.734+0000 7f39c619bb80 1 -- start start 2023-10-18T12:36:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.734+0000 7f39c619bb80 1 Processor -- start 2023-10-18T12:36:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959eeba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959e8e630 con 0x558959e7e650 2023-10-18T12:36:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959eebfb0 con 0x558959e84580 2023-10-18T12:36:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558959eec0f0 con 0x558959e83c60 2023-10-18T12:36:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39c619bb80 10 client.0 _ll_register_callbacks cb 0x558959cb7b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39b0d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959eeba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49776/0 (socket says 172.21.15.70:49776) 2023-10-18T12:36:47.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39b0d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959eeba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:40076/0 (socket says 172.21.15.70:40076) 2023-10-18T12:36:47.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39ab7fe700 1 -- 172.21.15.70:0/889399617 learned_addr learned my addr 172.21.15.70:0/889399617 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:47.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.735+0000 7f39abfff700 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39ab7fe700 1 -- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 msgr2=0x558959eeba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959eeba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39ab7fe700 1 -- 172.21.15.70:0/889399617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 msgr2=0x558959e8d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:47.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39ab7fe700 1 -- 172.21.15.70:0/889399617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558959bfbe60 con 0x558959e83c60 2023-10-18T12:36:47.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39ab7fe700 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ddc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39a009b440 tx=0x7f39a009bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb6b63f0c15b9f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f398b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:36:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f398b7fe700 1 -- 172.21.15.70:0/889399617 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a009b280 con 0x558959e83c60 2023-10-18T12:36:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f398b7fe700 1 -- 172.21.15.70:0/889399617 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f39a00b8040 con 0x558959e83c60 2023-10-18T12:36:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558959e7f0c0 con 0x558959e83c60 2023-10-18T12:36:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f398b7fe700 1 -- 172.21.15.70:0/889399617 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a009b4b0 con 0x558959e83c60 2023-10-18T12:36:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39c619bb80 10 client.27082 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.736+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558959be4ce0 con 0x558959e83c60 2023-10-18T12:36:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f398b7fe700 1 -- 172.21.15.70:0/889399617 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558959e7f0c0 con 0x558959e83c60 2023-10-18T12:36:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f398b7fe700 1 -- 172.21.15.70:0/889399617 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f39a009a9a0 con 0x558959e83c60 2023-10-18T12:36:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39c619bb80 10 client.27082 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39c619bb80 10 client.27082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39c619bb80 20 client.27082 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39c619bb80 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959eed520 0x558959eef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39c619bb80 4 client.27082 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:36:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f39a009bbb0 con 0x558959eed520 2023-10-18T12:36:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.737+0000 7f39abfff700 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959eed520 0x558959eef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.738+0000 7f39abfff700 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959eed520 0x558959eef940 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af760fc4a234dd78 server_cookie=c14d83618aa63574 in_seq=0 out_seq=0 2023-10-18T12:36:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.738+0000 7f398b7fe700 10 client.27082 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.738+0000 7f398b7fe700 1 -- 172.21.15.70:0/889399617 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f39a009bbb0 con 0x558959eed520 2023-10-18T12:36:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.738+0000 7f398b7fe700 10 client.27082 handle_command_reply: tid=0 2023-10-18T12:36:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.738+0000 7f39c619bb80 1 client.27082 shutdown 2023-10-18T12:36:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 msgr2=0x558959e8ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ddc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39a009b440 tx=0x7f39a009bd20 comp rx=0 tx=0).stop 2023-10-18T12:36:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 shutdown_connections 2023-10-18T12:36:47.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x558959eed520 0x558959eef940 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558959e84580 0x558959eeba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f398b7fe700 0 client.27082 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:47.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558959e7e650 0x558959e8d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 --2- 172.21.15.70:0/889399617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558959e83c60 0x558959e8ddc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 >> 172.21.15.70:0/889399617 conn(0x558959df27a0 msgr2=0x558959e82960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 shutdown_connections 2023-10-18T12:36:47.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 1 -- 172.21.15.70:0/889399617 wait complete. 2023-10-18T12:36:47.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.739+0000 7f39c619bb80 20 client.27082 trim_cache size 0 max 16384 2023-10-18T12:36:47.756 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:36:47.757 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:36:47.757 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "11bd9d0c-0d7d-4d55-a536-20ea1ec89941", 2023-10-18T12:36:47.757 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:36:47.757 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:36:47.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f39940a15f0 msgr2=0x7f39940a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f39940a15f0 0x7f39940a3ab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f39a400f870 tx=0x7f39a400fc30 comp rx=0 tx=0).stop 2023-10-18T12:36:47.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 msgr2=0x7f39ac1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac1992e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39a0005910 tx=0x7f39a0004730 comp rx=0 tx=0).stop 2023-10-18T12:36:47.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 shutdown_connections 2023-10-18T12:36:47.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39ac106720 0x7f39ac19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39ac100710 0x7f39ac198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f39940a15f0 0x7f39940a3ab0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 --2- 172.21.15.70:0/2160837998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ac1010d0 0x7f39ac1992e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 >> 172.21.15.70:0/2160837998 conn(0x7f39ac0fc200 msgr2=0x7f39ac10bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 shutdown_connections 2023-10-18T12:36:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.740+0000 7f39b27ff700 1 -- 172.21.15.70:0/2160837998 wait complete. 2023-10-18T12:36:47.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:36:47.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1228798 11.45 MB/sec execute 436 sec latency 13.355 ms 2023-10-18T12:36:47.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 -- 172.21.15.70:0/1280307673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24100720 msgr2=0x7f8a24100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 --2- 172.21.15.70:0/1280307673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24100720 0x7f8a24100ba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009990 tx=0x7f8a0c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 -- 172.21.15.70:0/1280307673 shutdown_connections 2023-10-18T12:36:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 --2- 172.21.15.70:0/1280307673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a241010e0 0x7f8a2410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 --2- 172.21.15.70:0/1280307673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a24106730 0x7f8a24106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 --2- 172.21.15.70:0/1280307673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24100720 0x7f8a24100ba0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 -- 172.21.15.70:0/1280307673 >> 172.21.15.70:0/1280307673 conn(0x7f8a240fc1b0 msgr2=0x7f8a240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 -- 172.21.15.70:0/1280307673 shutdown_connections 2023-10-18T12:36:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.977+0000 7f8a297ff700 1 -- 172.21.15.70:0/1280307673 wait complete. 2023-10-18T12:36:47.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.978+0000 7f8a297ff700 1 Processor -- start 2023-10-18T12:36:47.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.978+0000 7f8a297ff700 1 -- start start 2023-10-18T12:36:47.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a297ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a24100720 0x7f8a24198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a297ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 0x7f8a241992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a297ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24106730 0x7f8a2419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a297ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419dcd0 con 0x7f8a24106730 2023-10-18T12:36:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a297ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419de40 con 0x7f8a241010e0 2023-10-18T12:36:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a297ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419e000 con 0x7f8a24100720 2023-10-18T12:36:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 0x7f8a241992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a237fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24106730 0x7f8a2419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 0x7f8a241992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51166/0 (socket says 172.21.15.70:51166) 2023-10-18T12:36:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a1bfff700 1 -- 172.21.15.70:0/2830333481 learned_addr learned my addr 172.21.15.70:0/2830333481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a24100720 0x7f8a24198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a237fe700 1 -- 172.21.15.70:0/2830333481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a24100720 msgr2=0x7f8a24198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a237fe700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a24100720 0x7f8a24198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a237fe700 1 -- 172.21.15.70:0/2830333481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 msgr2=0x7f8a241992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a237fe700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 0x7f8a241992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a237fe700 1 -- 172.21.15.70:0/2830333481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009520 con 0x7f8a24106730 2023-10-18T12:36:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.979+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 0x7f8a241992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.980+0000 7f8a237fe700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24106730 0x7f8a2419d690 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00efc0 tx=0x7f8a1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.980+0000 7f8a20ff9700 1 -- 172.21.15.70:0/2830333481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c018070 con 0x7f8a24106730 2023-10-18T12:36:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.980+0000 7f8a20ff9700 1 -- 172.21.15.70:0/2830333481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1c00f040 con 0x7f8a24106730 2023-10-18T12:36:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.980+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0095f0 con 0x7f8a24106730 2023-10-18T12:36:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.980+0000 7f8a20ff9700 1 -- 172.21.15.70:0/2830333481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00cd40 con 0x7f8a24106730 2023-10-18T12:36:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.980+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a241a5940 con 0x7f8a24106730 2023-10-18T12:36:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.983+0000 7f8a20ff9700 1 -- 172.21.15.70:0/2830333481 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8a1c003a00 con 0x7f8a24106730 2023-10-18T12:36:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.984+0000 7f8a20ff9700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8a100a1670 0x7f8a100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.984+0000 7f8a3d107b80 1 Processor -- start 2023-10-18T12:36:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.984+0000 7f8a20ff9700 1 -- 172.21.15.70:0/2830333481 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f8a1c0ccff0 con 0x7f8a24106730 2023-10-18T12:36:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.984+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8a100a1670 0x7f8a100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.984+0000 7f8a3d107b80 1 -- start start 2023-10-18T12:36:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df715f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df715f5d0 0x561df715fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df70d26e0 con 0x561df7165500 2023-10-18T12:36:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df70b8c60 con 0x561df7164be0 2023-10-18T12:36:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df70b6580 con 0x561df715f5d0 2023-10-18T12:36:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a22ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df715f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df715f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38492/0 (socket says 172.21.15.70:38492) 2023-10-18T12:36:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a1bfff700 1 -- 172.21.15.70:0/327024074 learned_addr learned my addr 172.21.15.70:0/327024074 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a237fe700 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df715f5d0 0x561df715fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.985+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8a100a1670 0x7f8a100a3b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009dc0 tx=0x7f8a14009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:48.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a237fe700 1 -- 172.21.15.70:0/327024074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 msgr2=0x561df7164fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a237fe700 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7164fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a237fe700 1 -- 172.21.15.70:0/327024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 msgr2=0x561df715f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a237fe700 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df715f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a237fe700 1 -- 172.21.15.70:0/327024074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df706f3f0 con 0x561df715f5d0 2023-10-18T12:36:48.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df715f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:48.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a237fe700 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df715f5d0 0x561df715fa50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ca3a0 tx=0x7f8a1c099b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f7095d0b39e03 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:48.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a197fa700 1 -- 172.21.15.70:0/327024074 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0b9040 con 0x561df715f5d0 2023-10-18T12:36:48.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a197fa700 1 -- 172.21.15.70:0/327024074 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1c004ad0 con 0x561df715f5d0 2023-10-18T12:36:48.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.986+0000 7f8a197fa700 1 -- 172.21.15.70:0/327024074 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c099f20 con 0x561df715f5d0 2023-10-18T12:36:48.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 -- 172.21.15.70:0/327024074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df715f5d0 msgr2=0x561df715fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:48.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df715f5d0 0x561df715fa50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ca3a0 tx=0x7f8a1c099b10 comp rx=0 tx=0).stop 2023-10-18T12:36:48.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 -- 172.21.15.70:0/327024074 shutdown_connections 2023-10-18T12:36:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df715f5d0 0x561df715fa50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7164fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/327024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df715f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 -- 172.21.15.70:0/327024074 >> 172.21.15.70:0/327024074 conn(0x561df70d3720 msgr2=0x561df7159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 -- 172.21.15.70:0/327024074 shutdown_connections 2023-10-18T12:36:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 -- 172.21.15.70:0/327024074 wait complete. 2023-10-18T12:36:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.987+0000 7f8a3d107b80 1 -- start start 2023-10-18T12:36:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 Processor -- start 2023-10-18T12:36:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 0x561df7168800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7168d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df70b1e70 con 0x561df715f5d0 2023-10-18T12:36:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df704e710 con 0x561df7164be0 2023-10-18T12:36:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df704e3b0 con 0x561df7165500 2023-10-18T12:36:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a3d107b80 10 client.0 _ll_register_callbacks cb 0x561df6f98a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 0x561df7168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 0x561df7168800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38498/0 (socket says 172.21.15.70:38498) 2023-10-18T12:36:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.988+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2147171174 learned_addr learned my addr 172.21.15.70:0/2147171174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a237fe700 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2147171174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 msgr2=0x561df7169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 msgr2=0x561df7168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2147171174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2410def0 con 0x561df715f5d0 2023-10-18T12:36:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 0x561df7168800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a140045f0 tx=0x7f8a140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21dde26c553f6c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:36:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2147171174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14021070 con 0x561df715f5d0 2023-10-18T12:36:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561df70b6ef0 con 0x561df715f5d0 2023-10-18T12:36:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.989+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2147171174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f8a14003e20 con 0x561df715f5d0 2023-10-18T12:36:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2147171174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1401b430 con 0x561df715f5d0 2023-10-18T12:36:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 10 client.17064 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8a241a5940 con 0x561df715f5d0 2023-10-18T12:36:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2147171174 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561df70b6ef0 con 0x561df715f5d0 2023-10-18T12:36:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2147171174 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a14025790 con 0x561df715f5d0 2023-10-18T12:36:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 10 client.17064 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 20 client.17064 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71790e0 0x561df717b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 4 client.17064 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:36:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561df712e8b0 con 0x561df71790e0 2023-10-18T12:36:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.990+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71790e0 0x561df717b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.991+0000 7f8a0a7fc700 10 client.17064 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.991+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71790e0 0x561df717b500 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b12b450e5d8818 server_cookie=9f52444526b9bb8f in_seq=0 out_seq=0 2023-10-18T12:36:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.991+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2147171174 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561df712e8b0 con 0x561df71790e0 2023-10-18T12:36:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.991+0000 7f8a0a7fc700 10 client.17064 handle_command_reply: tid=0 2023-10-18T12:36:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 client.17064 shutdown 2023-10-18T12:36:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 msgr2=0x561df7168800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 0x561df7168800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a140045f0 tx=0x7f8a140047c0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 shutdown_connections 2023-10-18T12:36:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71790e0 0x561df717b500 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df715f5d0 0x561df7168800 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a0a7fc700 0 client.17064 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2147171174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df7164be0 0x561df7168d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 >> 172.21.15.70:0/2147171174 conn(0x561df70d3720 msgr2=0x561df715e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 shutdown_connections 2023-10-18T12:36:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2147171174 wait complete. 2023-10-18T12:36:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:47.992+0000 7f8a3d107b80 20 client.17064 trim_cache size 0 max 16384 2023-10-18T12:36:48.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.010+0000 7f8a3d107b80 1 Processor -- start 2023-10-18T12:36:48.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.010+0000 7f8a3d107b80 1 -- start start 2023-10-18T12:36:48.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 0x561df7168d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df6f32920 con 0x561df7164be0 2023-10-18T12:36:48.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df716bc90 con 0x561df715f5d0 2023-10-18T12:36:48.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df715f020 con 0x561df7165500 2023-10-18T12:36:48.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 0x561df7168d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a22ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 0x561df7168d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38508/0 (socket says 172.21.15.70:38508) 2023-10-18T12:36:48.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 -- 172.21.15.70:0/391066512 learned_addr learned my addr 172.21.15.70:0/391066512 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:48.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a237fe700 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 -- 172.21.15.70:0/391066512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 msgr2=0x561df7169290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 -- 172.21.15.70:0/391066512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 msgr2=0x561df716ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 -- 172.21.15.70:0/391066512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df6edcde0 con 0x561df7164be0 2023-10-18T12:36:48.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:48.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.011+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 0x561df7168d50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0098e0 tx=0x7f8a0c00b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf17086425f6fdd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:48.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a19ffb700 1 -- 172.21.15.70:0/391066512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c01d040 con 0x561df7164be0 2023-10-18T12:36:48.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a19ffb700 1 -- 172.21.15.70:0/391066512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c01e040 con 0x561df7164be0 2023-10-18T12:36:48.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a19ffb700 1 -- 172.21.15.70:0/391066512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c004500 con 0x561df7164be0 2023-10-18T12:36:48.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 -- 172.21.15.70:0/391066512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 msgr2=0x561df7168d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:48.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 0x561df7168d50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0098e0 tx=0x7f8a0c00b280 comp rx=0 tx=0).stop 2023-10-18T12:36:48.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 -- 172.21.15.70:0/391066512 shutdown_connections 2023-10-18T12:36:48.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7165500 0x561df7169290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/391066512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7164be0 0x561df7168d50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 -- 172.21.15.70:0/391066512 >> 172.21.15.70:0/391066512 conn(0x561df70d3720 msgr2=0x561df7159810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:48.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 -- 172.21.15.70:0/391066512 shutdown_connections 2023-10-18T12:36:48.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.012+0000 7f8a3d107b80 1 -- 172.21.15.70:0/391066512 wait complete. 2023-10-18T12:36:48.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.013+0000 7f8a3d107b80 1 -- start start 2023-10-18T12:36:48.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.013+0000 7f8a3d107b80 1 Processor -- start 2023-10-18T12:36:48.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.013+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.013+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 0x561df716ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a3d107b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df716f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df71cd110 con 0x561df7165500 2023-10-18T12:36:48.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df71cd280 con 0x561df715f5d0 2023-10-18T12:36:48.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a3d107b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df71cd440 con 0x561df7164be0 2023-10-18T12:36:48.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a3d107b80 10 client.0 _ll_register_callbacks cb 0x561df6f98a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:36:48.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a22ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 0x561df716ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 0x561df716ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:59420/0 (socket says 172.21.15.70:59420) 2023-10-18T12:36:48.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a237fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df716f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 -- 172.21.15.70:0/2097186665 learned_addr learned my addr 172.21.15.70:0/2097186665 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:36:48.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a22ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51200/0 (socket says 172.21.15.70:51200) 2023-10-18T12:36:48.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 -- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 msgr2=0x561df716e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 -- 172.21.15.70:0/2097186665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 msgr2=0x561df716f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:36:48.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df716f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 -- 172.21.15.70:0/2097186665 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a241a5940 con 0x561df7164be0 2023-10-18T12:36:48.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:36:48.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a1bfff700 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 0x561df716ee60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005d70 tx=0x7f8a0c0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42f61ce84bf05e58 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:36:48.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.014+0000 7f8a0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:36:48.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2097186665 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c01d040 con 0x561df7164be0 2023-10-18T12:36:48.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a1c004ad0 con 0x561df7164be0 2023-10-18T12:36:48.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2097186665 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f8a0c029cf0 con 0x561df7164be0 2023-10-18T12:36:48.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2097186665 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a0c005700 con 0x561df7164be0 2023-10-18T12:36:48.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 10 client.27100 fetch_fsmap learned FSMap version 20 2023-10-18T12:36:48.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8a0c00bb00 con 0x561df7164be0 2023-10-18T12:36:48.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2097186665 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1c004ad0 con 0x561df7164be0 2023-10-18T12:36:48.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2097186665 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a0c00fba0 con 0x561df7164be0 2023-10-18T12:36:48.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 10 client.27100 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:36:48.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 10 client.27100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:36:48.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 20 client.27100 populate_metadata read hostname 'smithi070' 2023-10-18T12:36:48.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71ce790 0x561df71d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:36:48.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 4 client.27100 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:36:48.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.015+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561df70aae50 con 0x561df71ce790 2023-10-18T12:36:48.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.016+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71ce790 0x561df71d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:36:48.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.016+0000 7f8a0a7fc700 10 client.27100 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:48.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.016+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71ce790 0x561df71d0bb0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9550e0818c458ea3 server_cookie=bb2e3fd5460b8168 in_seq=0 out_seq=0 2023-10-18T12:36:48.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a0a7fc700 1 -- 172.21.15.70:0/2097186665 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561df70aae50 con 0x561df71ce790 2023-10-18T12:36:48.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a0a7fc700 10 client.27100 handle_command_reply: tid=0 2023-10-18T12:36:48.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a3d107b80 1 client.27100 shutdown 2023-10-18T12:36:48.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 msgr2=0x561df716ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:48.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 0x561df716ee60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005d70 tx=0x7f8a0c0094b0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 shutdown_connections 2023-10-18T12:36:48.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.018+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561df71ce790 0x561df71d0bb0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7165500 0x561df716f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561df715f5d0 0x561df716e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a0a7fc700 0 client.27100 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:36:48.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 1 --2- 172.21.15.70:0/2097186665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561df7164be0 0x561df716ee60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 >> 172.21.15.70:0/2097186665 conn(0x561df70d3720 msgr2=0x561df7162640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:48.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 shutdown_connections 2023-10-18T12:36:48.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 1 -- 172.21.15.70:0/2097186665 wait complete. 2023-10-18T12:36:48.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a3d107b80 20 client.27100 trim_cache size 0 max 16384 2023-10-18T12:36:48.040 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:36:48.040 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:36:48.040 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:36:48.040 INFO:teuthology.orchestra.run.smithi070.stdout: "11bd9d0c-0d7d-4d55-a536-20ea1ec89941": { 2023-10-18T12:36:48.040 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:36:48.041 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "11bd9d0c-0d7d-4d55-a536-20ea1ec89941", 2023-10-18T12:36:48.041 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:36:48.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:36:48.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:36:48.041 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:36:48.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8a100a1670 msgr2=0x7f8a100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:48.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8a100a1670 0x7f8a100a3b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009dc0 tx=0x7f8a14009260 comp rx=0 tx=0).stop 2023-10-18T12:36:48.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24106730 msgr2=0x7f8a2419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:36:48.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24106730 0x7f8a2419d690 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00efc0 tx=0x7f8a1c00c3a0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 shutdown_connections 2023-10-18T12:36:48.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a24106730 0x7f8a2419d690 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a24100720 0x7f8a24198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8a100a1670 0x7f8a100a3b30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 --2- 172.21.15.70:0/2830333481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a241010e0 0x7f8a241992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:36:48.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.019+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 >> 172.21.15.70:0/2830333481 conn(0x7f8a240fc1b0 msgr2=0x7f8a240fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:36:48.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.020+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 shutdown_connections 2023-10-18T12:36:48.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:36:48.020+0000 7f8a297ff700 1 -- 172.21.15.70:0/2830333481 wait complete. 2023-10-18T12:36:48.045 INFO:tasks.cephfs.filesystem:scrub status for tag:11bd9d0c-0d7d-4d55-a536-20ea1ec89941 - {'path': '/', 'tag': '11bd9d0c-0d7d-4d55-a536-20ea1ec89941', 'options': 'recursive,force'} 2023-10-18T12:36:48.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:36:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:36:48.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:36:48.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:36:48.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:36:48.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:36:48.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:36:48.877 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1230336 11.44 MB/sec execute 437 sec latency 15.693 ms 2023-10-18T12:36:49.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:48 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:36:49.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:48 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:49.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:48 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:36:49.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:48 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:49.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:48 smithi152 ceph-mon[134767]: pgmap v646: 129 pgs: 129 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2023-10-18T12:36:49.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:48 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:36:49.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:48 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:36:49.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:48 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:49.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:48 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:36:49.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:48 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:49.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:48 smithi186 ceph-mon[135253]: pgmap v646: 129 pgs: 129 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2023-10-18T12:36:49.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:48 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:36:49.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:48 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:36:49.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:48 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:49.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:48 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:36:49.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:48 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:36:49.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:48 smithi070 ceph-mon[98639]: pgmap v646: 129 pgs: 129 active+clean; 606 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2023-10-18T12:36:49.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:48 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:36:49.880 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1232644 11.45 MB/sec execute 438 sec latency 40.463 ms 2023-10-18T12:36:50.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:50 smithi070 ceph-mon[98639]: pgmap v647: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2023-10-18T12:36:50.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:50 smithi152 ceph-mon[134767]: pgmap v647: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2023-10-18T12:36:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:50 smithi186 ceph-mon[135253]: pgmap v647: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2023-10-18T12:36:50.880 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1233827 11.43 MB/sec execute 439 sec latency 65.224 ms 2023-10-18T12:36:51.880 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1236051 11.43 MB/sec execute 440 sec latency 90.080 ms 2023-10-18T12:36:52.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1237742 11.42 MB/sec execute 441 sec latency 13.373 ms 2023-10-18T12:36:53.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:52 smithi152 ceph-mon[134767]: pgmap v648: 129 pgs: 129 active+clean; 574 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2023-10-18T12:36:53.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:52 smithi186 ceph-mon[135253]: pgmap v648: 129 pgs: 129 active+clean; 574 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2023-10-18T12:36:53.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:52 smithi070 ceph-mon[98639]: pgmap v648: 129 pgs: 129 active+clean; 574 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2023-10-18T12:36:53.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1240076 11.43 MB/sec execute 442 sec latency 15.620 ms 2023-10-18T12:36:54.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1242053 11.42 MB/sec execute 443 sec latency 41.128 ms 2023-10-18T12:36:55.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:54 smithi152 ceph-mon[134767]: pgmap v649: 129 pgs: 129 active+clean; 566 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 225 op/s 2023-10-18T12:36:55.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:54 smithi186 ceph-mon[135253]: pgmap v649: 129 pgs: 129 active+clean; 566 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 225 op/s 2023-10-18T12:36:55.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:54 smithi070 ceph-mon[98639]: pgmap v649: 129 pgs: 129 active+clean; 566 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 225 op/s 2023-10-18T12:36:55.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1243758 11.42 MB/sec execute 444 sec latency 96.925 ms 2023-10-18T12:36:56.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1245903 11.42 MB/sec execute 445 sec latency 27.922 ms 2023-10-18T12:36:57.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:56 smithi152 ceph-mon[134767]: pgmap v650: 129 pgs: 129 active+clean; 566 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 180 op/s 2023-10-18T12:36:57.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:56 smithi186 ceph-mon[135253]: pgmap v650: 129 pgs: 129 active+clean; 566 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 180 op/s 2023-10-18T12:36:57.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:56 smithi070 ceph-mon[98639]: pgmap v650: 129 pgs: 129 active+clean; 566 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 180 op/s 2023-10-18T12:36:57.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1247831 11.42 MB/sec execute 446 sec latency 17.707 ms 2023-10-18T12:36:58.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1250419 11.43 MB/sec execute 447 sec latency 14.168 ms 2023-10-18T12:36:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:36:58 smithi152 ceph-mon[134767]: pgmap v651: 129 pgs: 129 active+clean; 575 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2023-10-18T12:36:59.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:36:58 smithi186 ceph-mon[135253]: pgmap v651: 129 pgs: 129 active+clean; 575 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2023-10-18T12:36:59.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:36:58 smithi070 ceph-mon[98639]: pgmap v651: 129 pgs: 129 active+clean; 575 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2023-10-18T12:36:59.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1252500 11.42 MB/sec execute 448 sec latency 13.419 ms 2023-10-18T12:37:00.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:00 smithi070 ceph-mon[98639]: pgmap v652: 129 pgs: 129 active+clean; 578 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 220 op/s 2023-10-18T12:37:00.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:00 smithi152 ceph-mon[134767]: pgmap v652: 129 pgs: 129 active+clean; 578 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 220 op/s 2023-10-18T12:37:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:00 smithi186 ceph-mon[135253]: pgmap v652: 129 pgs: 129 active+clean; 578 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 220 op/s 2023-10-18T12:37:00.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1253841 11.42 MB/sec execute 449 sec latency 98.562 ms 2023-10-18T12:37:01.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1254117 11.40 MB/sec execute 450 sec latency 214.830 ms 2023-10-18T12:37:02.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1256083 11.39 MB/sec execute 451 sec latency 63.805 ms 2023-10-18T12:37:03.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:02 smithi152 ceph-mon[134767]: pgmap v653: 129 pgs: 129 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 232 op/s 2023-10-18T12:37:03.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:02 smithi186 ceph-mon[135253]: pgmap v653: 129 pgs: 129 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 232 op/s 2023-10-18T12:37:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:02 smithi070 ceph-mon[98639]: pgmap v653: 129 pgs: 129 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 232 op/s 2023-10-18T12:37:03.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1258014 11.39 MB/sec execute 452 sec latency 13.764 ms 2023-10-18T12:37:04.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1260189 11.39 MB/sec execute 453 sec latency 27.709 ms 2023-10-18T12:37:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:04 smithi152 ceph-mon[134767]: pgmap v654: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 263 op/s 2023-10-18T12:37:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:04 smithi152 ceph-mon[134767]: osdmap e102: 12 total, 12 up, 12 in 2023-10-18T12:37:05.224 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:04 smithi186 ceph-mon[135253]: pgmap v654: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 263 op/s 2023-10-18T12:37:05.224 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:04 smithi186 ceph-mon[135253]: osdmap e102: 12 total, 12 up, 12 in 2023-10-18T12:37:05.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:04 smithi070 ceph-mon[98639]: pgmap v654: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 263 op/s 2023-10-18T12:37:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:04 smithi070 ceph-mon[98639]: osdmap e102: 12 total, 12 up, 12 in 2023-10-18T12:37:05.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1261629 11.38 MB/sec execute 454 sec latency 63.865 ms 2023-10-18T12:37:06.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1263929 11.38 MB/sec execute 455 sec latency 68.201 ms 2023-10-18T12:37:07.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:06 smithi152 ceph-mon[134767]: pgmap v656: 129 pgs: 129 active+clean; 589 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2023-10-18T12:37:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:06 smithi186 ceph-mon[135253]: pgmap v656: 129 pgs: 129 active+clean; 589 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2023-10-18T12:37:07.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:06 smithi070 ceph-mon[98639]: pgmap v656: 129 pgs: 129 active+clean; 589 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2023-10-18T12:37:07.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1265791 11.38 MB/sec execute 456 sec latency 12.748 ms 2023-10-18T12:37:08.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1268212 11.39 MB/sec execute 457 sec latency 41.130 ms 2023-10-18T12:37:09.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:08 smithi152 ceph-mon[134767]: pgmap v657: 129 pgs: 129 active+clean; 591 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2023-10-18T12:37:09.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:08 smithi186 ceph-mon[135253]: pgmap v657: 129 pgs: 129 active+clean; 591 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2023-10-18T12:37:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:08 smithi070 ceph-mon[98639]: pgmap v657: 129 pgs: 129 active+clean; 591 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2023-10-18T12:37:09.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1270231 11.38 MB/sec execute 458 sec latency 13.246 ms 2023-10-18T12:37:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:10 smithi152 ceph-mon[134767]: pgmap v658: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 272 op/s 2023-10-18T12:37:10.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:10 smithi186 ceph-mon[135253]: pgmap v658: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 272 op/s 2023-10-18T12:37:10.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:10 smithi070 ceph-mon[98639]: pgmap v658: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 272 op/s 2023-10-18T12:37:10.881 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1271891 11.38 MB/sec execute 459 sec latency 78.420 ms 2023-10-18T12:37:11.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1273723 11.37 MB/sec execute 460 sec latency 32.089 ms 2023-10-18T12:37:12.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.37 MB/sec execute 461 sec latency 603.173 ms 2023-10-18T12:37:13.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:12 smithi152 ceph-mon[134767]: pgmap v659: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2023-10-18T12:37:13.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:12 smithi186 ceph-mon[135253]: pgmap v659: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2023-10-18T12:37:13.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:12 smithi070 ceph-mon[98639]: pgmap v659: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2023-10-18T12:37:13.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.34 MB/sec execute 462 sec latency 1603.270 ms 2023-10-18T12:37:14.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.32 MB/sec execute 463 sec latency 2603.432 ms 2023-10-18T12:37:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:14 smithi152 ceph-mon[134767]: pgmap v660: 129 pgs: 129 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 188 op/s 2023-10-18T12:37:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:14 smithi152 ceph-mon[134767]: osdmap e103: 12 total, 12 up, 12 in 2023-10-18T12:37:15.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:14 smithi186 ceph-mon[135253]: pgmap v660: 129 pgs: 129 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 188 op/s 2023-10-18T12:37:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:14 smithi186 ceph-mon[135253]: osdmap e103: 12 total, 12 up, 12 in 2023-10-18T12:37:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:14 smithi070 ceph-mon[98639]: pgmap v660: 129 pgs: 129 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 188 op/s 2023-10-18T12:37:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:14 smithi070 ceph-mon[98639]: osdmap e103: 12 total, 12 up, 12 in 2023-10-18T12:37:15.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.30 MB/sec execute 464 sec latency 3603.519 ms 2023-10-18T12:37:16.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.27 MB/sec execute 465 sec latency 4603.587 ms 2023-10-18T12:37:17.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:16 smithi152 ceph-mon[134767]: pgmap v662: 129 pgs: 129 active+clean; 610 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 208 op/s 2023-10-18T12:37:17.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:16 smithi186 ceph-mon[135253]: pgmap v662: 129 pgs: 129 active+clean; 610 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 208 op/s 2023-10-18T12:37:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:16 smithi070 ceph-mon[98639]: pgmap v662: 129 pgs: 129 active+clean; 610 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 208 op/s 2023-10-18T12:37:17.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.25 MB/sec execute 466 sec latency 5603.737 ms 2023-10-18T12:37:18.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.291+0000 7f9d63bff700 1 -- 172.21.15.70:0/703090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c113940 msgr2=0x7f9d5c115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.292+0000 7f9d63bff700 1 --2- 172.21.15.70:0/703090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c113940 0x7f9d5c115d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009fc0 tx=0x7f9d5400b330 comp rx=0 tx=0).stop 2023-10-18T12:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 -- 172.21.15.70:0/703090082 shutdown_connections 2023-10-18T12:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 --2- 172.21.15.70:0/703090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c113940 0x7f9d5c115d30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 --2- 172.21.15.70:0/703090082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 --2- 172.21.15.70:0/703090082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c108110 0x7f9d5c113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 -- 172.21.15.70:0/703090082 >> 172.21.15.70:0/703090082 conn(0x7f9d5c06b840 msgr2=0x7f9d5c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 -- 172.21.15.70:0/703090082 shutdown_connections 2023-10-18T12:37:18.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.293+0000 7f9d63bff700 1 -- 172.21.15.70:0/703090082 wait complete. 2023-10-18T12:37:18.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.294+0000 7f9d63bff700 1 Processor -- start 2023-10-18T12:37:18.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.295+0000 7f9d63bff700 1 -- start start 2023-10-18T12:37:18.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.296+0000 7f9d63bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c11d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.296+0000 7f9d63bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c108110 0x7f9d5c122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.296+0000 7f9d63bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c113940 0x7f9d5c11d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.296+0000 7f9d63bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c118460 con 0x7f9d5c108110 2023-10-18T12:37:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.296+0000 7f9d63bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c1185d0 con 0x7f9d5c1077f0 2023-10-18T12:37:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.296+0000 7f9d63bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c118790 con 0x7f9d5c113940 2023-10-18T12:37:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6199b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c11d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6199b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c11d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:36022/0 (socket says 172.21.15.70:36022) 2023-10-18T12:37:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6199b700 1 -- 172.21.15.70:0/2499060494 learned_addr learned my addr 172.21.15.70:0/2499060494 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6219c700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c113940 0x7f9d5c11d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6199b700 1 -- 172.21.15.70:0/2499060494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c113940 msgr2=0x7f9d5c11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c113940 0x7f9d5c11d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.297+0000 7f9d6199b700 1 -- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c108110 msgr2=0x7f9d5c122e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.298+0000 7f9d6119a700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c108110 0x7f9d5c122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.298+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c108110 0x7f9d5c122e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.298+0000 7f9d6199b700 1 -- 172.21.15.70:0/2499060494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580095f0 con 0x7f9d5c1077f0 2023-10-18T12:37:18.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.298+0000 7f9d6219c700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c113940 0x7f9d5c11d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:18.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.298+0000 7f9d6119a700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c108110 0x7f9d5c122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:37:18.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.299+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c11d110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800ec10 tx=0x7f9d5800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.299+0000 7f9d52ffd700 1 -- 172.21.15.70:0/2499060494 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800cd40 con 0x7f9d5c1077f0 2023-10-18T12:37:18.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.299+0000 7f9d52ffd700 1 -- 172.21.15.70:0/2499060494 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d580044e0 con 0x7f9d5c1077f0 2023-10-18T12:37:18.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.299+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54009c40 con 0x7f9d5c1077f0 2023-10-18T12:37:18.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.299+0000 7f9d52ffd700 1 -- 172.21.15.70:0/2499060494 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58010690 con 0x7f9d5c1077f0 2023-10-18T12:37:18.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.300+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c118df0 con 0x7f9d5c1077f0 2023-10-18T12:37:18.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.304+0000 7f9d52ffd700 1 -- 172.21.15.70:0/2499060494 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9d58003710 con 0x7f9d5c1077f0 2023-10-18T12:37:18.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.306+0000 7f9d774deb80 1 Processor -- start 2023-10-18T12:37:18.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.306+0000 7f9d52ffd700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d480a1780 0x7f9d480a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.306+0000 7f9d774deb80 1 -- start start 2023-10-18T12:37:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.306+0000 7f9d52ffd700 1 -- 172.21.15.70:0/2499060494 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9d58014070 con 0x7f9d5c1077f0 2023-10-18T12:37:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d6219c700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d480a1780 0x7f9d480a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 0x561fb1f10fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f11500 0x561fb1f0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f0ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d774deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1e7e6e0 con 0x561fb1f11500 2023-10-18T12:37:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d774deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1e64c60 con 0x561fb1f0b5d0 2023-10-18T12:37:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d774deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1e62580 con 0x561fb1f10be0 2023-10-18T12:37:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d6199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 0x561fb1f10fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d6199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 0x561fb1f10fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49004/0 (socket says 172.21.15.70:49004) 2023-10-18T12:37:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d6199b700 1 -- 172.21.15.70:0/2389469750 learned_addr learned my addr 172.21.15.70:0/2389469750 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d6219c700 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f0ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.307+0000 7f9d6119a700 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f11500 0x561fb1f0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6199b700 1 -- 172.21.15.70:0/2389469750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 msgr2=0x561fb1f0ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f0ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6199b700 1 -- 172.21.15.70:0/2389469750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f11500 msgr2=0x561fb1f0b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f11500 0x561fb1f0b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6199b700 1 -- 172.21.15.70:0/2389469750 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb1e1b3f0 con 0x561fb1f10be0 2023-10-18T12:37:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6219c700 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f0ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6219c700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d480a1780 0x7f9d480a3c40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009040 tx=0x7f9d54003c60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 0x561fb1f10fc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9d580a23e0 tx=0x7f9d5809a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c1535377899edb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d47fff700 1 -- 172.21.15.70:0/2389469750 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5809a580 con 0x561fb1f10be0 2023-10-18T12:37:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d47fff700 1 -- 172.21.15.70:0/2389469750 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d5809e6d0 con 0x561fb1f10be0 2023-10-18T12:37:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.308+0000 7f9d47fff700 1 -- 172.21.15.70:0/2389469750 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5809aa40 con 0x561fb1f10be0 2023-10-18T12:37:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 -- 172.21.15.70:0/2389469750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 msgr2=0x561fb1f10fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 0x561fb1f10fc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9d580a23e0 tx=0x7f9d5809a140 comp rx=0 tx=0).stop 2023-10-18T12:37:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 -- 172.21.15.70:0/2389469750 shutdown_connections 2023-10-18T12:37:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f0ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f10be0 0x561fb1f10fc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2389469750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f11500 0x561fb1f0b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 -- 172.21.15.70:0/2389469750 >> 172.21.15.70:0/2389469750 conn(0x561fb1e7f720 msgr2=0x561fb1f05b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 -- 172.21.15.70:0/2389469750 shutdown_connections 2023-10-18T12:37:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.309+0000 7f9d774deb80 1 -- 172.21.15.70:0/2389469750 wait complete. 2023-10-18T12:37:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 -- start start 2023-10-18T12:37:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 Processor -- start 2023-10-18T12:37:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f17470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f14ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1e5de70 con 0x561fb1f10be0 2023-10-18T12:37:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1dfa710 con 0x561fb1f11500 2023-10-18T12:37:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1dfa3b0 con 0x561fb1f0b5d0 2023-10-18T12:37:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.310+0000 7f9d774deb80 10 client.0 _ll_register_callbacks cb 0x561fb1d44a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f17470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6219c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f14ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f14760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58620/0 (socket says 172.21.15.70:58620) 2023-10-18T12:37:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 -- 172.21.15.70:0/2125913610 learned_addr learned my addr 172.21.15.70:0/2125913610 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 -- 172.21.15.70:0/2125913610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 msgr2=0x561fb1f17470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f17470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 -- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 msgr2=0x561fb1f14ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f14ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 -- 172.21.15.70:0/2125913610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c118df0 con 0x561fb1f10be0 2023-10-18T12:37:18.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d6119a700 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f14760 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00f4f0 tx=0x7f9d4c00f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bca9f8ab84a5c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.311+0000 7f9d457fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:18.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d457fa700 1 -- 172.21.15.70:0/2125913610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c013640 con 0x561fb1f10be0 2023-10-18T12:37:18.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d457fa700 1 -- 172.21.15.70:0/2125913610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9d4c0137a0 con 0x561fb1f10be0 2023-10-18T12:37:18.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fb1e62ef0 con 0x561fb1f10be0 2023-10-18T12:37:18.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d457fa700 1 -- 172.21.15.70:0/2125913610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4c013960 con 0x561fb1f10be0 2023-10-18T12:37:18.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 10 client.17079 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:18.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x561fb1c88de0 con 0x561fb1f10be0 2023-10-18T12:37:18.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d457fa700 1 -- 172.21.15.70:0/2125913610 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561fb1e62ef0 con 0x561fb1f10be0 2023-10-18T12:37:18.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d457fa700 1 -- 172.21.15.70:0/2125913610 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d4c019020 con 0x561fb1f10be0 2023-10-18T12:37:18.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 10 client.17079 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:18.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:18.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 20 client.17079 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:18.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f24fa0 0x561fb1f273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 4 client.17079 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:18.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.312+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fb1eda8b0 con 0x561fb1f24fa0 2023-10-18T12:37:18.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.313+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f24fa0 0x561fb1f273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.313+0000 7f9d457fa700 10 client.17079 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:18.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.313+0000 7f9d6199b700 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f24fa0 0x561fb1f273c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e85adc9bab6e23 server_cookie=3f040a22c85b9b0e in_seq=0 out_seq=0 2023-10-18T12:37:18.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d457fa700 1 -- 172.21.15.70:0/2125913610 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561fb1eda8b0 con 0x561fb1f24fa0 2023-10-18T12:37:18.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d457fa700 10 client.17079 handle_command_reply: tid=0 2023-10-18T12:37:18.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 client.17079 shutdown 2023-10-18T12:37:18.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 msgr2=0x561fb1f14760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f14760 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00f4f0 tx=0x7f9d4c00f800 comp rx=0 tx=0).stop 2023-10-18T12:37:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 shutdown_connections 2023-10-18T12:37:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f24fa0 0x561fb1f273c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f14ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f17470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 --2- 172.21.15.70:0/2125913610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f14760 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 >> 172.21.15.70:0/2125913610 conn(0x561fb1e7f720 msgr2=0x561fb1f0a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.314+0000 7f9d457fa700 0 client.17079 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.315+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 shutdown_connections 2023-10-18T12:37:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.315+0000 7f9d774deb80 1 -- 172.21.15.70:0/2125913610 wait complete. 2023-10-18T12:37:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.315+0000 7f9d774deb80 20 client.17079 trim_cache size 0 max 16384 2023-10-18T12:37:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 Processor -- start 2023-10-18T12:37:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 -- start start 2023-10-18T12:37:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f0e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f11500 0x561fb1f0ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1cde920 con 0x561fb1f10be0 2023-10-18T12:37:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1f075e0 con 0x561fb1f0b5d0 2023-10-18T12:37:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d774deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1f02230 con 0x561fb1f11500 2023-10-18T12:37:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d6119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f0e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d6119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f0e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58626/0 (socket says 172.21.15.70:58626) 2023-10-18T12:37:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.332+0000 7f9d6119a700 1 -- 172.21.15.70:0/4147215427 learned_addr learned my addr 172.21.15.70:0/4147215427 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6219c700 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f11500 0x561fb1f0ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6199b700 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6119a700 1 -- 172.21.15.70:0/4147215427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f11500 msgr2=0x561fb1f0ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6119a700 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f11500 0x561fb1f0ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6119a700 1 -- 172.21.15.70:0/4147215427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 msgr2=0x561fb1f1beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6119a700 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6119a700 1 -- 172.21.15.70:0/4147215427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb1c71c60 con 0x561fb1f10be0 2023-10-18T12:37:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6199b700 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d6119a700 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f0e720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c008010 tx=0x7f9d4c006a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bbf92662d6289b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d50bff700 1 -- 172.21.15.70:0/4147215427 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c012540 con 0x561fb1f10be0 2023-10-18T12:37:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d50bff700 1 -- 172.21.15.70:0/4147215427 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4c00c040 con 0x561fb1f10be0 2023-10-18T12:37:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.333+0000 7f9d50bff700 1 -- 172.21.15.70:0/4147215427 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c012eb0 con 0x561fb1f10be0 2023-10-18T12:37:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 -- 172.21.15.70:0/4147215427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 msgr2=0x561fb1f0e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f0e720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c008010 tx=0x7f9d4c006a10 comp rx=0 tx=0).stop 2023-10-18T12:37:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 -- 172.21.15.70:0/4147215427 shutdown_connections 2023-10-18T12:37:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f11500 0x561fb1f0ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 --2- 172.21.15.70:0/4147215427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f0e720 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 -- 172.21.15.70:0/4147215427 >> 172.21.15.70:0/4147215427 conn(0x561fb1e7f720 msgr2=0x561fb1f13030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 -- 172.21.15.70:0/4147215427 shutdown_connections 2023-10-18T12:37:18.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 -- 172.21.15.70:0/4147215427 wait complete. 2023-10-18T12:37:18.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.334+0000 7f9d774deb80 1 -- start start 2023-10-18T12:37:18.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 Processor -- start 2023-10-18T12:37:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1f78da0 con 0x561fb1f10be0 2023-10-18T12:37:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1f78ee0 con 0x561fb1f11500 2023-10-18T12:37:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1f79020 con 0x561fb1f0b5d0 2023-10-18T12:37:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d774deb80 10 client.0 _ll_register_callbacks cb 0x561fb1d44a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.335+0000 7f9d6119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f1ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58632/0 (socket says 172.21.15.70:58632) 2023-10-18T12:37:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 -- 172.21.15.70:0/3953221361 learned_addr learned my addr 172.21.15.70:0/3953221361 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6199b700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6219c700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 -- 172.21.15.70:0/3953221361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 msgr2=0x561fb1f1a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 -- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 msgr2=0x561fb1f1b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f1b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 -- 172.21.15.70:0/3953221361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fb1c88de0 con 0x561fb1f10be0 2023-10-18T12:37:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d6119a700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f1ac70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c006ae0 tx=0x7f9d4c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c4266df6634fa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d457fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d457fa700 1 -- 172.21.15.70:0/3953221361 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c012d40 con 0x561fb1f10be0 2023-10-18T12:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d457fa700 1 -- 172.21.15.70:0/3953221361 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9d4c013440 con 0x561fb1f10be0 2023-10-18T12:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.336+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d5809e6d0 con 0x561fb1f10be0 2023-10-18T12:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d457fa700 1 -- 172.21.15.70:0/3953221361 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4c012ea0 con 0x561fb1f10be0 2023-10-18T12:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 10 client.17085 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x561fb1caf250 con 0x561fb1f10be0 2023-10-18T12:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d457fa700 1 -- 172.21.15.70:0/3953221361 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d5809e6d0 con 0x561fb1f10be0 2023-10-18T12:37:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d457fa700 1 -- 172.21.15.70:0/3953221361 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d4c00f520 con 0x561fb1f10be0 2023-10-18T12:37:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 10 client.17085 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 20 client.17085 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:18.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f7a3a0 0x561fb1f7c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 4 client.17085 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:37:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561fb1e56e50 con 0x561fb1f7a3a0 2023-10-18T12:37:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.337+0000 7f9d6199b700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f7a3a0 0x561fb1f7c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d6199b700 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f7a3a0 0x561fb1f7c790 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed03c36e701bceb2 server_cookie=f6b3734da3e3cc17 in_seq=0 out_seq=0 2023-10-18T12:37:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d457fa700 10 client.17085 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d457fa700 1 -- 172.21.15.70:0/3953221361 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561fb1e56e50 con 0x561fb1f7a3a0 2023-10-18T12:37:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d457fa700 10 client.17085 handle_command_reply: tid=0 2023-10-18T12:37:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 client.17085 shutdown 2023-10-18T12:37:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 msgr2=0x561fb1f1ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f1ac70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c006ae0 tx=0x7f9d4c00fbd0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 shutdown_connections 2023-10-18T12:37:18.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561fb1f7a3a0 0x561fb1f7c790 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fb1f11500 0x561fb1f1b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561fb1f0b5d0 0x561fb1f1a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d457fa700 0 client.17085 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:18.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 --2- 172.21.15.70:0/3953221361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561fb1f10be0 0x561fb1f1ac70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 >> 172.21.15.70:0/3953221361 conn(0x561fb1e7f720 msgr2=0x561fb1f0f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 shutdown_connections 2023-10-18T12:37:18.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.338+0000 7f9d774deb80 1 -- 172.21.15.70:0/3953221361 wait complete. 2023-10-18T12:37:18.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d774deb80 20 client.17085 trim_cache size 0 max 16384 2023-10-18T12:37:18.358 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:37:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:37:18.358 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d480a1780 msgr2=0x7f9d480a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d480a1780 0x7f9d480a3c40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009040 tx=0x7f9d54003c60 comp rx=0 tx=0).stop 2023-10-18T12:37:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 msgr2=0x7f9d5c11d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c11d110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800ec10 tx=0x7f9d5800ef20 comp rx=0 tx=0).stop 2023-10-18T12:37:18.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 shutdown_connections 2023-10-18T12:37:18.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d5c113940 0x7f9d5c11d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9d480a1780 0x7f9d480a3c40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c1077f0 0x7f9d5c11d110 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 --2- 172.21.15.70:0/2499060494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d5c108110 0x7f9d5c122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 >> 172.21.15.70:0/2499060494 conn(0x7f9d5c06b840 msgr2=0x7f9d5c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 shutdown_connections 2023-10-18T12:37:18.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.339+0000 7f9d63bff700 1 -- 172.21.15.70:0/2499060494 wait complete. 2023-10-18T12:37:18.362 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:37:18.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:37:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.558+0000 7fa0acdff700 1 -- 172.21.15.70:0/61399106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a81066b0 msgr2=0x7fa0a8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.558+0000 7fa0acdff700 1 --2- 172.21.15.70:0/61399106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a81066b0 0x7fa0a8106a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa0980098f0 tx=0x7fa098009c00 comp rx=0 tx=0).stop 2023-10-18T12:37:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 -- 172.21.15.70:0/61399106 shutdown_connections 2023-10-18T12:37:18.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 --2- 172.21.15.70:0/61399106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a8101060 0x7fa0a810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 --2- 172.21.15.70:0/61399106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a81066b0 0x7fa0a8106a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 --2- 172.21.15.70:0/61399106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81006a0 0x7fa0a8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 -- 172.21.15.70:0/61399106 >> 172.21.15.70:0/61399106 conn(0x7fa0a80fc170 msgr2=0x7fa0a80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 -- 172.21.15.70:0/61399106 shutdown_connections 2023-10-18T12:37:18.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.559+0000 7fa0acdff700 1 -- 172.21.15.70:0/61399106 wait complete. 2023-10-18T12:37:18.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 Processor -- start 2023-10-18T12:37:18.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 -- start start 2023-10-18T12:37:18.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a81006a0 0x7fa0a8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a8101060 0x7fa0a8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 0x7fa0a8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8074ac0 con 0x7fa0a81066b0 2023-10-18T12:37:18.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81a94c0 con 0x7fa0a81006a0 2023-10-18T12:37:18.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0acdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81a9680 con 0x7fa0a8101060 2023-10-18T12:37:18.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0a6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 0x7fa0a8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0a6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 0x7fa0a8073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58648/0 (socket says 172.21.15.70:58648) 2023-10-18T12:37:18.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0a6d9d700 1 -- 172.21.15.70:0/1059977295 learned_addr learned my addr 172.21.15.70:0/1059977295 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:18.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0a659c700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a81006a0 0x7fa0a8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.560+0000 7fa0a5d9b700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a8101060 0x7fa0a8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0a6d9d700 1 -- 172.21.15.70:0/1059977295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a8101060 msgr2=0x7fa0a8073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0a6d9d700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a8101060 0x7fa0a8073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0a6d9d700 1 -- 172.21.15.70:0/1059977295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a81006a0 msgr2=0x7fa0a8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0a6d9d700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a81006a0 0x7fa0a8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0a6d9d700 1 -- 172.21.15.70:0/1059977295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0095f0 con 0x7fa0a81066b0 2023-10-18T12:37:18.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0a6d9d700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 0x7fa0a8073570 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000ef30 tx=0x7fa0a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0018070 con 0x7fa0a81066b0 2023-10-18T12:37:18.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a000f040 con 0x7fa0a81066b0 2023-10-18T12:37:18.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa098009520 con 0x7fa0a81066b0 2023-10-18T12:37:18.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000cd00 con 0x7fa0a81066b0 2023-10-18T12:37:18.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.561+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a81a9c80 con 0x7fa0a81066b0 2023-10-18T12:37:18.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.562+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a8108c30 con 0x7fa0a81066b0 2023-10-18T12:37:18.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.564+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa0a0003a00 con 0x7fa0a81066b0 2023-10-18T12:37:18.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.566+0000 7fa0977fe700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0900a1670 0x7fa0900a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:18.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.566+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa0a00cd910 con 0x7fa0a81066b0 2023-10-18T12:37:18.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.566+0000 7fa0a659c700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0900a1670 0x7fa0900a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:18.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.566+0000 7fa0a659c700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0900a1670 0x7fa0900a3b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa0980098c0 tx=0x7fa098019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:18.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.568+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa0a0096900 con 0x7fa0a81066b0 2023-10-18T12:37:18.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.816+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0a8101de0 con 0x7fa0a81066b0 2023-10-18T12:37:18.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.816+0000 7fa0977fe700 1 -- 172.21.15.70:0/1059977295 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fa0a0099f20 con 0x7fa0a81066b0 2023-10-18T12:37:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:37:18.820 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:37:18.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0900a1670 msgr2=0x7fa0900a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0900a1670 0x7fa0900a3b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa0980098c0 tx=0x7fa098019040 comp rx=0 tx=0).stop 2023-10-18T12:37:18.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 msgr2=0x7fa0a8073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:18.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 0x7fa0a8073570 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000ef30 tx=0x7fa0a000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 shutdown_connections 2023-10-18T12:37:18.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a81066b0 0x7fa0a8073570 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa0900a1670 0x7fa0900a3b30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a81006a0 0x7fa0a8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 --2- 172.21.15.70:0/1059977295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0a8101060 0x7fa0a8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:18.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 >> 172.21.15.70:0/1059977295 conn(0x7fa0a80fc170 msgr2=0x7fa0a80fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:18.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 shutdown_connections 2023-10-18T12:37:18.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:18.819+0000 7fa0acdff700 1 -- 172.21.15.70:0/1059977295 wait complete. 2023-10-18T12:37:18.824 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:37:18.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:37:18.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.22 MB/sec execute 467 sec latency 6603.835 ms 2023-10-18T12:37:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 -- 172.21.15.70:0/850774084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc100730 msgr2=0x7f17dc100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 --2- 172.21.15.70:0/850774084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc100730 0x7f17dc100bb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009990 tx=0x7f17d0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 -- 172.21.15.70:0/850774084 shutdown_connections 2023-10-18T12:37:19.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 --2- 172.21.15.70:0/850774084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17dc1010f0 0x7f17dc10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 --2- 172.21.15.70:0/850774084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17dc106740 0x7f17dc106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 --2- 172.21.15.70:0/850774084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc100730 0x7f17dc100bb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 -- 172.21.15.70:0/850774084 >> 172.21.15.70:0/850774084 conn(0x7f17dc0fc200 msgr2=0x7f17dc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:19.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 -- 172.21.15.70:0/850774084 shutdown_connections 2023-10-18T12:37:19.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.038+0000 7f17e1bff700 1 -- 172.21.15.70:0/850774084 wait complete. 2023-10-18T12:37:19.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.039+0000 7f17e1bff700 1 Processor -- start 2023-10-18T12:37:19.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.039+0000 7f17e1bff700 1 -- start start 2023-10-18T12:37:19.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.039+0000 7f17e1bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17dc100730 0x7f17dc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.039+0000 7f17e1bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17dc1010f0 0x7f17dc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.039+0000 7f17e1bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 0x7f17dc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17e1bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc199340 con 0x7f17dc106740 2023-10-18T12:37:19.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17e1bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc1994b0 con 0x7f17dc100730 2023-10-18T12:37:19.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17e1bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc199670 con 0x7f17dc1010f0 2023-10-18T12:37:19.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 0x7f17dc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17dc100730 0x7f17dc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17daffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17dc1010f0 0x7f17dc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 0x7f17dc198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58660/0 (socket says 172.21.15.70:58660) 2023-10-18T12:37:19.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 -- 172.21.15.70:0/2291099683 learned_addr learned my addr 172.21.15.70:0/2291099683 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:19.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 -- 172.21.15.70:0/2291099683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17dc1010f0 msgr2=0x7f17dc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17dc1010f0 0x7f17dc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 -- 172.21.15.70:0/2291099683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17dc100730 msgr2=0x7f17dc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17dc100730 0x7f17dc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.040+0000 7f17dbfff700 1 -- 172.21.15.70:0/2291099683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00095f0 con 0x7f17dc106740 2023-10-18T12:37:19.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.041+0000 7f17dbfff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 0x7f17dc198b70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f17d400ef30 tx=0x7f17d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:19.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.041+0000 7f17d8ff9700 1 -- 172.21.15.70:0/2291099683 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4018070 con 0x7f17dc106740 2023-10-18T12:37:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.041+0000 7f17d8ff9700 1 -- 172.21.15.70:0/2291099683 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17d400f040 con 0x7f17dc106740 2023-10-18T12:37:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.041+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc1998f0 con 0x7f17dc106740 2023-10-18T12:37:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.041+0000 7f17d8ff9700 1 -- 172.21.15.70:0/2291099683 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400cd00 con 0x7f17dc106740 2023-10-18T12:37:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.041+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc1a5a50 con 0x7f17dc106740 2023-10-18T12:37:19.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.044+0000 7f17d8ff9700 1 -- 172.21.15.70:0/2291099683 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f17d4003a00 con 0x7f17dc106740 2023-10-18T12:37:19.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.045+0000 7f17d8ff9700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f17c40a15f0 0x7f17c40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.045+0000 7f17f54e5b80 1 Processor -- start 2023-10-18T12:37:19.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.045+0000 7f17d8ff9700 1 -- 172.21.15.70:0/2291099683 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f17d40ccfe0 con 0x7f17dc106740 2023-10-18T12:37:19.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 -- start start 2023-10-18T12:37:19.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17db7fe700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f17c40a15f0 0x7f17c40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc531fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc52c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc52ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc49f6e0 con 0x5596bc531be0 2023-10-18T12:37:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc485c60 con 0x5596bc532500 2023-10-18T12:37:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc483580 con 0x5596bc52c5d0 2023-10-18T12:37:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc531fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc531fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58664/0 (socket says 172.21.15.70:58664) 2023-10-18T12:37:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17db7fe700 1 -- 172.21.15.70:0/28098339 learned_addr learned my addr 172.21.15.70:0/28098339 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17dbfff700 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc52ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.046+0000 7f17daffd700 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc52c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17dbfff700 1 -- 172.21.15.70:0/28098339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 msgr2=0x5596bc52c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17dbfff700 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc52c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17dbfff700 1 -- 172.21.15.70:0/28098339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 msgr2=0x5596bc531fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17dbfff700 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc531fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17dbfff700 1 -- 172.21.15.70:0/28098339 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596bc43c3f0 con 0x5596bc52c5d0 2023-10-18T12:37:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17db7fe700 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc531fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17db7fe700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f17c40a15f0 0x7f17c40a3ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0139f0 tx=0x7f17cc009b90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17dbfff700 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc52ca50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17d40ca3d0 tx=0x7f17d4099830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927ccddab56cb276 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17c1ffb700 1 -- 172.21.15.70:0/28098339 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400ce80 con 0x5596bc52c5d0 2023-10-18T12:37:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17c1ffb700 1 -- 172.21.15.70:0/28098339 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17d4004ad0 con 0x5596bc52c5d0 2023-10-18T12:37:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.047+0000 7f17c1ffb700 1 -- 172.21.15.70:0/28098339 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d40a1ff0 con 0x5596bc52c5d0 2023-10-18T12:37:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 -- 172.21.15.70:0/28098339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 msgr2=0x5596bc52ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc52ca50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17d40ca3d0 tx=0x7f17d4099830 comp rx=0 tx=0).stop 2023-10-18T12:37:19.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 -- 172.21.15.70:0/28098339 shutdown_connections 2023-10-18T12:37:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc52ca50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc531fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/28098339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc52c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 -- 172.21.15.70:0/28098339 >> 172.21.15.70:0/28098339 conn(0x5596bc4a0720 msgr2=0x5596bc526b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 -- 172.21.15.70:0/28098339 shutdown_connections 2023-10-18T12:37:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.048+0000 7f17f54e5b80 1 -- 172.21.15.70:0/28098339 wait complete. 2023-10-18T12:37:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 -- start start 2023-10-18T12:37:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 Processor -- start 2023-10-18T12:37:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc538470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc531be0 0x5596bc535760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 0x5596bc535d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc47ee70 con 0x5596bc532500 2023-10-18T12:37:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc41b710 con 0x5596bc52c5d0 2023-10-18T12:37:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.049+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc41b3b0 con 0x5596bc531be0 2023-10-18T12:37:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17f54e5b80 10 client.0 _ll_register_callbacks cb 0x5596bc365a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 0x5596bc535d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17daffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc531be0 0x5596bc535760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 0x5596bc535d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58676/0 (socket says 172.21.15.70:58676) 2023-10-18T12:37:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 -- 172.21.15.70:0/3587094579 learned_addr learned my addr 172.21.15.70:0/3587094579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17daffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc531be0 0x5596bc535760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49108/0 (socket says 172.21.15.70:49108) 2023-10-18T12:37:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17db7fe700 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc538470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 -- 172.21.15.70:0/3587094579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc531be0 msgr2=0x5596bc535760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc531be0 0x5596bc535760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 -- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 msgr2=0x5596bc538470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc538470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 -- 172.21.15.70:0/3587094579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc01b040 con 0x5596bc532500 2023-10-18T12:37:19.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17dbfff700 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 0x5596bc535d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f17d40ca480 tx=0x7f17d409a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0323599e00bc6f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:19.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.050+0000 7f17baffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:19.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17baffd700 1 -- 172.21.15.70:0/3587094579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d409a3f0 con 0x5596bc532500 2023-10-18T12:37:19.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17baffd700 1 -- 172.21.15.70:0/3587094579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f17d40a1ff0 con 0x5596bc532500 2023-10-18T12:37:19.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596bc483ef0 con 0x5596bc532500 2023-10-18T12:37:19.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17baffd700 1 -- 172.21.15.70:0/3587094579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17d4099cf0 con 0x5596bc532500 2023-10-18T12:37:19.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 10 client.17103 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:19.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f17d40b9040 con 0x5596bc532500 2023-10-18T12:37:19.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17baffd700 1 -- 172.21.15.70:0/3587094579 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596bc483ef0 con 0x5596bc532500 2023-10-18T12:37:19.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17baffd700 1 -- 172.21.15.70:0/3587094579 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f17d409eb80 con 0x5596bc532500 2023-10-18T12:37:19.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 10 client.17103 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:19.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:19.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 20 client.17103 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:19.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc5458e0 0x5596bc547cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 4 client.17103 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:19.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.051+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596bc4fb8b0 con 0x5596bc5458e0 2023-10-18T12:37:19.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.052+0000 7f17db7fe700 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc5458e0 0x5596bc547cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.052+0000 7f17baffd700 10 client.17103 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.052+0000 7f17db7fe700 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc5458e0 0x5596bc547cd0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88de88c7708ef34 server_cookie=ba561d876e03abfe in_seq=0 out_seq=0 2023-10-18T12:37:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17baffd700 1 -- 172.21.15.70:0/3587094579 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596bc4fb8b0 con 0x5596bc5458e0 2023-10-18T12:37:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17baffd700 10 client.17103 handle_command_reply: tid=0 2023-10-18T12:37:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 client.17103 shutdown 2023-10-18T12:37:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 msgr2=0x5596bc535d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 0x5596bc535d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f17d40ca480 tx=0x7f17d409a160 comp rx=0 tx=0).stop 2023-10-18T12:37:19.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 shutdown_connections 2023-10-18T12:37:19.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc5458e0 0x5596bc547cd0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc532500 0x5596bc535d90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc538470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/3587094579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc531be0 0x5596bc535760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 >> 172.21.15.70:0/3587094579 conn(0x5596bc4a0720 msgr2=0x5596bc52b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:19.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.053+0000 7f17baffd700 0 client.17103 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:19.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.054+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 shutdown_connections 2023-10-18T12:37:19.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.054+0000 7f17f54e5b80 1 -- 172.21.15.70:0/3587094579 wait complete. 2023-10-18T12:37:19.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.054+0000 7f17f54e5b80 20 client.17103 trim_cache size 0 max 16384 2023-10-18T12:37:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 Processor -- start 2023-10-18T12:37:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 -- start start 2023-10-18T12:37:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc53ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc538080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc526810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc2ff920 con 0x5596bc531be0 2023-10-18T12:37:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc521a70 con 0x5596bc532500 2023-10-18T12:37:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc536670 con 0x5596bc52c5d0 2023-10-18T12:37:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17daffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc538080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17daffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc538080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58688/0 (socket says 172.21.15.70:58688) 2023-10-18T12:37:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17daffd700 1 -- 172.21.15.70:0/1284498936 learned_addr learned my addr 172.21.15.70:0/1284498936 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.067+0000 7f17db7fe700 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc53ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17dbfff700 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc526810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17daffd700 1 -- 172.21.15.70:0/1284498936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 msgr2=0x5596bc53ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17daffd700 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc53ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17daffd700 1 -- 172.21.15.70:0/1284498936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 msgr2=0x5596bc526810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17daffd700 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc526810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17daffd700 1 -- 172.21.15.70:0/1284498936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596bc54c8c0 con 0x5596bc531be0 2023-10-18T12:37:19.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17dbfff700 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc526810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:19.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17daffd700 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc538080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009990 tx=0x7f17d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b4bf3466177784 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:19.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17c27fc700 1 -- 172.21.15.70:0/1284498936 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d001d030 con 0x5596bc531be0 2023-10-18T12:37:19.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17c27fc700 1 -- 172.21.15.70:0/1284498936 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17d001e040 con 0x5596bc531be0 2023-10-18T12:37:19.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.068+0000 7f17c27fc700 1 -- 172.21.15.70:0/1284498936 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0004760 con 0x5596bc531be0 2023-10-18T12:37:19.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1284498936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 msgr2=0x5596bc538080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc538080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009990 tx=0x7f17d0005fd0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1284498936 shutdown_connections 2023-10-18T12:37:19.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc532500 0x5596bc526810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc52c5d0 0x5596bc53ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1284498936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc538080 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1284498936 >> 172.21.15.70:0/1284498936 conn(0x5596bc4a0720 msgr2=0x5596bc52b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1284498936 shutdown_connections 2023-10-18T12:37:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1284498936 wait complete. 2023-10-18T12:37:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.069+0000 7f17f54e5b80 1 -- start start 2023-10-18T12:37:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 Processor -- start 2023-10-18T12:37:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc592cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc593200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc532500 0x5596bc58d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc58dc50 con 0x5596bc531be0 2023-10-18T12:37:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc58dd90 con 0x5596bc52c5d0 2023-10-18T12:37:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596bc58df50 con 0x5596bc532500 2023-10-18T12:37:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17f54e5b80 10 client.0 _ll_register_callbacks cb 0x5596bc365a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc592cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17daffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc593200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17db7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc592cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:36124/0 (socket says 172.21.15.70:36124) 2023-10-18T12:37:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17db7fe700 1 -- 172.21.15.70:0/1850756023 learned_addr learned my addr 172.21.15.70:0/1850756023 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.070+0000 7f17dbfff700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc532500 0x5596bc58d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17daffd700 1 -- 172.21.15.70:0/1850756023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc532500 msgr2=0x5596bc58d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17daffd700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc532500 0x5596bc58d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17daffd700 1 -- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 msgr2=0x5596bc592cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:19.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17daffd700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc592cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17daffd700 1 -- 172.21.15.70:0/1850756023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc10df00 con 0x5596bc531be0 2023-10-18T12:37:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17db7fe700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc592cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17daffd700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc593200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5596bc4fb9c0 tx=0x7f17d00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a7ce1202f8d3f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17baffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17baffd700 1 -- 172.21.15.70:0/1850756023 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d001d030 con 0x5596bc531be0 2023-10-18T12:37:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17baffd700 1 -- 172.21.15.70:0/1850756023 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f17d0005580 con 0x5596bc531be0 2023-10-18T12:37:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17d4004ad0 con 0x5596bc531be0 2023-10-18T12:37:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17baffd700 1 -- 172.21.15.70:0/1850756023 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17d0032070 con 0x5596bc531be0 2023-10-18T12:37:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17f54e5b80 10 client.17109 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:19.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.071+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5596bc542910 con 0x5596bc531be0 2023-10-18T12:37:19.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17baffd700 1 -- 172.21.15.70:0/1850756023 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17d4004ad0 con 0x5596bc531be0 2023-10-18T12:37:19.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17baffd700 1 -- 172.21.15.70:0/1850756023 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f17d000f460 con 0x5596bc531be0 2023-10-18T12:37:19.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17f54e5b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:19.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17f54e5b80 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:19.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17f54e5b80 20 client.17109 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:19.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc59a6e0 0x5596bc59cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:19.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17f54e5b80 4 client.17109 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:37:19.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596bc477e50 con 0x5596bc59a6e0 2023-10-18T12:37:19.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.072+0000 7f17db7fe700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc59a6e0 0x5596bc59cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:19.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17baffd700 10 client.17109 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:19.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17db7fe700 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc59a6e0 0x5596bc59cb00 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6789b38de4b1437f server_cookie=ca5d3e20b5c1f6f5 in_seq=0 out_seq=0 2023-10-18T12:37:19.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17baffd700 1 -- 172.21.15.70:0/1850756023 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596bc477e50 con 0x5596bc59a6e0 2023-10-18T12:37:19.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17baffd700 10 client.17109 handle_command_reply: tid=0 2023-10-18T12:37:19.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17f54e5b80 1 client.17109 shutdown 2023-10-18T12:37:19.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 msgr2=0x5596bc593200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.073+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc593200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5596bc4fb9c0 tx=0x7f17d00058e0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 shutdown_connections 2023-10-18T12:37:19.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5596bc59a6e0 0x5596bc59cb00 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596bc532500 0x5596bc58d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17baffd700 0 client.17109 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:19.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5596bc52c5d0 0x5596bc592cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 --2- 172.21.15.70:0/1850756023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5596bc531be0 0x5596bc593200 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 >> 172.21.15.70:0/1850756023 conn(0x5596bc4a0720 msgr2=0x5596bc5204b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:19.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 shutdown_connections 2023-10-18T12:37:19.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 1 -- 172.21.15.70:0/1850756023 wait complete. 2023-10-18T12:37:19.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17f54e5b80 20 client.17109 trim_cache size 0 max 16384 2023-10-18T12:37:19.103 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:37:19.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f17c40a15f0 msgr2=0x7f17c40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f17c40a15f0 0x7f17c40a3ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0139f0 tx=0x7f17cc009b90 comp rx=0 tx=0).stop 2023-10-18T12:37:19.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 msgr2=0x7f17dc198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:19.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 0x7f17dc198b70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f17d400ef30 tx=0x7f17d400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 shutdown_connections 2023-10-18T12:37:19.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17dc106740 0x7f17dc198b70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f17c40a15f0 0x7f17c40a3ab0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17dc100730 0x7f17dc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 --2- 172.21.15.70:0/2291099683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17dc1010f0 0x7f17dc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.074+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 >> 172.21.15.70:0/2291099683 conn(0x7f17dc0fc200 msgr2=0x7f17dc0fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.075+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 shutdown_connections 2023-10-18T12:37:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:19.075+0000 7f17e1bff700 1 -- 172.21.15.70:0/2291099683 wait complete. 2023-10-18T12:37:19.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:18 smithi070 ceph-mon[98639]: pgmap v663: 129 pgs: 129 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 176 op/s 2023-10-18T12:37:19.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:18 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1059977295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:19.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:18 smithi152 ceph-mon[134767]: pgmap v663: 129 pgs: 129 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 176 op/s 2023-10-18T12:37:19.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:18 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1059977295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:19.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:18 smithi186 ceph-mon[135253]: pgmap v663: 129 pgs: 129 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 176 op/s 2023-10-18T12:37:19.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:18 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1059977295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:19.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1274879 11.20 MB/sec execute 468 sec latency 7603.930 ms 2023-10-18T12:37:20.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:37:20.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:37:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 -- 172.21.15.70:0/2570614350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c701010e0 msgr2=0x7f1c7010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2570614350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c701010e0 0x7f1c7010d5f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b0a0 tx=0x7f1c6800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 -- 172.21.15.70:0/2570614350 shutdown_connections 2023-10-18T12:37:20.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2570614350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c701010e0 0x7f1c7010d5f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2570614350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c70106730 0x7f1c70106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2570614350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c70100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 -- 172.21.15.70:0/2570614350 >> 172.21.15.70:0/2570614350 conn(0x7f1c700fc1b0 msgr2=0x7f1c700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 -- 172.21.15.70:0/2570614350 shutdown_connections 2023-10-18T12:37:20.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.317+0000 7f1c763ff700 1 -- 172.21.15.70:0/2570614350 wait complete. 2023-10-18T12:37:20.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.318+0000 7f1c763ff700 1 Processor -- start 2023-10-18T12:37:20.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.318+0000 7f1c763ff700 1 -- start start 2023-10-18T12:37:20.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c763ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c7019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c763ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c701010e0 0x7f1c7019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c763ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 0x7f1c70198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c763ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c701991f0 con 0x7f1c70106730 2023-10-18T12:37:20.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c763ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70199360 con 0x7f1c70100720 2023-10-18T12:37:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c763ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70199520 con 0x7f1c701010e0 2023-10-18T12:37:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c7019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c7499c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 0x7f1c70198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c7019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:36148/0 (socket says 172.21.15.70:36148) 2023-10-18T12:37:20.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c7499c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 0x7f1c70198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58716/0 (socket says 172.21.15.70:58716) 2023-10-18T12:37:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2369270075 learned_addr learned my addr 172.21.15.70:0/2369270075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c701010e0 0x7f1c7019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2369270075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c701010e0 msgr2=0x7f1c7019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c701010e0 0x7f1c7019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 msgr2=0x7f1c70198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 0x7f1c70198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2369270075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009d00 con 0x7f1c70100720 2023-10-18T12:37:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.319+0000 7f1c7499c700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 0x7f1c70198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.320+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c7019ea60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000e9a0 tx=0x7f1c6000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.320+0000 7f1c6d7fa700 1 -- 172.21.15.70:0/2369270075 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60009790 con 0x7f1c70100720 2023-10-18T12:37:20.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.320+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c70199800 con 0x7f1c70100720 2023-10-18T12:37:20.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.320+0000 7f1c6d7fa700 1 -- 172.21.15.70:0/2369270075 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c60010430 con 0x7f1c70100720 2023-10-18T12:37:20.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.320+0000 7f1c6d7fa700 1 -- 172.21.15.70:0/2369270075 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60018680 con 0x7f1c70100720 2023-10-18T12:37:20.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.320+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c701a5a50 con 0x7f1c70100720 2023-10-18T12:37:20.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.323+0000 7f1c6d7fa700 1 -- 172.21.15.70:0/2369270075 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1c600105a0 con 0x7f1c70100720 2023-10-18T12:37:20.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.324+0000 7f1c89dc7b80 1 Processor -- start 2023-10-18T12:37:20.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.324+0000 7f1c89dc7b80 1 -- start start 2023-10-18T12:37:20.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c6d7fa700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c580a15f0 0x7f1c580a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c6d7fa700 1 -- 172.21.15.70:0/2369270075 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f1c60014070 con 0x7f1c70100720 2023-10-18T12:37:20.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1ab6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab6580 0x5637f1ab0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab0650 0x5637f1ab0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1a23760 con 0x5637f1ab6580 2023-10-18T12:37:20.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1a09ce0 con 0x5637f1ab5c60 2023-10-18T12:37:20.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1a07600 con 0x5637f1ab0650 2023-10-18T12:37:20.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c7499c700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c580a15f0 0x7f1c580a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1ab6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab6580 0x5637f1ab0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1ab6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:36164/0 (socket says 172.21.15.70:36164) 2023-10-18T12:37:20.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c7499c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab0650 0x5637f1ab0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.325+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2065158442 learned_addr learned my addr 172.21.15.70:0/2065158442 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2065158442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab0650 msgr2=0x5637f1ab0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab0650 0x5637f1ab0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2065158442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab6580 msgr2=0x5637f1ab0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab6580 0x5637f1ab0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2065158442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f19c0470 con 0x5637f1ab5c60 2023-10-18T12:37:20.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab6580 0x5637f1ab0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:20.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c7499c700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c580a15f0 0x7f1c580a3ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009040 tx=0x7f1c6800be10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1ab6040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c60096cb0 tx=0x7f1c60099c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c4af060e6c6ddb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c567fc700 1 -- 172.21.15.70:0/2065158442 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6009e390 con 0x5637f1ab5c60 2023-10-18T12:37:20.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c567fc700 1 -- 172.21.15.70:0/2065158442 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c600a25d0 con 0x5637f1ab5c60 2023-10-18T12:37:20.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.326+0000 7f1c567fc700 1 -- 172.21.15.70:0/2065158442 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6009a4a0 con 0x5637f1ab5c60 2023-10-18T12:37:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2065158442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 msgr2=0x5637f1ab6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1ab6040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c60096cb0 tx=0x7f1c60099c90 comp rx=0 tx=0).stop 2023-10-18T12:37:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2065158442 shutdown_connections 2023-10-18T12:37:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab0650 0x5637f1ab0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1ab6040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2065158442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab6580 0x5637f1ab0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2065158442 >> 172.21.15.70:0/2065158442 conn(0x5637f1a247a0 msgr2=0x5637f1aaabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2065158442 shutdown_connections 2023-10-18T12:37:20.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.327+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2065158442 wait complete. 2023-10-18T12:37:20.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.328+0000 7f1c89dc7b80 1 -- start start 2023-10-18T12:37:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.328+0000 7f1c89dc7b80 1 Processor -- start 2023-10-18T12:37:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab0650 0x5637f1ab9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 0x5637f1ab9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1aba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1a02ef0 con 0x5637f1ab5c60 2023-10-18T12:37:20.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f199f790 con 0x5637f1ab0650 2023-10-18T12:37:20.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f199f430 con 0x5637f1ab6580 2023-10-18T12:37:20.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c89dc7b80 10 client.0 _ll_register_callbacks cb 0x5637f18e9b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:20.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 0x5637f1ab9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 0x5637f1ab9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58738/0 (socket says 172.21.15.70:58738) 2023-10-18T12:37:20.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c6f7fe700 1 -- 172.21.15.70:0/3044210347 learned_addr learned my addr 172.21.15.70:0/3044210347 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab0650 0x5637f1ab9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.329+0000 7f1c7499c700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1aba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6f7fe700 1 -- 172.21.15.70:0/3044210347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 msgr2=0x5637f1aba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1aba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6f7fe700 1 -- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab0650 msgr2=0x5637f1ab9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab0650 0x5637f1ab9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6f7fe700 1 -- 172.21.15.70:0/3044210347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c640098e0 con 0x5637f1ab5c60 2023-10-18T12:37:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c7499c700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1aba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab0650 0x5637f1ab9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 0x5637f1ab9da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1c64012820 tx=0x7f1c64012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99203a96414e5b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/3044210347 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64017070 con 0x5637f1ab5c60 2023-10-18T12:37:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/3044210347 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f1c64004030 con 0x5637f1ab5c60 2023-10-18T12:37:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637f1a07f70 con 0x5637f1ab5c60 2023-10-18T12:37:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/3044210347 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c640041f0 con 0x5637f1ab5c60 2023-10-18T12:37:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c89dc7b80 10 client.17121 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f1c701a5a50 con 0x5637f1ab5c60 2023-10-18T12:37:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.330+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/3044210347 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637f1a07f70 con 0x5637f1ab5c60 2023-10-18T12:37:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/3044210347 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c6400f630 con 0x5637f1ab5c60 2023-10-18T12:37:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c89dc7b80 10 client.17121 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c89dc7b80 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c89dc7b80 20 client.17121 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1aca040 0x5637f1acc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c89dc7b80 4 client.17121 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:20.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637f1a7f930 con 0x5637f1aca040 2023-10-18T12:37:20.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.331+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1aca040 0x5637f1acc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.332+0000 7f1c4f7fe700 10 client.17121 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.332+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1aca040 0x5637f1acc460 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95a798a605cd372 server_cookie=481b0f9ab0d1ae5 in_seq=0 out_seq=0 2023-10-18T12:37:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/3044210347 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637f1a7f930 con 0x5637f1aca040 2023-10-18T12:37:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c4f7fe700 10 client.17121 handle_command_reply: tid=0 2023-10-18T12:37:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 client.17121 shutdown 2023-10-18T12:37:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 msgr2=0x5637f1ab9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 0x5637f1ab9da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1c64012820 tx=0x7f1c64012b30 comp rx=0 tx=0).stop 2023-10-18T12:37:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 shutdown_connections 2023-10-18T12:37:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1aca040 0x5637f1acc460 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1aba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c4f7fe700 0 client.17121 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab0650 0x5637f1ab9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/3044210347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab5c60 0x5637f1ab9da0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 >> 172.21.15.70:0/3044210347 conn(0x5637f1a247a0 msgr2=0x5637f1aaf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 shutdown_connections 2023-10-18T12:37:20.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/3044210347 wait complete. 2023-10-18T12:37:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.333+0000 7f1c89dc7b80 20 client.17121 trim_cache size 0 max 16384 2023-10-18T12:37:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 Processor -- start 2023-10-18T12:37:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 -- start start 2023-10-18T12:37:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1ac0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab5c60 0x5637f1ab3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab6580 0x5637f1ab49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f18839a0 con 0x5637f1ab0650 2023-10-18T12:37:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1abaed0 con 0x5637f1ab6580 2023-10-18T12:37:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1aa72b0 con 0x5637f1ab5c60 2023-10-18T12:37:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.355+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1ac0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1ac0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58750/0 (socket says 172.21.15.70:58750) 2023-10-18T12:37:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 -- 172.21.15.70:0/1686100323 learned_addr learned my addr 172.21.15.70:0/1686100323 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c7499c700 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab6580 0x5637f1ab49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab5c60 0x5637f1ab3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 -- 172.21.15.70:0/1686100323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab5c60 msgr2=0x5637f1ab3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab5c60 0x5637f1ab3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 -- 172.21.15.70:0/1686100323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab6580 msgr2=0x5637f1ab49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab6580 0x5637f1ab49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 -- 172.21.15.70:0/1686100323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f182de60 con 0x5637f1ab0650 2023-10-18T12:37:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab5c60 0x5637f1ab3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1ac0f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c2d0 tx=0x7f1c600caf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d21a5dda39cf8e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c56ffd700 1 -- 172.21.15.70:0/1686100323 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600985e0 con 0x5637f1ab0650 2023-10-18T12:37:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c56ffd700 1 -- 172.21.15.70:0/1686100323 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c60010880 con 0x5637f1ab0650 2023-10-18T12:37:20.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.356+0000 7f1c56ffd700 1 -- 172.21.15.70:0/1686100323 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6009a700 con 0x5637f1ab0650 2023-10-18T12:37:20.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/1686100323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 msgr2=0x5637f1ac0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1ac0f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c2d0 tx=0x7f1c600caf30 comp rx=0 tx=0).stop 2023-10-18T12:37:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/1686100323 shutdown_connections 2023-10-18T12:37:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab6580 0x5637f1ab49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1ac0f30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/1686100323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab5c60 0x5637f1ab3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/1686100323 >> 172.21.15.70:0/1686100323 conn(0x5637f1a247a0 msgr2=0x5637f1ab3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/1686100323 shutdown_connections 2023-10-18T12:37:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.357+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/1686100323 wait complete. 2023-10-18T12:37:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 -- start start 2023-10-18T12:37:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 Processor -- start 2023-10-18T12:37:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1abf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1abfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1ac0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1b1de80 con 0x5637f1ab0650 2023-10-18T12:37:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1b1dff0 con 0x5637f1ab5c60 2023-10-18T12:37:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f1b1e1b0 con 0x5637f1ab6580 2023-10-18T12:37:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c89dc7b80 10 client.0 _ll_register_callbacks cb 0x5637f18e9b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1abf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1abf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58764/0 (socket says 172.21.15.70:58764) 2023-10-18T12:37:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.358+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2271950248 learned_addr learned my addr 172.21.15.70:0/2271950248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c7499c700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1ac0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1abfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2271950248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 msgr2=0x5637f1ac0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1ac0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 msgr2=0x5637f1abfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1abfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6ffff700 1 -- 172.21.15.70:0/2271950248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637f19c0470 con 0x5637f1ab0650 2023-10-18T12:37:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1abfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c6ffff700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1abf790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c60098590 tx=0x7f1c600109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593c659933744d01 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/2271950248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6009a6a0 con 0x5637f1ab0650 2023-10-18T12:37:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/2271950248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f1c60099730 con 0x5637f1ab0650 2023-10-18T12:37:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c600a25d0 con 0x5637f1ab0650 2023-10-18T12:37:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/2271950248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c600998f0 con 0x5637f1ab0650 2023-10-18T12:37:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c89dc7b80 10 client.17127 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:20.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.359+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5637f1816ce0 con 0x5637f1ab0650 2023-10-18T12:37:20.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/2271950248 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c600a25d0 con 0x5637f1ab0650 2023-10-18T12:37:20.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/2271950248 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c600b8330 con 0x5637f1ab0650 2023-10-18T12:37:20.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c89dc7b80 10 client.17127 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:20.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c89dc7b80 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:20.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c89dc7b80 20 client.17127 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:20.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1b1f4c0 0x5637f1b218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c89dc7b80 4 client.17127 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:37:20.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5637f19fbed0 con 0x5637f1b1f4c0 2023-10-18T12:37:20.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.360+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1b1f4c0 0x5637f1b218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.361+0000 7f1c4f7fe700 10 client.17127 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.361+0000 7f1c6f7fe700 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1b1f4c0 0x5637f1b218b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7b0b54c85fde8a server_cookie=faf4b81ec4f8e984 in_seq=0 out_seq=0 2023-10-18T12:37:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c4f7fe700 1 -- 172.21.15.70:0/2271950248 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5637f19fbed0 con 0x5637f1b1f4c0 2023-10-18T12:37:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c4f7fe700 10 client.17127 handle_command_reply: tid=0 2023-10-18T12:37:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 client.17127 shutdown 2023-10-18T12:37:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 msgr2=0x5637f1abf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1abf790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c60098590 tx=0x7f1c600109f0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 shutdown_connections 2023-10-18T12:37:20.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5637f1b1f4c0 0x5637f1b218b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637f1ab6580 0x5637f1ac0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637f1ab0650 0x5637f1abf790 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 --2- 172.21.15.70:0/2271950248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5637f1ab5c60 0x5637f1abfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 >> 172.21.15.70:0/2271950248 conn(0x5637f1a247a0 msgr2=0x5637f1aa4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c4f7fe700 0 client.17127 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 shutdown_connections 2023-10-18T12:37:20.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.362+0000 7f1c89dc7b80 1 -- 172.21.15.70:0/2271950248 wait complete. 2023-10-18T12:37:20.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c89dc7b80 20 client.17127 trim_cache size 0 max 16384 2023-10-18T12:37:20.380 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:20.380 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:37:20.381 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "fb321648-66b4-4cae-bda1-828f0c6ee51c", 2023-10-18T12:37:20.381 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:37:20.381 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:20.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c580a15f0 msgr2=0x7f1c580a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c580a15f0 0x7f1c580a3ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009040 tx=0x7f1c6800be10 comp rx=0 tx=0).stop 2023-10-18T12:37:20.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 msgr2=0x7f1c7019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c7019ea60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000e9a0 tx=0x7f1c6000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 shutdown_connections 2023-10-18T12:37:20.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c70106730 0x7f1c70198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f1c580a15f0 0x7f1c580a3ab0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c70100720 0x7f1c7019ea60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 --2- 172.21.15.70:0/2369270075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c701010e0 0x7f1c7019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 >> 172.21.15.70:0/2369270075 conn(0x7f1c700fc1b0 msgr2=0x7f1c7010be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 shutdown_connections 2023-10-18T12:37:20.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.363+0000 7f1c763ff700 1 -- 172.21.15.70:0/2369270075 wait complete. 2023-10-18T12:37:20.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:37:20.444 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:20 smithi070 ceph-mon[98639]: pgmap v664: 129 pgs: 129 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 106 op/s 2023-10-18T12:37:20.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.598+0000 7f187c3ff700 1 -- 172.21.15.70:0/4290070846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18741077f0 msgr2=0x7f1874107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.598+0000 7f187c3ff700 1 --2- 172.21.15.70:0/4290070846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18741077f0 0x7f1874107bd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b0a0 tx=0x7f186c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.600+0000 7f187c3ff700 1 -- 172.21.15.70:0/4290070846 shutdown_connections 2023-10-18T12:37:20.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.600+0000 7f187c3ff700 1 --2- 172.21.15.70:0/4290070846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1874113940 0x7f1874115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.600+0000 7f187c3ff700 1 --2- 172.21.15.70:0/4290070846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18741077f0 0x7f1874107bd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.600+0000 7f187c3ff700 1 --2- 172.21.15.70:0/4290070846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1874108110 0x7f1874113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.600+0000 7f187c3ff700 1 -- 172.21.15.70:0/4290070846 >> 172.21.15.70:0/4290070846 conn(0x7f187406bc20 msgr2=0x7f187406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.601+0000 7f187c3ff700 1 -- 172.21.15.70:0/4290070846 shutdown_connections 2023-10-18T12:37:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.601+0000 7f187c3ff700 1 -- 172.21.15.70:0/4290070846 wait complete. 2023-10-18T12:37:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.602+0000 7f187c3ff700 1 Processor -- start 2023-10-18T12:37:20.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.603+0000 7f187c3ff700 1 -- start start 2023-10-18T12:37:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.603+0000 7f187c3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18741077f0 0x7f187419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.603+0000 7f187c3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 0x7f187419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.603+0000 7f187c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1874113940 0x7f18741a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187c3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18741a2160 con 0x7f1874108110 2023-10-18T12:37:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187c3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18741a22d0 con 0x7f18741077f0 2023-10-18T12:37:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18741a2490 con 0x7f1874113940 2023-10-18T12:37:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187999a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 0x7f187419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187999a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 0x7f187419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58780/0 (socket says 172.21.15.70:58780) 2023-10-18T12:37:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187999a700 1 -- 172.21.15.70:0/3311659283 learned_addr learned my addr 172.21.15.70:0/3311659283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187a19b700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18741077f0 0x7f187419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187999a700 1 -- 172.21.15.70:0/3311659283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1874113940 msgr2=0x7f18741a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187999a700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1874113940 0x7f18741a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.604+0000 7f187999a700 1 -- 172.21.15.70:0/3311659283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18741077f0 msgr2=0x7f187419d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.605+0000 7f187999a700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18741077f0 0x7f187419d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.605+0000 7f187999a700 1 -- 172.21.15.70:0/3311659283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c009d00 con 0x7f1874108110 2023-10-18T12:37:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.605+0000 7f187999a700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 0x7f187419d780 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f186400b530 tx=0x7f186400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.605+0000 7f186b7fe700 1 -- 172.21.15.70:0/3311659283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864011830 con 0x7f1874108110 2023-10-18T12:37:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.606+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18741a2770 con 0x7f1874108110 2023-10-18T12:37:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.606+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18741a9fb0 con 0x7f1874108110 2023-10-18T12:37:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.606+0000 7f186b7fe700 1 -- 172.21.15.70:0/3311659283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1864011e50 con 0x7f1874108110 2023-10-18T12:37:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.606+0000 7f186b7fe700 1 -- 172.21.15.70:0/3311659283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864011830 con 0x7f1874108110 2023-10-18T12:37:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.613+0000 7f188fdf6b80 1 Processor -- start 2023-10-18T12:37:20.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 -- start start 2023-10-18T12:37:20.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35417be0 0x563e35417fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e35412090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35412a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e353856e0 con 0x563e354125d0 2023-10-18T12:37:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3536bc60 con 0x563e35417be0 2023-10-18T12:37:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.615+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35369580 con 0x563e35418500 2023-10-18T12:37:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.616+0000 7f187a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e35412090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.616+0000 7f187a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e35412090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:49186/0 (socket says 172.21.15.70:49186) 2023-10-18T12:37:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.616+0000 7f187a19b700 1 -- 172.21.15.70:0/2122598220 learned_addr learned my addr 172.21.15.70:0/2122598220 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.617+0000 7f187a99c700 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35417be0 0x563e35417fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.617+0000 7f187a19b700 1 -- 172.21.15.70:0/2122598220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35417be0 msgr2=0x563e35417fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.617+0000 7f187a19b700 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35417be0 0x563e35417fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.617+0000 7f187a19b700 1 -- 172.21.15.70:0/2122598220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 msgr2=0x563e35412a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T12:37:20.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.617+0000 7f187a19b700 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35412a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.617+0000 7f187a19b700 1 -- 172.21.15.70:0/2122598220 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e353223f0 con 0x563e35418500 2023-10-18T12:37:20.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.618+0000 7f186b7fe700 1 -- 172.21.15.70:0/3311659283 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1864010ce0 con 0x7f1874108110 2023-10-18T12:37:20.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.619+0000 7f187a19b700 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e35412090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b780 tx=0x7f186c0094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76cb33ac522ff776 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.619+0000 7f1868bfe700 1 -- 172.21.15.70:0/2122598220 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00e040 con 0x563e35418500 2023-10-18T12:37:20.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.619+0000 7f1868bfe700 1 -- 172.21.15.70:0/2122598220 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f186c007d20 con 0x563e35418500 2023-10-18T12:37:20.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.619+0000 7f1868bfe700 1 -- 172.21.15.70:0/2122598220 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c007810 con 0x563e35418500 2023-10-18T12:37:20.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 -- 172.21.15.70:0/2122598220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 msgr2=0x563e35412090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e35412090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b780 tx=0x7f186c0094d0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 -- 172.21.15.70:0/2122598220 shutdown_connections 2023-10-18T12:37:20.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35412a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35417be0 0x563e35417fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/2122598220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e35412090 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.620+0000 7f188fdf6b80 1 -- 172.21.15.70:0/2122598220 >> 172.21.15.70:0/2122598220 conn(0x563e35386720 msgr2=0x563e3540cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.621+0000 7f186b7fe700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f18600a16d0 0x7f18600a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.621+0000 7f186b7fe700 1 -- 172.21.15.70:0/3311659283 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f18640cc9a0 con 0x7f1874108110 2023-10-18T12:37:20.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.623+0000 7f188fdf6b80 1 -- 172.21.15.70:0/2122598220 shutdown_connections 2023-10-18T12:37:20.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.623+0000 7f187a19b700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f18600a16d0 0x7f18600a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.625+0000 7f188fdf6b80 1 -- 172.21.15.70:0/2122598220 wait complete. 2023-10-18T12:37:20.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.625+0000 7f188fdf6b80 1 -- start start 2023-10-18T12:37:20.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.626+0000 7f188fdf6b80 1 Processor -- start 2023-10-18T12:37:20.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.624+0000 7f187a19b700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f18600a16d0 0x7f18600a3b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f186c009610 tx=0x7f186c00b3b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.627+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e3541e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.627+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35417be0 0x563e3541b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.628+0000 7f187a19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e3541e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.628+0000 7f187a19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e3541e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58788/0 (socket says 172.21.15.70:58788) 2023-10-18T12:37:20.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.628+0000 7f187a19b700 1 -- 172.21.15.70:0/444713983 learned_addr learned my addr 172.21.15.70:0/444713983 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.629+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e3541bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.629+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35364e70 con 0x563e354125d0 2023-10-18T12:37:20.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.629+0000 7f187a99c700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e3541bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.630+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35301710 con 0x563e35418500 2023-10-18T12:37:20.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.630+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e353013b0 con 0x563e35417be0 2023-10-18T12:37:20.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.630+0000 7f188fdf6b80 10 client.0 _ll_register_callbacks cb 0x563e3524ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:20.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.631+0000 7f187999a700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35417be0 0x563e3541b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.631+0000 7f187999a700 1 -- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 msgr2=0x563e3541bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.631+0000 7f187999a700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e3541bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.632+0000 7f187999a700 1 -- 172.21.15.70:0/444713983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 msgr2=0x563e3541e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.632+0000 7f187999a700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e3541e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.632+0000 7f187999a700 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1874116660 con 0x563e35417be0 2023-10-18T12:37:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.632+0000 7f187a19b700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e3541e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.633+0000 7f187999a700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35417be0 0x563e3541b740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f18640c9ea0 tx=0x7f1864099180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd572bf256d4330 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.633+0000 7f185dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:37:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.634+0000 7f185dffb700 1 -- 172.21.15.70:0/444713983 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640a1ac0 con 0x563e35417be0 2023-10-18T12:37:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.634+0000 7f185dffb700 1 -- 172.21.15.70:0/444713983 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f1864018310 con 0x563e35417be0 2023-10-18T12:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.634+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e35369ef0 con 0x563e35417be0 2023-10-18T12:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.634+0000 7f185dffb700 1 -- 172.21.15.70:0/444713983 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18640184d0 con 0x563e35417be0 2023-10-18T12:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.634+0000 7f188fdf6b80 10 client.27166 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.634+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563e3518fde0 con 0x563e35417be0 2023-10-18T12:37:20.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.635+0000 7f185dffb700 1 -- 172.21.15.70:0/444713983 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e35369ef0 con 0x563e35417be0 2023-10-18T12:37:20.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.635+0000 7f185dffb700 1 -- 172.21.15.70:0/444713983 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f18640a1630 con 0x563e35417be0 2023-10-18T12:37:20.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.635+0000 7f188fdf6b80 10 client.27166 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:20.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.635+0000 7f188fdf6b80 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:20.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.635+0000 7f188fdf6b80 20 client.27166 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:20.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.636+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e3542be70 0x563e3542e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.636+0000 7f188fdf6b80 4 client.27166 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.636+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e353e18b0 con 0x563e3542be70 2023-10-18T12:37:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.636+0000 7f187a19b700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e3542be70 0x563e3542e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.641+0000 7f187a19b700 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e3542be70 0x563e3542e2b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589e8629bc7ba7a4 server_cookie=34dfc3cd49e17b70 in_seq=0 out_seq=0 2023-10-18T12:37:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.641+0000 7f185dffb700 10 client.27166 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.647+0000 7f185dffb700 1 -- 172.21.15.70:0/444713983 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e353e18b0 con 0x563e3542be70 2023-10-18T12:37:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.647+0000 7f185dffb700 10 client.27166 handle_command_reply: tid=0 2023-10-18T12:37:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.647+0000 7f188fdf6b80 1 client.27166 shutdown 2023-10-18T12:37:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35417be0 msgr2=0x563e3541b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35417be0 0x563e3541b740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f18640c9ea0 tx=0x7f1864099180 comp rx=0 tx=0).stop 2023-10-18T12:37:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 shutdown_connections 2023-10-18T12:37:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e3542be70 0x563e3542e2b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e3541bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e3541e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/444713983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35417be0 0x563e3541b740 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 >> 172.21.15.70:0/444713983 conn(0x563e35386720 msgr2=0x563e35411080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.648+0000 7f185dffb700 0 client.27166 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.649+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 shutdown_connections 2023-10-18T12:37:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.649+0000 7f188fdf6b80 1 -- 172.21.15.70:0/444713983 wait complete. 2023-10-18T12:37:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.649+0000 7f188fdf6b80 20 client.27166 trim_cache size 0 max 16384 2023-10-18T12:37:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:20 smithi152 ceph-mon[134767]: pgmap v664: 129 pgs: 129 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 106 op/s 2023-10-18T12:37:20.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 Processor -- start 2023-10-18T12:37:20.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 -- start start 2023-10-18T12:37:20.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35415dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e35422ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35427630 0x563e35427ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e351e5920 con 0x563e354125d0 2023-10-18T12:37:20.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e353e18b0 con 0x563e35418500 2023-10-18T12:37:20.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.686+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35369ef0 con 0x563e35427630 2023-10-18T12:37:20.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e35422ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e35422ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:36246/0 (socket says 172.21.15.70:36246) 2023-10-18T12:37:20.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 -- 172.21.15.70:0/1307610602 learned_addr learned my addr 172.21.15.70:0/1307610602 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187a99c700 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35427630 0x563e35427ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187a19b700 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35415dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 -- 172.21.15.70:0/1307610602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35427630 msgr2=0x563e35427ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35427630 0x563e35427ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 -- 172.21.15.70:0/1307610602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 msgr2=0x563e35415dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35415dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 -- 172.21.15.70:0/1307610602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e35178c60 con 0x563e35418500 2023-10-18T12:37:20.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f187999a700 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e35422ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f18640c9f50 tx=0x7f1864004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a877118ba0b97a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.687+0000 7f18693ff700 1 -- 172.21.15.70:0/1307610602 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864095670 con 0x563e35418500 2023-10-18T12:37:20.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f18693ff700 1 -- 172.21.15.70:0/1307610602 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18640958d0 con 0x563e35418500 2023-10-18T12:37:20.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f18693ff700 1 -- 172.21.15.70:0/1307610602 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640a1630 con 0x563e35418500 2023-10-18T12:37:20.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1307610602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 msgr2=0x563e35422ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e35422ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f18640c9f50 tx=0x7f1864004860 comp rx=0 tx=0).stop 2023-10-18T12:37:20.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1307610602 shutdown_connections 2023-10-18T12:37:20.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35427630 0x563e35427ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e354125d0 0x563e35415dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1307610602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e35418500 0x563e35422ee0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.688+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1307610602 >> 172.21.15.70:0/1307610602 conn(0x563e35386720 msgr2=0x563e354156b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.689+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1307610602 shutdown_connections 2023-10-18T12:37:20.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.689+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1307610602 wait complete. 2023-10-18T12:37:20.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.689+0000 7f188fdf6b80 1 -- start start 2023-10-18T12:37:20.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.689+0000 7f188fdf6b80 1 Processor -- start 2023-10-18T12:37:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 0x563e35422380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e354228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e35427630 0x563e3547fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35422fa0 con 0x563e35427630 2023-10-18T12:37:20.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35480250 con 0x563e354125d0 2023-10-18T12:37:20.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35480390 con 0x563e35418500 2023-10-18T12:37:20.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f188fdf6b80 10 client.0 _ll_register_callbacks cb 0x563e3524ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:20.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f187a19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 0x563e35422380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f187a19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 0x563e35422380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:36256/0 (socket says 172.21.15.70:36256) 2023-10-18T12:37:20.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f187a19b700 1 -- 172.21.15.70:0/1923654708 learned_addr learned my addr 172.21.15.70:0/1923654708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:20.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f187a99c700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e35427630 0x563e3547fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.690+0000 7f187999a700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e354228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a19b700 1 -- 172.21.15.70:0/1923654708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 msgr2=0x563e354228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a19b700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e354228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a19b700 1 -- 172.21.15.70:0/1923654708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e35427630 msgr2=0x563e3547fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:20.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a19b700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e35427630 0x563e3547fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a19b700 1 -- 172.21.15.70:0/1923654708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e3518fde0 con 0x563e354125d0 2023-10-18T12:37:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a99c700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e35427630 0x563e3547fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f187a19b700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 0x563e35422380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f186c003dd0 tx=0x7f186c003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c161154093faecd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f185dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f185dffb700 1 -- 172.21.15.70:0/1923654708 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00e040 con 0x563e354125d0 2023-10-18T12:37:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.691+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f186c007d20 con 0x563e354125d0 2023-10-18T12:37:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f185dffb700 1 -- 172.21.15.70:0/1923654708 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f186c037040 con 0x563e354125d0 2023-10-18T12:37:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f185dffb700 1 -- 172.21.15.70:0/1923654708 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f186c03a390 con 0x563e354125d0 2023-10-18T12:37:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f188fdf6b80 10 client.27191 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563e351b6250 con 0x563e354125d0 2023-10-18T12:37:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f185dffb700 1 -- 172.21.15.70:0/1923654708 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f186c007d20 con 0x563e354125d0 2023-10-18T12:37:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f185dffb700 1 -- 172.21.15.70:0/1923654708 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f186c01b620 con 0x563e354125d0 2023-10-18T12:37:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f188fdf6b80 10 client.27191 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.692+0000 7f188fdf6b80 10 client.27191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f188fdf6b80 20 client.27191 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e35481870 0x563e35483c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f188fdf6b80 4 client.27191 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:37:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e3535de50 con 0x563e35481870 2023-10-18T12:37:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f187999a700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e35481870 0x563e35483c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f185dffb700 10 client.27191 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.693+0000 7f187999a700 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e35481870 0x563e35483c60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789c786deedcddc2 server_cookie=79ccea3291154f47 in_seq=0 out_seq=0 2023-10-18T12:37:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.694+0000 7f185dffb700 1 -- 172.21.15.70:0/1923654708 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563e3535de50 con 0x563e35481870 2023-10-18T12:37:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.694+0000 7f185dffb700 10 client.27191 handle_command_reply: tid=0 2023-10-18T12:37:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.694+0000 7f188fdf6b80 1 client.27191 shutdown 2023-10-18T12:37:20.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 msgr2=0x563e35422380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 0x563e35422380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f186c003dd0 tx=0x7f186c003e00 comp rx=0 tx=0).stop 2023-10-18T12:37:20.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 shutdown_connections 2023-10-18T12:37:20.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e35481870 0x563e35483c60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e35427630 0x563e3547fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e354125d0 0x563e35422380 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 --2- 172.21.15.70:0/1923654708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e35418500 0x563e354228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f185dffb700 0 client.27191 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 >> 172.21.15.70:0/1923654708 conn(0x563e35386720 msgr2=0x563e35419f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 shutdown_connections 2023-10-18T12:37:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 1 -- 172.21.15.70:0/1923654708 wait complete. 2023-10-18T12:37:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.695+0000 7f188fdf6b80 20 client.27191 trim_cache size 0 max 16384 2023-10-18T12:37:20.715 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:20.715 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T12:37:20.715 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:37:20.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fb321648-66b4-4cae-bda1-828f0c6ee51c": { 2023-10-18T12:37:20.715 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:37:20.716 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "fb321648-66b4-4cae-bda1-828f0c6ee51c", 2023-10-18T12:37:20.716 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:37:20.717 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:37:20.717 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:37:20.717 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:20.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f18600a16d0 msgr2=0x7f18600a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f18600a16d0 0x7f18600a3b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f186c009610 tx=0x7f186c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 msgr2=0x7f187419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:20.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 0x7f187419d780 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f186400b530 tx=0x7f186400b8f0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 shutdown_connections 2023-10-18T12:37:20.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1874113940 0x7f18741a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f18600a16d0 0x7f18600a3b90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18741077f0 0x7f187419d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 --2- 172.21.15.70:0/3311659283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1874108110 0x7f187419d780 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:20.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 >> 172.21.15.70:0/3311659283 conn(0x7f187406bc20 msgr2=0x7f187406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:20.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 shutdown_connections 2023-10-18T12:37:20.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:20.696+0000 7f187c3ff700 1 -- 172.21.15.70:0/3311659283 wait complete. 2023-10-18T12:37:20.721 INFO:tasks.cephfs.filesystem:scrub status for tag:fb321648-66b4-4cae-bda1-828f0c6ee51c - {'path': '/', 'tag': 'fb321648-66b4-4cae-bda1-828f0c6ee51c', 'options': 'recursive,force'} 2023-10-18T12:37:20.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:20 smithi186 ceph-mon[135253]: pgmap v664: 129 pgs: 129 active+clean; 614 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 106 op/s 2023-10-18T12:37:20.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1275043 11.18 MB/sec execute 469 sec latency 8464.723 ms 2023-10-18T12:37:21.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:21 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:37:21.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:21 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:21.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:21 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:37:21.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:21 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:21.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:21 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:37:21.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:21 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:21.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:21 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:37:21.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:21 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:21 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:37:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:21 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:21 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:37:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:21 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:21.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1276322 11.17 MB/sec execute 470 sec latency 33.292 ms 2023-10-18T12:37:22.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:22 smithi070 ceph-mon[98639]: pgmap v665: 129 pgs: 129 active+clean; 612 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2023-10-18T12:37:22.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:22 smithi152 ceph-mon[134767]: pgmap v665: 129 pgs: 129 active+clean; 612 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2023-10-18T12:37:22.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:22 smithi186 ceph-mon[135253]: pgmap v665: 129 pgs: 129 active+clean; 612 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2023-10-18T12:37:22.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1278613 11.17 MB/sec execute 471 sec latency 19.395 ms 2023-10-18T12:37:23.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:23 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:37:23.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:23 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:37:23.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:23 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:37:23.882 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1279998 11.16 MB/sec execute 472 sec latency 17.056 ms 2023-10-18T12:37:24.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:24 smithi152 ceph-mon[134767]: pgmap v666: 129 pgs: 129 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2023-10-18T12:37:24.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:24 smithi186 ceph-mon[135253]: pgmap v666: 129 pgs: 129 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2023-10-18T12:37:24.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:24 smithi070 ceph-mon[98639]: pgmap v666: 129 pgs: 129 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2023-10-18T12:37:24.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1282365 11.17 MB/sec execute 473 sec latency 14.445 ms 2023-10-18T12:37:25.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1283889 11.15 MB/sec execute 474 sec latency 70.724 ms 2023-10-18T12:37:26.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1285903 11.16 MB/sec execute 475 sec latency 34.145 ms 2023-10-18T12:37:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:26 smithi152 ceph-mon[134767]: pgmap v667: 129 pgs: 129 active+clean; 574 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 124 op/s 2023-10-18T12:37:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:26 smithi186 ceph-mon[135253]: pgmap v667: 129 pgs: 129 active+clean; 574 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 124 op/s 2023-10-18T12:37:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:26 smithi070 ceph-mon[98639]: pgmap v667: 129 pgs: 129 active+clean; 574 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 124 op/s 2023-10-18T12:37:27.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1287539 11.15 MB/sec execute 476 sec latency 99.980 ms 2023-10-18T12:37:28.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1289698 11.15 MB/sec execute 477 sec latency 13.805 ms 2023-10-18T12:37:29.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:28 smithi152 ceph-mon[134767]: pgmap v668: 129 pgs: 129 active+clean; 555 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2023-10-18T12:37:29.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:28 smithi186 ceph-mon[135253]: pgmap v668: 129 pgs: 129 active+clean; 555 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2023-10-18T12:37:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:28 smithi070 ceph-mon[98639]: pgmap v668: 129 pgs: 129 active+clean; 555 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2023-10-18T12:37:29.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1291882 11.15 MB/sec execute 478 sec latency 35.705 ms 2023-10-18T12:37:30.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:30 smithi070 ceph-mon[98639]: pgmap v669: 129 pgs: 129 active+clean; 563 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 150 op/s 2023-10-18T12:37:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:30 smithi152 ceph-mon[134767]: pgmap v669: 129 pgs: 129 active+clean; 563 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 150 op/s 2023-10-18T12:37:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:30 smithi186 ceph-mon[135253]: pgmap v669: 129 pgs: 129 active+clean; 563 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 150 op/s 2023-10-18T12:37:30.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1293468 11.15 MB/sec execute 479 sec latency 62.436 ms 2023-10-18T12:37:31.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1295630 11.15 MB/sec execute 480 sec latency 50.248 ms 2023-10-18T12:37:32.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1296932 11.14 MB/sec execute 481 sec latency 62.188 ms 2023-10-18T12:37:33.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:32 smithi152 ceph-mon[134767]: pgmap v670: 129 pgs: 129 active+clean; 566 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2023-10-18T12:37:33.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:32 smithi186 ceph-mon[135253]: pgmap v670: 129 pgs: 129 active+clean; 566 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2023-10-18T12:37:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:32 smithi070 ceph-mon[98639]: pgmap v670: 129 pgs: 129 active+clean; 566 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2023-10-18T12:37:33.883 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1299028 11.14 MB/sec execute 482 sec latency 59.188 ms 2023-10-18T12:37:34.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1300583 11.13 MB/sec execute 483 sec latency 45.725 ms 2023-10-18T12:37:35.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:34 smithi152 ceph-mon[134767]: pgmap v671: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 205 op/s 2023-10-18T12:37:35.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:34 smithi186 ceph-mon[135253]: pgmap v671: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 205 op/s 2023-10-18T12:37:35.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:34 smithi070 ceph-mon[98639]: pgmap v671: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 205 op/s 2023-10-18T12:37:35.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1303179 11.14 MB/sec execute 484 sec latency 33.876 ms 2023-10-18T12:37:36.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1304145 11.13 MB/sec execute 485 sec latency 99.220 ms 2023-10-18T12:37:37.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:36 smithi152 ceph-mon[134767]: pgmap v672: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 198 op/s 2023-10-18T12:37:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:36 smithi186 ceph-mon[135253]: pgmap v672: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 198 op/s 2023-10-18T12:37:37.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:36 smithi070 ceph-mon[98639]: pgmap v672: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 198 op/s 2023-10-18T12:37:37.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1306713 11.13 MB/sec execute 486 sec latency 13.770 ms 2023-10-18T12:37:38.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1307844 11.12 MB/sec execute 487 sec latency 49.046 ms 2023-10-18T12:37:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:38 smithi152 ceph-mon[134767]: pgmap v673: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 181 op/s 2023-10-18T12:37:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:38 smithi186 ceph-mon[135253]: pgmap v673: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 181 op/s 2023-10-18T12:37:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:38 smithi070 ceph-mon[98639]: pgmap v673: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 181 op/s 2023-10-18T12:37:39.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1310459 11.13 MB/sec execute 488 sec latency 33.680 ms 2023-10-18T12:37:40.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:40 smithi070 ceph-mon[98639]: pgmap v674: 129 pgs: 129 active+clean; 585 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 216 op/s 2023-10-18T12:37:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:40 smithi152 ceph-mon[134767]: pgmap v674: 129 pgs: 129 active+clean; 585 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 216 op/s 2023-10-18T12:37:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:40 smithi186 ceph-mon[135253]: pgmap v674: 129 pgs: 129 active+clean; 585 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 216 op/s 2023-10-18T12:37:40.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1311943 11.12 MB/sec execute 489 sec latency 20.839 ms 2023-10-18T12:37:41.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1313982 11.13 MB/sec execute 490 sec latency 120.733 ms 2023-10-18T12:37:42.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1315503 11.12 MB/sec execute 491 sec latency 15.537 ms 2023-10-18T12:37:43.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:42 smithi152 ceph-mon[134767]: pgmap v675: 129 pgs: 129 active+clean; 586 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 186 op/s 2023-10-18T12:37:43.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:42 smithi186 ceph-mon[135253]: pgmap v675: 129 pgs: 129 active+clean; 586 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 186 op/s 2023-10-18T12:37:43.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:42 smithi070 ceph-mon[98639]: pgmap v675: 129 pgs: 129 active+clean; 586 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 186 op/s 2023-10-18T12:37:43.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1317853 11.12 MB/sec execute 492 sec latency 14.734 ms 2023-10-18T12:37:44.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1319526 11.11 MB/sec execute 493 sec latency 71.028 ms 2023-10-18T12:37:45.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:44 smithi152 ceph-mon[134767]: pgmap v676: 129 pgs: 129 active+clean; 594 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-10-18T12:37:45.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:44 smithi186 ceph-mon[135253]: pgmap v676: 129 pgs: 129 active+clean; 594 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-10-18T12:37:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:44 smithi070 ceph-mon[98639]: pgmap v676: 129 pgs: 129 active+clean; 594 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 208 op/s 2023-10-18T12:37:45.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1321459 11.12 MB/sec execute 494 sec latency 100.120 ms 2023-10-18T12:37:46.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1323134 11.11 MB/sec execute 495 sec latency 32.543 ms 2023-10-18T12:37:47.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:46 smithi152 ceph-mon[134767]: pgmap v677: 129 pgs: 129 active+clean; 599 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 184 op/s 2023-10-18T12:37:47.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:46 smithi186 ceph-mon[135253]: pgmap v677: 129 pgs: 129 active+clean; 599 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 184 op/s 2023-10-18T12:37:47.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:46 smithi070 ceph-mon[98639]: pgmap v677: 129 pgs: 129 active+clean; 599 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 184 op/s 2023-10-18T12:37:47.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1325194 11.11 MB/sec execute 496 sec latency 13.727 ms 2023-10-18T12:37:48.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:47 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:37:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:47 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:37:48.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:47 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:37:48.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1327115 11.10 MB/sec execute 497 sec latency 20.869 ms 2023-10-18T12:37:49.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:49 smithi070 ceph-mon[98639]: pgmap v678: 129 pgs: 129 active+clean; 599 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 182 op/s 2023-10-18T12:37:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:49 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:49 smithi152 ceph-mon[134767]: pgmap v678: 129 pgs: 129 active+clean; 599 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 182 op/s 2023-10-18T12:37:49.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:49 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:49 smithi186 ceph-mon[135253]: pgmap v678: 129 pgs: 129 active+clean; 599 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 182 op/s 2023-10-18T12:37:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:49 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:49.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1328846 11.10 MB/sec execute 498 sec latency 15.317 ms 2023-10-18T12:37:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:37:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:37:50.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:37:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:37:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:37:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:37:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:37:50.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:37:50.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1330925 11.10 MB/sec execute 499 sec latency 57.273 ms 2023-10-18T12:37:50.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 -- 172.21.15.70:0/2898476129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 msgr2=0x7f9a5c10ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 --2- 172.21.15.70:0/2898476129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c10ca70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009990 tx=0x7f9a50009ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 -- 172.21.15.70:0/2898476129 shutdown_connections 2023-10-18T12:37:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 --2- 172.21.15.70:0/2898476129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a5c10cfb0 0x7f9a5c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 --2- 172.21.15.70:0/2898476129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a5c072bc0 0x7f9a5c072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 --2- 172.21.15.70:0/2898476129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c10ca70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 -- 172.21.15.70:0/2898476129 >> 172.21.15.70:0/2898476129 conn(0x7f9a5c0fc200 msgr2=0x7f9a5c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.937+0000 7f9a619ff700 1 -- 172.21.15.70:0/2898476129 shutdown_connections 2023-10-18T12:37:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.938+0000 7f9a619ff700 1 -- 172.21.15.70:0/2898476129 wait complete. 2023-10-18T12:37:50.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.939+0000 7f9a619ff700 1 Processor -- start 2023-10-18T12:37:50.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.939+0000 7f9a619ff700 1 -- start start 2023-10-18T12:37:50.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a619ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a5c072bc0 0x7f9a5c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a619ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a619ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a5c10cfb0 0x7f9a5c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a619ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c19dd30 con 0x7f9a5c10cfb0 2023-10-18T12:37:50.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a619ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c19dea0 con 0x7f9a5c0734e0 2023-10-18T12:37:50.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a619ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c19e060 con 0x7f9a5c072bc0 2023-10-18T12:37:50.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34908/0 (socket says 172.21.15.70:34908) 2023-10-18T12:37:50.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/34357921 learned_addr learned my addr 172.21.15.70:0/34357921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:50.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a5b7fe700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a5c10cfb0 0x7f9a5c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.940+0000 7f9a5affd700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a5c072bc0 0x7f9a5c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/34357921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a5c072bc0 msgr2=0x7f9a5c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a5c072bc0 0x7f9a5c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a5c10cfb0 msgr2=0x7f9a5c19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a5c10cfb0 0x7f9a5c19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/34357921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500095f0 con 0x7f9a5c0734e0 2023-10-18T12:37:50.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c199350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005910 tx=0x7f9a50004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:50.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a609fd700 1 -- 172.21.15.70:0/34357921 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5001d070 con 0x7f9a5c0734e0 2023-10-18T12:37:50.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a609fd700 1 -- 172.21.15.70:0/34357921 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a50022470 con 0x7f9a5c0734e0 2023-10-18T12:37:50.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c19e2e0 con 0x7f9a5c0734e0 2023-10-18T12:37:50.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a609fd700 1 -- 172.21.15.70:0/34357921 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50017650 con 0x7f9a5c0734e0 2023-10-18T12:37:50.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.941+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c1a5b80 con 0x7f9a5c0734e0 2023-10-18T12:37:50.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.944+0000 7f9a609fd700 1 -- 172.21.15.70:0/34357921 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9a500177b0 con 0x7f9a5c0734e0 2023-10-18T12:37:50.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.946+0000 7f9a609fd700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9a440a1670 0x7f9a440a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.946+0000 7f9a609fd700 1 -- 172.21.15.70:0/34357921 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9a500ce670 con 0x7f9a5c0734e0 2023-10-18T12:37:50.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.946+0000 7f9a753e3b80 1 Processor -- start 2023-10-18T12:37:50.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.946+0000 7f9a5affd700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9a440a1670 0x7f9a440a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.946+0000 7f9a753e3b80 1 -- start start 2023-10-18T12:37:50.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b84efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b849090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b849a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b7bc6e0 con 0x559a2b8495d0 2023-10-18T12:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b7a2c60 con 0x559a2b84f500 2023-10-18T12:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b7a0580 con 0x559a2b84ebe0 2023-10-18T12:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5affd700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9a440a1670 0x7f9a440a3b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009c60 tx=0x7f9a4c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b849090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b84efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b849a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b849090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34910/0 (socket says 172.21.15.70:34910) 2023-10-18T12:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/2586485565 learned_addr learned my addr 172.21.15.70:0/2586485565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/2586485565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 msgr2=0x559a2b84efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b84efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/2586485565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 msgr2=0x559a2b849a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b849a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.947+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/2586485565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a2b7593f0 con 0x559a2b84f500 2023-10-18T12:37:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.948+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b849090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9a50097d90 tx=0x7f9a50009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ee54ab4a39ae4e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.948+0000 7f9a41ffb700 1 -- 172.21.15.70:0/2586485565 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5009ac80 con 0x559a2b84f500 2023-10-18T12:37:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.948+0000 7f9a41ffb700 1 -- 172.21.15.70:0/2586485565 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a5009ba20 con 0x559a2b84f500 2023-10-18T12:37:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.948+0000 7f9a41ffb700 1 -- 172.21.15.70:0/2586485565 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5009f120 con 0x559a2b84f500 2023-10-18T12:37:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2586485565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 msgr2=0x559a2b849090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b849090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9a50097d90 tx=0x7f9a50009ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2586485565 shutdown_connections 2023-10-18T12:37:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b849a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b84efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2586485565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b849090 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2586485565 >> 172.21.15.70:0/2586485565 conn(0x559a2b7bd720 msgr2=0x559a2b843b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2586485565 shutdown_connections 2023-10-18T12:37:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.949+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2586485565 wait complete. 2023-10-18T12:37:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 -- start start 2023-10-18T12:37:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 Processor -- start 2023-10-18T12:37:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b855580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b852800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b852f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b79be70 con 0x559a2b8495d0 2023-10-18T12:37:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b738710 con 0x559a2b84f500 2023-10-18T12:37:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b7383b0 con 0x559a2b84ebe0 2023-10-18T12:37:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.950+0000 7f9a753e3b80 10 client.0 _ll_register_callbacks cb 0x559a2b682a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b852f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b852f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34926/0 (socket says 172.21.15.70:34926) 2023-10-18T12:37:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b855580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b852800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b852800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41786/0 (socket says 172.21.15.70:41786) 2023-10-18T12:37:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5b7fe700 1 -- 172.21.15.70:0/3885583364 learned_addr learned my addr 172.21.15.70:0/3885583364 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 msgr2=0x559a2b852f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b852f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/3885583364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 msgr2=0x559a2b855580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b855580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/3885583364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c1a5b80 con 0x559a2b84ebe0 2023-10-18T12:37:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5b7fe700 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b852f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b852800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x559a2b855af0 tx=0x7f9a50022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0bc35c625ef43d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.951+0000 7f9a3affd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:37:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a3affd700 1 -- 172.21.15.70:0/3885583364 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500b9070 con 0x559a2b84ebe0 2023-10-18T12:37:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a2b7a0ef0 con 0x559a2b84ebe0 2023-10-18T12:37:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a3affd700 1 -- 172.21.15.70:0/3885583364 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9a5009b320 con 0x559a2b84ebe0 2023-10-18T12:37:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a3affd700 1 -- 172.21.15.70:0/3885583364 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a5009ac80 con 0x559a2b84ebe0 2023-10-18T12:37:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 10 client.27184 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a2b5c6de0 con 0x559a2b84ebe0 2023-10-18T12:37:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a3affd700 1 -- 172.21.15.70:0/3885583364 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a2b7a0ef0 con 0x559a2b84ebe0 2023-10-18T12:37:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a3affd700 1 -- 172.21.15.70:0/3885583364 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a5007e020 con 0x559a2b84ebe0 2023-10-18T12:37:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 10 client.27184 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 10 client.27184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 20 client.27184 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b862680 0x559a2b864a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 4 client.27184 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.952+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a2b8188b0 con 0x559a2b862680 2023-10-18T12:37:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.953+0000 7f9a5affd700 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b862680 0x559a2b864a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.953+0000 7f9a3affd700 10 client.27184 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.953+0000 7f9a5affd700 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b862680 0x559a2b864a70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d677791b221c09 server_cookie=fe83fd5f1cd2f350 in_seq=0 out_seq=0 2023-10-18T12:37:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.954+0000 7f9a3affd700 1 -- 172.21.15.70:0/3885583364 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559a2b8188b0 con 0x559a2b862680 2023-10-18T12:37:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.954+0000 7f9a3affd700 10 client.27184 handle_command_reply: tid=0 2023-10-18T12:37:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.954+0000 7f9a753e3b80 1 client.27184 shutdown 2023-10-18T12:37:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 msgr2=0x559a2b852800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b852800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x559a2b855af0 tx=0x7f9a50022af0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 shutdown_connections 2023-10-18T12:37:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b862680 0x559a2b864a70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b852f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b8495d0 0x559a2b855580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/3885583364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84ebe0 0x559a2b852800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a3affd700 0 client.27184 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:50.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 >> 172.21.15.70:0/3885583364 conn(0x559a2b7bd720 msgr2=0x559a2b843ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:50.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 shutdown_connections 2023-10-18T12:37:50.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 1 -- 172.21.15.70:0/3885583364 wait complete. 2023-10-18T12:37:50.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.955+0000 7f9a753e3b80 20 client.27184 trim_cache size 0 max 16384 2023-10-18T12:37:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 Processor -- start 2023-10-18T12:37:50.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 -- start start 2023-10-18T12:37:50.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 0x559a2b85ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b859c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b8550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b61c920 con 0x559a2b84ebe0 2023-10-18T12:37:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b83bfd0 con 0x559a2b84f500 2023-10-18T12:37:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b844100 con 0x559a2b8495d0 2023-10-18T12:37:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a5affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 0x559a2b85ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b859c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a5affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 0x559a2b85ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41796/0 (socket says 172.21.15.70:41796) 2023-10-18T12:37:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a5affd700 1 -- 172.21.15.70:0/2870758806 learned_addr learned my addr 172.21.15.70:0/2870758806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.974+0000 7f9a5b7fe700 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b8550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a5affd700 1 -- 172.21.15.70:0/2870758806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 msgr2=0x559a2b8550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a5affd700 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b8550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a5affd700 1 -- 172.21.15.70:0/2870758806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 msgr2=0x559a2b859c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a5affd700 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b859c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a5affd700 1 -- 172.21.15.70:0/2870758806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a2b5afc60 con 0x559a2b8495d0 2023-10-18T12:37:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a5affd700 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 0x559a2b85ea90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x559a2b865310 tx=0x7f9a4c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0010bd1497ead5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a427fc700 1 -- 172.21.15.70:0/2870758806 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c022040 con 0x559a2b8495d0 2023-10-18T12:37:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a427fc700 1 -- 172.21.15.70:0/2870758806 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4c023040 con 0x559a2b8495d0 2023-10-18T12:37:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.975+0000 7f9a427fc700 1 -- 172.21.15.70:0/2870758806 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c005280 con 0x559a2b8495d0 2023-10-18T12:37:50.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2870758806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 msgr2=0x559a2b85ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 0x559a2b85ea90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x559a2b865310 tx=0x7f9a4c014000 comp rx=0 tx=0).stop 2023-10-18T12:37:50.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2870758806 shutdown_connections 2023-10-18T12:37:50.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b84f500 0x559a2b8550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b8495d0 0x559a2b85ea90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/2870758806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b859c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2870758806 >> 172.21.15.70:0/2870758806 conn(0x559a2b7bd720 msgr2=0x559a2b85e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:50.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2870758806 shutdown_connections 2023-10-18T12:37:50.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.976+0000 7f9a753e3b80 1 -- 172.21.15.70:0/2870758806 wait complete. 2023-10-18T12:37:50.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.977+0000 7f9a753e3b80 1 -- start start 2023-10-18T12:37:50.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.977+0000 7f9a753e3b80 1 Processor -- start 2023-10-18T12:37:50.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.977+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b8495d0 0x559a2b858430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.977+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b858970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.977+0000 7f9a753e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84f500 0x559a2b858eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b8b6550 con 0x559a2b84ebe0 2023-10-18T12:37:50.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b8b66c0 con 0x559a2b8495d0 2023-10-18T12:37:50.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a753e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a2b8b6880 con 0x559a2b84f500 2023-10-18T12:37:50.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a753e3b80 10 client.0 _ll_register_callbacks cb 0x559a2b682a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:50.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b858970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b858970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35856/0 (socket says 172.21.15.70:35856) 2023-10-18T12:37:50.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b8495d0 0x559a2b858430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/1088809234 learned_addr learned my addr 172.21.15.70:0/1088809234 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:50.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5b7fe700 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84f500 0x559a2b858eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/1088809234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84f500 msgr2=0x559a2b858eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84f500 0x559a2b858eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b8495d0 msgr2=0x559a2b858430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:50.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b8495d0 0x559a2b858430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 -- 172.21.15.70:0/1088809234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c017070 con 0x559a2b84ebe0 2023-10-18T12:37:50.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a5a7fc700 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b858970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005cd0 tx=0x7f9a500cbae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9e5fa324ba19e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:50.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.978+0000 7f9a3affd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:50.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a3affd700 1 -- 172.21.15.70:0/1088809234 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500b9040 con 0x559a2b84ebe0 2023-10-18T12:37:50.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a3affd700 1 -- 172.21.15.70:0/1088809234 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9a5009b320 con 0x559a2b84ebe0 2023-10-18T12:37:50.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a2b84a040 con 0x559a2b84ebe0 2023-10-18T12:37:50.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a3affd700 1 -- 172.21.15.70:0/1088809234 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a500a3620 con 0x559a2b84ebe0 2023-10-18T12:37:50.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 10 client.17160 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:50.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a2b5ed250 con 0x559a2b84ebe0 2023-10-18T12:37:50.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a3affd700 1 -- 172.21.15.70:0/1088809234 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a2b84a040 con 0x559a2b84ebe0 2023-10-18T12:37:50.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a3affd700 1 -- 172.21.15.70:0/1088809234 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a5007e070 con 0x559a2b84ebe0 2023-10-18T12:37:50.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 10 client.17160 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:50.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:50.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 20 client.17160 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:50.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b8b7b60 0x559a2b8b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:50.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 4 client.17160 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:37:50.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.979+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f9a5009ba20 con 0x559a2b8b7b60 2023-10-18T12:37:50.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a5affd700 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b8b7b60 0x559a2b8b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:50.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a5affd700 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b8b7b60 0x559a2b8b9f50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feba1c4ade4a248 server_cookie=2fa4dd0e22c6c437 in_seq=0 out_seq=0 2023-10-18T12:37:50.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a3affd700 10 client.17160 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:50.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a3affd700 1 -- 172.21.15.70:0/1088809234 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f9a5009ba20 con 0x559a2b8b7b60 2023-10-18T12:37:50.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a3affd700 10 client.17160 handle_command_reply: tid=0 2023-10-18T12:37:50.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a753e3b80 1 client.17160 shutdown 2023-10-18T12:37:50.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 msgr2=0x559a2b858970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.980+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b858970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005cd0 tx=0x7f9a500cbae0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 shutdown_connections 2023-10-18T12:37:50.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x559a2b8b7b60 0x559a2b8b9f50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a2b84f500 0x559a2b858eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a2b8495d0 0x559a2b858430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a3affd700 0 client.17160 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:50.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 --2- 172.21.15.70:0/1088809234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559a2b84ebe0 0x559a2b858970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 >> 172.21.15.70:0/1088809234 conn(0x559a2b7bd720 msgr2=0x559a2b84d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:50.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 shutdown_connections 2023-10-18T12:37:50.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 1 -- 172.21.15.70:0/1088809234 wait complete. 2023-10-18T12:37:50.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a753e3b80 20 client.17160 trim_cache size 0 max 16384 2023-10-18T12:37:50.996 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:50.997 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:37:50.997 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:37:50.997 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:50.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9a440a1670 msgr2=0x7f9a440a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a619ff700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9a440a1670 0x7f9a440a3b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009c60 tx=0x7f9a4c009260 comp rx=0 tx=0).stop 2023-10-18T12:37:50.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 msgr2=0x7f9a5c199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:50.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.981+0000 7f9a619ff700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c199350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005910 tx=0x7f9a50004730 comp rx=0 tx=0).stop 2023-10-18T12:37:50.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 shutdown_connections 2023-10-18T12:37:50.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a5c10cfb0 0x7f9a5c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a5c072bc0 0x7f9a5c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9a440a1670 0x7f9a440a3b30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 --2- 172.21.15.70:0/34357921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a5c0734e0 0x7f9a5c199350 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:50.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 >> 172.21.15.70:0/34357921 conn(0x7f9a5c0fc200 msgr2=0x7f9a5c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:50.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 shutdown_connections 2023-10-18T12:37:51.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:50.982+0000 7f9a619ff700 1 -- 172.21.15.70:0/34357921 wait complete. 2023-10-18T12:37:51.004 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:37:51.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:37:51.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 -- 172.21.15.70:0/2975028739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc072bc0 msgr2=0x7ff1bc072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 --2- 172.21.15.70:0/2975028739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc072fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009990 tx=0x7ff1ac009ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 -- 172.21.15.70:0/2975028739 shutdown_connections 2023-10-18T12:37:51.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 --2- 172.21.15.70:0/2975028739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 0x7ff1bc10f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 --2- 172.21.15.70:0/2975028739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc072fa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 --2- 172.21.15.70:0/2975028739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc10c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 -- 172.21.15.70:0/2975028739 >> 172.21.15.70:0/2975028739 conn(0x7ff1bc0fbfb0 msgr2=0x7ff1bc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 -- 172.21.15.70:0/2975028739 shutdown_connections 2023-10-18T12:37:51.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.222+0000 7ff1c139d700 1 -- 172.21.15.70:0/2975028739 wait complete. 2023-10-18T12:37:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.223+0000 7ff1c139d700 1 Processor -- start 2023-10-18T12:37:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.223+0000 7ff1c139d700 1 -- start start 2023-10-18T12:37:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c139d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c139d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c139d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 0x7ff1bc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c139d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc199080 con 0x7ff1bc072bc0 2023-10-18T12:37:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c139d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc1991f0 con 0x7ff1bc0734e0 2023-10-18T12:37:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c139d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc1993b0 con 0x7ff1bc10cd40 2023-10-18T12:37:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34944/0 (socket says 172.21.15.70:34944) 2023-10-18T12:37:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35866/0 (socket says 172.21.15.70:35866) 2023-10-18T12:37:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/1089484736 learned_addr learned my addr 172.21.15.70:0/1089484736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1c0b9c700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 0x7ff1bc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/1089484736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 msgr2=0x7ff1bc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 0x7ff1bc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 msgr2=0x7ff1bc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.224+0000 7ff1bb7fe700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/1089484736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac0095f0 con 0x7ff1bc0734e0 2023-10-18T12:37:51.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1bbfff700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:51.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1c0b9c700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 0x7ff1bc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:51.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1bb7fe700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc19ee30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e9a0 tx=0x7ff1b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0009790 con 0x7ff1bc0734e0 2023-10-18T12:37:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1b0010430 con 0x7ff1bc0734e0 2023-10-18T12:37:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc199690 con 0x7ff1bc0734e0 2023-10-18T12:37:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0018680 con 0x7ff1bc0734e0 2023-10-18T12:37:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.225+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc1a58e0 con 0x7ff1bc0734e0 2023-10-18T12:37:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.226+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc107f50 con 0x7ff1bc0734e0 2023-10-18T12:37:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.228+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff1b00105a0 con 0x7ff1bc0734e0 2023-10-18T12:37:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.230+0000 7ff1b97fa700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff1a40a15f0 0x7ff1a40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.230+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7ff1b0014070 con 0x7ff1bc0734e0 2023-10-18T12:37:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.230+0000 7ff1bbfff700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff1a40a15f0 0x7ff1a40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.230+0000 7ff1bbfff700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff1a40a15f0 0x7ff1a40a3ab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00b2e0 tx=0x7ff1ac005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.232+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff1b00965f0 con 0x7ff1bc0734e0 2023-10-18T12:37:51.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:51 smithi070 ceph-mon[98639]: pgmap v679: 129 pgs: 129 active+clean; 611 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 213 op/s 2023-10-18T12:37:51.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.515+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1bc199ee0 con 0x7ff1bc0734e0 2023-10-18T12:37:51.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.516+0000 7ff1b97fa700 1 -- 172.21.15.70:0/1089484736 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7ff1b00cc090 con 0x7ff1bc0734e0 2023-10-18T12:37:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:37:51.519 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:37:51.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff1a40a15f0 msgr2=0x7ff1a40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff1a40a15f0 0x7ff1a40a3ab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00b2e0 tx=0x7ff1ac005fd0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 msgr2=0x7ff1bc19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc19ee30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e9a0 tx=0x7ff1b000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 shutdown_connections 2023-10-18T12:37:51.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1bc10cd40 0x7ff1bc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1bc072bc0 0x7ff1bc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff1a40a15f0 0x7ff1a40a3ab0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 --2- 172.21.15.70:0/1089484736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1bc0734e0 0x7ff1bc19ee30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 >> 172.21.15.70:0/1089484736 conn(0x7ff1bc0fbfb0 msgr2=0x7ff1bc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 shutdown_connections 2023-10-18T12:37:51.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.518+0000 7ff1c139d700 1 -- 172.21.15.70:0/1089484736 wait complete. 2023-10-18T12:37:51.524 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:37:51.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:37:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:51 smithi152 ceph-mon[134767]: pgmap v679: 129 pgs: 129 active+clean; 611 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 213 op/s 2023-10-18T12:37:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:51 smithi186 ceph-mon[135253]: pgmap v679: 129 pgs: 129 active+clean; 611 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 213 op/s 2023-10-18T12:37:51.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 -- 172.21.15.70:0/1582039282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144100710 msgr2=0x7fc144100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 --2- 172.21.15.70:0/1582039282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144100710 0x7fc144100b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc138009990 tx=0x7fc138009ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 -- 172.21.15.70:0/1582039282 shutdown_connections 2023-10-18T12:37:51.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 --2- 172.21.15.70:0/1582039282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1441010d0 0x7fc14410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 --2- 172.21.15.70:0/1582039282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144106720 0x7fc144106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 --2- 172.21.15.70:0/1582039282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144100710 0x7fc144100b90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 -- 172.21.15.70:0/1582039282 >> 172.21.15.70:0/1582039282 conn(0x7fc1440fc200 msgr2=0x7fc1440fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 -- 172.21.15.70:0/1582039282 shutdown_connections 2023-10-18T12:37:51.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.753+0000 7fc149bff700 1 -- 172.21.15.70:0/1582039282 wait complete. 2023-10-18T12:37:51.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.754+0000 7fc149bff700 1 Processor -- start 2023-10-18T12:37:51.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.754+0000 7fc149bff700 1 -- start start 2023-10-18T12:37:51.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc149bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 0x7fc144072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc149bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1441010d0 0x7fc144073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc149bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144106720 0x7fc144073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc149bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1441a4f20 con 0x7fc144106720 2023-10-18T12:37:51.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc149bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1441a5090 con 0x7fc144100710 2023-10-18T12:37:51.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc149bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1441a5250 con 0x7fc1441010d0 2023-10-18T12:37:51.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 0x7fc144072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 0x7fc144072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34948/0 (socket says 172.21.15.70:34948) 2023-10-18T12:37:51.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 -- 172.21.15.70:0/2338932862 learned_addr learned my addr 172.21.15.70:0/2338932862 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:51.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc143fff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144106720 0x7fc144073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc142ffd700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1441010d0 0x7fc144073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 -- 172.21.15.70:0/2338932862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1441010d0 msgr2=0x7fc144073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1441010d0 0x7fc144073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 -- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144106720 msgr2=0x7fc144073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144106720 0x7fc144073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.755+0000 7fc1437fe700 1 -- 172.21.15.70:0/2338932862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1380095f0 con 0x7fc144100710 2023-10-18T12:37:51.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.756+0000 7fc1437fe700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 0x7fc144072af0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc13400e9a0 tx=0x7fc13400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.756+0000 7fc140ff9700 1 -- 172.21.15.70:0/2338932862 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134009790 con 0x7fc144100710 2023-10-18T12:37:51.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.756+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1441a5530 con 0x7fc144100710 2023-10-18T12:37:51.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.756+0000 7fc140ff9700 1 -- 172.21.15.70:0/2338932862 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc134010430 con 0x7fc144100710 2023-10-18T12:37:51.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.756+0000 7fc140ff9700 1 -- 172.21.15.70:0/2338932862 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134018680 con 0x7fc144100710 2023-10-18T12:37:51.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.756+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1441a5a90 con 0x7fc144100710 2023-10-18T12:37:51.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.759+0000 7fc140ff9700 1 -- 172.21.15.70:0/2338932862 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc1340105a0 con 0x7fc144100710 2023-10-18T12:37:51.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.760+0000 7fc15d436b80 1 Processor -- start 2023-10-18T12:37:51.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.760+0000 7fc15d436b80 1 -- start start 2023-10-18T12:37:51.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c0972fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 0x55c5c096d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096d5d0 0x55c5c096da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc140ff9700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc12c0a1670 0x7fc12c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc15d436b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c08e06e0 con 0x55c5c096d5d0 2023-10-18T12:37:51.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc15d436b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c08c6c60 con 0x55c5c0973500 2023-10-18T12:37:51.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc15d436b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c08c4580 con 0x55c5c0972be0 2023-10-18T12:37:51.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc140ff9700 1 -- 172.21.15.70:0/2338932862 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fc134014070 con 0x7fc144100710 2023-10-18T12:37:51.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 0x55c5c096d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 0x55c5c096d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34952/0 (socket says 172.21.15.70:34952) 2023-10-18T12:37:51.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 -- 172.21.15.70:0/4016125723 learned_addr learned my addr 172.21.15.70:0/4016125723 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:51.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc143fff700 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c0972fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc142ffd700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc12c0a1670 0x7fc12c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 -- 172.21.15.70:0/4016125723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 msgr2=0x55c5c0972fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c0972fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 -- 172.21.15.70:0/4016125723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096d5d0 msgr2=0x55c5c096da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096d5d0 0x55c5c096da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.761+0000 7fc1437fe700 1 -- 172.21.15.70:0/4016125723 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c087d3f0 con 0x55c5c0973500 2023-10-18T12:37:51.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc1437fe700 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 0x55c5c096d090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc134099d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb3c5e5c83ea7e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc129ffb700 1 -- 172.21.15.70:0/4016125723 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13409e120 con 0x55c5c0973500 2023-10-18T12:37:51.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc129ffb700 1 -- 172.21.15.70:0/4016125723 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13409e300 con 0x55c5c0973500 2023-10-18T12:37:51.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc129ffb700 1 -- 172.21.15.70:0/4016125723 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340caba0 con 0x55c5c0973500 2023-10-18T12:37:51.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016125723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 msgr2=0x55c5c096d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 0x55c5c096d090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc134099d90 comp rx=0 tx=0).stop 2023-10-18T12:37:51.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.762+0000 7fc142ffd700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc12c0a1670 0x7fc12c0a3b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc144074220 tx=0x7fc138005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016125723 shutdown_connections 2023-10-18T12:37:51.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096d5d0 0x55c5c096da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c0972fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016125723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0973500 0x55c5c096d090 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016125723 >> 172.21.15.70:0/4016125723 conn(0x55c5c08e1720 msgr2=0x55c5c0967b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016125723 shutdown_connections 2023-10-18T12:37:51.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016125723 wait complete. 2023-10-18T12:37:51.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 -- start start 2023-10-18T12:37:51.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.763+0000 7fc15d436b80 1 Processor -- start 2023-10-18T12:37:51.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 0x55c5c09794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c09767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c0976e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c08bfe70 con 0x55c5c0973500 2023-10-18T12:37:51.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c085c710 con 0x55c5c0972be0 2023-10-18T12:37:51.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c085c3b0 con 0x55c5c096d5d0 2023-10-18T12:37:51.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc15d436b80 10 client.0 _ll_register_callbacks cb 0x55c5c07a6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:51.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc142ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c09767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 0x55c5c09794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc143fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c0976e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 0x55c5c09794f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41878/0 (socket says 172.21.15.70:41878) 2023-10-18T12:37:51.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc1437fe700 1 -- 172.21.15.70:0/3942590798 learned_addr learned my addr 172.21.15.70:0/3942590798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:51.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.764+0000 7fc142ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c09767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34956/0 (socket says 172.21.15.70:34956) 2023-10-18T12:37:51.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc1437fe700 1 -- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 msgr2=0x55c5c09767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc1437fe700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c09767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc1437fe700 1 -- 172.21.15.70:0/3942590798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 msgr2=0x55c5c0976e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc1437fe700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c0976e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc1437fe700 1 -- 172.21.15.70:0/3942590798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1441a5a90 con 0x55c5c096d5d0 2023-10-18T12:37:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc143fff700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c0976e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc142ffd700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c09767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc1437fe700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 0x55c5c09794f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc13409a140 tx=0x7fc134010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf09ed3bed73447 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc122ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:37:51.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc122ffd700 1 -- 172.21.15.70:0/3942590798 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340cadb0 con 0x55c5c096d5d0 2023-10-18T12:37:51.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc122ffd700 1 -- 172.21.15.70:0/3942590798 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fc134098550 con 0x55c5c096d5d0 2023-10-18T12:37:51.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5c08c4ef0 con 0x55c5c096d5d0 2023-10-18T12:37:51.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc122ffd700 1 -- 172.21.15.70:0/3942590798 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc13409a410 con 0x55c5c096d5d0 2023-10-18T12:37:51.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc15d436b80 10 client.27208 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:51.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.765+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c5c06eade0 con 0x55c5c096d5d0 2023-10-18T12:37:51.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc122ffd700 1 -- 172.21.15.70:0/3942590798 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5c08c4ef0 con 0x55c5c096d5d0 2023-10-18T12:37:51.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc122ffd700 1 -- 172.21.15.70:0/3942590798 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1340a20b0 con 0x55c5c096d5d0 2023-10-18T12:37:51.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc15d436b80 10 client.27208 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:51.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc15d436b80 10 client.27208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:51.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc15d436b80 20 client.27208 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:51.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc15d436b80 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c0986a70 0x55c5c0988e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc15d436b80 4 client.27208 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:51.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5c093c8b0 con 0x55c5c0986a70 2023-10-18T12:37:51.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.766+0000 7fc142ffd700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c0986a70 0x55c5c0988e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.767+0000 7fc122ffd700 10 client.27208 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:51.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.767+0000 7fc142ffd700 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c0986a70 0x55c5c0988e60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771f82ec0f1c5ea7 server_cookie=70ff1175afabe89d in_seq=0 out_seq=0 2023-10-18T12:37:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc122ffd700 1 -- 172.21.15.70:0/3942590798 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5c093c8b0 con 0x55c5c0986a70 2023-10-18T12:37:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc122ffd700 10 client.27208 handle_command_reply: tid=0 2023-10-18T12:37:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 client.27208 shutdown 2023-10-18T12:37:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 msgr2=0x55c5c09794f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 0x55c5c09794f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc13409a140 tx=0x7fc134010ab0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 shutdown_connections 2023-10-18T12:37:51.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c0986a70 0x55c5c0988e60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c0976e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c096d5d0 0x55c5c09794f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc122ffd700 0 client.27208 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:51.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 --2- 172.21.15.70:0/3942590798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c09767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 >> 172.21.15.70:0/3942590798 conn(0x55c5c08e1720 msgr2=0x55c5c096c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 shutdown_connections 2023-10-18T12:37:51.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 1 -- 172.21.15.70:0/3942590798 wait complete. 2023-10-18T12:37:51.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.769+0000 7fc15d436b80 20 client.27208 trim_cache size 0 max 16384 2023-10-18T12:37:51.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.783+0000 7fc15d436b80 1 Processor -- start 2023-10-18T12:37:51.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.783+0000 7fc15d436b80 1 -- start start 2023-10-18T12:37:51.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c097deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c09760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c096dd70 0x55c5c09765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc15d436b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c0740920 con 0x55c5c0973500 2023-10-18T12:37:51.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc15d436b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c0983960 con 0x55c5c096dd70 2023-10-18T12:37:51.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc15d436b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c0977770 con 0x55c5c0972be0 2023-10-18T12:37:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc142ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c09760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc142ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c09760a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35902/0 (socket says 172.21.15.70:35902) 2023-10-18T12:37:51.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc142ffd700 1 -- 172.21.15.70:0/1872355195 learned_addr learned my addr 172.21.15.70:0/1872355195 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:51.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc1437fe700 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c097deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.784+0000 7fc143fff700 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c096dd70 0x55c5c09765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc142ffd700 1 -- 172.21.15.70:0/1872355195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 msgr2=0x55c5c097deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc142ffd700 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c097deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc142ffd700 1 -- 172.21.15.70:0/1872355195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c096dd70 msgr2=0x55c5c09765e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc142ffd700 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c096dd70 0x55c5c09765e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc142ffd700 1 -- 172.21.15.70:0/1872355195 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c06d3c60 con 0x55c5c0973500 2023-10-18T12:37:51.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc1437fe700 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c097deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:51.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.785+0000 7fc142ffd700 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c09760a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098e0 tx=0x7fc138020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc151ec1ecdb9c51 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc12a7fc700 1 -- 172.21.15.70:0/1872355195 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138035af0 con 0x55c5c0973500 2023-10-18T12:37:51.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc12a7fc700 1 -- 172.21.15.70:0/1872355195 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13801e040 con 0x55c5c0973500 2023-10-18T12:37:51.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc12a7fc700 1 -- 172.21.15.70:0/1872355195 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138004890 con 0x55c5c0973500 2023-10-18T12:37:51.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 -- 172.21.15.70:0/1872355195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 msgr2=0x55c5c09760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c09760a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098e0 tx=0x7fc138020000 comp rx=0 tx=0).stop 2023-10-18T12:37:51.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 -- 172.21.15.70:0/1872355195 shutdown_connections 2023-10-18T12:37:51.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c096dd70 0x55c5c09765e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0972be0 0x55c5c097deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 --2- 172.21.15.70:0/1872355195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c0973500 0x55c5c09760a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.786+0000 7fc15d436b80 1 -- 172.21.15.70:0/1872355195 >> 172.21.15.70:0/1872355195 conn(0x55c5c08e1720 msgr2=0x55c5c09750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.787+0000 7fc15d436b80 1 -- 172.21.15.70:0/1872355195 shutdown_connections 2023-10-18T12:37:51.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.787+0000 7fc15d436b80 1 -- 172.21.15.70:0/1872355195 wait complete. 2023-10-18T12:37:51.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.787+0000 7fc15d436b80 1 -- start start 2023-10-18T12:37:51.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.787+0000 7fc15d436b80 1 Processor -- start 2023-10-18T12:37:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 0x55c5c097db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c096c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0973500 0x55c5c096c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c096cc10 con 0x55c5c096dd70 2023-10-18T12:37:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c096cd80 con 0x55c5c0972be0 2023-10-18T12:37:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c096cf40 con 0x55c5c0973500 2023-10-18T12:37:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc15d436b80 10 client.0 _ll_register_callbacks cb 0x55c5c07a6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 0x55c5c097db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc142ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c096c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc142ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c096c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34974/0 (socket says 172.21.15.70:34974) 2023-10-18T12:37:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc1437fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 0x55c5c097db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35910/0 (socket says 172.21.15.70:35910) 2023-10-18T12:37:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc142ffd700 1 -- 172.21.15.70:0/4016457192 learned_addr learned my addr 172.21.15.70:0/4016457192 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.788+0000 7fc143fff700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0973500 0x55c5c096c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc142ffd700 1 -- 172.21.15.70:0/4016457192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0973500 msgr2=0x55c5c096c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc142ffd700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0973500 0x55c5c096c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc142ffd700 1 -- 172.21.15.70:0/4016457192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 msgr2=0x55c5c097db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc142ffd700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 0x55c5c097db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc142ffd700 1 -- 172.21.15.70:0/4016457192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5c06eade0 con 0x55c5c0972be0 2023-10-18T12:37:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc1437fe700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 0x55c5c097db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc142ffd700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c096c120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc138009ce0 tx=0x7fc13800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598fe95a88042620 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc122ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc122ffd700 1 -- 172.21.15.70:0/4016457192 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138035350 con 0x55c5c0972be0 2023-10-18T12:37:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc13409e300 con 0x55c5c0972be0 2023-10-18T12:37:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc122ffd700 1 -- 172.21.15.70:0/4016457192 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fc1380354b0 con 0x55c5c0972be0 2023-10-18T12:37:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc122ffd700 1 -- 172.21.15.70:0/4016457192 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc138004500 con 0x55c5c0972be0 2023-10-18T12:37:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc15d436b80 10 client.27233 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.789+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fc14410dee0 con 0x55c5c0972be0 2023-10-18T12:37:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc122ffd700 1 -- 172.21.15.70:0/4016457192 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13409e300 con 0x55c5c0972be0 2023-10-18T12:37:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc122ffd700 1 -- 172.21.15.70:0/4016457192 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc13803d070 con 0x55c5c0972be0 2023-10-18T12:37:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc15d436b80 10 client.27233 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc15d436b80 10 client.27233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc15d436b80 20 client.27233 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c09db960 0x55c5c09ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc15d436b80 4 client.27233 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:37:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5c08b8e50 con 0x55c5c09db960 2023-10-18T12:37:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.790+0000 7fc1437fe700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c09db960 0x55c5c09ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc1437fe700 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c09db960 0x55c5c09ddd50 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7783dbcf859a7b51 server_cookie=7a1be110903c45c4 in_seq=0 out_seq=0 2023-10-18T12:37:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc122ffd700 10 client.27233 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc122ffd700 1 -- 172.21.15.70:0/4016457192 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5c08b8e50 con 0x55c5c09db960 2023-10-18T12:37:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc122ffd700 10 client.27233 handle_command_reply: tid=0 2023-10-18T12:37:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 client.27233 shutdown 2023-10-18T12:37:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 msgr2=0x55c5c096c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c096c120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc138009ce0 tx=0x7fc13800b410 comp rx=0 tx=0).stop 2023-10-18T12:37:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 shutdown_connections 2023-10-18T12:37:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c5c09db960 0x55c5c09ddd50 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5c0973500 0x55c5c096c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc122ffd700 0 client.27233 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5c096dd70 0x55c5c097db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 --2- 172.21.15.70:0/4016457192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c5c0972be0 0x55c5c096c120 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.791+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 >> 172.21.15.70:0/4016457192 conn(0x55c5c08e1720 msgr2=0x55c5c09760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.792+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 shutdown_connections 2023-10-18T12:37:51.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.792+0000 7fc15d436b80 1 -- 172.21.15.70:0/4016457192 wait complete. 2023-10-18T12:37:51.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.792+0000 7fc15d436b80 20 client.27233 trim_cache size 0 max 16384 2023-10-18T12:37:51.815 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:37:51.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc12c0a1670 msgr2=0x7fc12c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc12c0a1670 0x7fc12c0a3b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc144074220 tx=0x7fc138005fd0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 msgr2=0x7fc144072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:51.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 0x7fc144072af0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc13400e9a0 tx=0x7fc13400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 shutdown_connections 2023-10-18T12:37:51.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc144106720 0x7fc144073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fc12c0a1670 0x7fc12c0a3b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc144100710 0x7fc144072af0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 --2- 172.21.15.70:0/2338932862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1441010d0 0x7fc144073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:51.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 >> 172.21.15.70:0/2338932862 conn(0x7fc1440fc200 msgr2=0x7fc1440fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:51.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 shutdown_connections 2023-10-18T12:37:51.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:51.793+0000 7fc149bff700 1 -- 172.21.15.70:0/2338932862 wait complete. 2023-10-18T12:37:51.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1332509 11.10 MB/sec execute 500 sec latency 18.475 ms 2023-10-18T12:37:52.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:52 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1089484736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:52 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1089484736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:52 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1089484736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:52.820 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:37:52.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:37:52.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1334915 11.10 MB/sec execute 501 sec latency 42.971 ms 2023-10-18T12:37:53.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 -- 172.21.15.70:0/2146092356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 msgr2=0x7fa95c1111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 --2- 172.21.15.70:0/2146092356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c1111e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0098f0 tx=0x7fa94c009c00 comp rx=0 tx=0).stop 2023-10-18T12:37:53.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 -- 172.21.15.70:0/2146092356 shutdown_connections 2023-10-18T12:37:53.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 --2- 172.21.15.70:0/2146092356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 0x7fa95c113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 --2- 172.21.15.70:0/2146092356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c072b90 0x7fa95c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 --2- 172.21.15.70:0/2146092356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c1111e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 -- 172.21.15.70:0/2146092356 >> 172.21.15.70:0/2146092356 conn(0x7fa95c0fc200 msgr2=0x7fa95c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 -- 172.21.15.70:0/2146092356 shutdown_connections 2023-10-18T12:37:53.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.034+0000 7fa963dff700 1 -- 172.21.15.70:0/2146092356 wait complete. 2023-10-18T12:37:53.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.035+0000 7fa963dff700 1 Processor -- start 2023-10-18T12:37:53.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.035+0000 7fa963dff700 1 -- start start 2023-10-18T12:37:53.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa963dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c072b90 0x7fa95c19d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa963dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa963dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 0x7fa95c1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa963dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1a2170 con 0x7fa95c111720 2023-10-18T12:37:53.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa963dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1a22e0 con 0x7fa95c0734b0 2023-10-18T12:37:53.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa963dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1a24a0 con 0x7fa95c072b90 2023-10-18T12:37:53.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 0x7fa95c1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96139a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa961b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c072b90 0x7fa95c19d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96139a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c19d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34992/0 (socket says 172.21.15.70:34992) 2023-10-18T12:37:53.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96139a700 1 -- 172.21.15.70:0/1201784954 learned_addr learned my addr 172.21.15.70:0/1201784954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 -- 172.21.15.70:0/1201784954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c072b90 msgr2=0x7fa95c19d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c072b90 0x7fa95c19d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 -- 172.21.15.70:0/1201784954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 msgr2=0x7fa95c19d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c19d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 -- 172.21.15.70:0/1201784954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c009520 con 0x7fa95c111720 2023-10-18T12:37:53.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96139a700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.036+0000 7fa96239c700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 0x7fa95c1a1b30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ef90 tx=0x7fa95400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.037+0000 7fa952ffd700 1 -- 172.21.15.70:0/1201784954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954018070 con 0x7fa95c111720 2023-10-18T12:37:53.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.037+0000 7fa952ffd700 1 -- 172.21.15.70:0/1201784954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95400f040 con 0x7fa95c111720 2023-10-18T12:37:53.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.037+0000 7fa952ffd700 1 -- 172.21.15.70:0/1201784954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95400cc20 con 0x7fa95c111720 2023-10-18T12:37:53.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.037+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c1a2720 con 0x7fa95c111720 2023-10-18T12:37:53.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.037+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1a9fc0 con 0x7fa95c111720 2023-10-18T12:37:53.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.040+0000 7fa952ffd700 1 -- 172.21.15.70:0/1201784954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa954003a00 con 0x7fa95c111720 2023-10-18T12:37:53.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.041+0000 7fa9776e4b80 1 Processor -- start 2023-10-18T12:37:53.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.041+0000 7fa9776e4b80 1 -- start start 2023-10-18T12:37:53.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.041+0000 7fa952ffd700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9480a15f0 0x7fa9480a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702157040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702151110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7020c4760 con 0x55b702151650 2023-10-18T12:37:53.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7020aace0 con 0x55b702157580 2023-10-18T12:37:53.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7020a8600 con 0x55b702156c60 2023-10-18T12:37:53.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa952ffd700 1 -- 172.21.15.70:0/1201784954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa9540ccfb0 con 0x7fa95c111720 2023-10-18T12:37:53.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa96139a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41908/0 (socket says 172.21.15.70:41908) 2023-10-18T12:37:53.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 -- 172.21.15.70:0/2882312561 learned_addr learned my addr 172.21.15.70:0/2882312561 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa96239c700 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa96139a700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9480a15f0 0x7fa9480a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 -- 172.21.15.70:0/2882312561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 msgr2=0x55b702151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 -- 172.21.15.70:0/2882312561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 msgr2=0x55b702151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa961b9b700 1 -- 172.21.15.70:0/2882312561 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b702061470 con 0x55b702156c60 2023-10-18T12:37:53.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.042+0000 7fa96239c700 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.043+0000 7fa961b9b700 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702157040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa958009d60 tx=0x7fa958011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7ded6d98e74f38 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.043+0000 7fa96139a700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9480a15f0 0x7fa9480a3ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa95c19e7a0 tx=0x7fa94c005e20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.043+0000 7fa947fff700 1 -- 172.21.15.70:0/2882312561 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958019630 con 0x55b702156c60 2023-10-18T12:37:53.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.043+0000 7fa947fff700 1 -- 172.21.15.70:0/2882312561 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958019c50 con 0x55b702156c60 2023-10-18T12:37:53.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.043+0000 7fa947fff700 1 -- 172.21.15.70:0/2882312561 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ebb0 con 0x55b702156c60 2023-10-18T12:37:53.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2882312561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 msgr2=0x55b702157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702157040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa958009d60 tx=0x7fa958011720 comp rx=0 tx=0).stop 2023-10-18T12:37:53.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2882312561 shutdown_connections 2023-10-18T12:37:53.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702157040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2882312561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2882312561 >> 172.21.15.70:0/2882312561 conn(0x55b7020c57a0 msgr2=0x55b70214bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2882312561 shutdown_connections 2023-10-18T12:37:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2882312561 wait complete. 2023-10-18T12:37:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.044+0000 7fa9776e4b80 1 -- start start 2023-10-18T12:37:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.045+0000 7fa9776e4b80 1 Processor -- start 2023-10-18T12:37:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.045+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b70215d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.045+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b70215a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.045+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7020a3ef0 con 0x55b702151650 2023-10-18T12:37:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702040790 con 0x55b702156c60 2023-10-18T12:37:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702040430 con 0x55b702157580 2023-10-18T12:37:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa9776e4b80 10 client.0 _ll_register_callbacks cb 0x55b701f8ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b70215d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b70215d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35936/0 (socket says 172.21.15.70:35936) 2023-10-18T12:37:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa96139a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b70215a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 -- 172.21.15.70:0/1594983040 learned_addr learned my addr 172.21.15.70:0/1594983040 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa96239c700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 -- 172.21.15.70:0/1594983040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 msgr2=0x55b70215afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 -- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 msgr2=0x55b70215a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b70215a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa961b9b700 1 -- 172.21.15.70:0/1594983040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1a9fc0 con 0x55b702151650 2023-10-18T12:37:53.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.046+0000 7fa96139a700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b70215a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa961b9b700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b70215d5d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa95801c040 tx=0x7fa958009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feb31d0347a8a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9457fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9457fa700 1 -- 172.21.15.70:0/1594983040 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ed10 con 0x55b702151650 2023-10-18T12:37:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9457fa700 1 -- 172.21.15.70:0/1594983040 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa95800ee70 con 0x55b702151650 2023-10-18T12:37:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7020a8f70 con 0x55b702151650 2023-10-18T12:37:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9457fa700 1 -- 172.21.15.70:0/1594983040 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa958004030 con 0x55b702151650 2023-10-18T12:37:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9776e4b80 10 client.17193 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa9580099c0 con 0x55b702151650 2023-10-18T12:37:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9457fa700 1 -- 172.21.15.70:0/1594983040 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7020a8f70 con 0x55b702151650 2023-10-18T12:37:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.047+0000 7fa9457fa700 1 -- 172.21.15.70:0/1594983040 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa95802c060 con 0x55b702151650 2023-10-18T12:37:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9776e4b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9776e4b80 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9776e4b80 20 client.17193 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b70216aea0 0x55b70216d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9776e4b80 4 client.17193 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b702120930 con 0x55b70216aea0 2023-10-18T12:37:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa96139a700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b70216aea0 0x55b70216d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa96139a700 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b70216aea0 0x55b70216d290 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310153bc583c5f8e server_cookie=ddb3aea3885199a9 in_seq=0 out_seq=0 2023-10-18T12:37:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.048+0000 7fa9457fa700 10 client.17193 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.049+0000 7fa9457fa700 1 -- 172.21.15.70:0/1594983040 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b702120930 con 0x55b70216aea0 2023-10-18T12:37:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.049+0000 7fa9457fa700 10 client.17193 handle_command_reply: tid=0 2023-10-18T12:37:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.049+0000 7fa9776e4b80 1 client.17193 shutdown 2023-10-18T12:37:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 msgr2=0x55b70215d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b70215d5d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa95801c040 tx=0x7fa958009800 comp rx=0 tx=0).stop 2023-10-18T12:37:53.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 shutdown_connections 2023-10-18T12:37:53.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b70216aea0 0x55b70216d290 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9457fa700 0 client.17193 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b70215d5d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/1594983040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b70215a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 >> 172.21.15.70:0/1594983040 conn(0x55b7020c57a0 msgr2=0x55b702150100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 shutdown_connections 2023-10-18T12:37:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 1 -- 172.21.15.70:0/1594983040 wait complete. 2023-10-18T12:37:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.050+0000 7fa9776e4b80 20 client.17193 trim_cache size 0 max 16384 2023-10-18T12:37:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 Processor -- start 2023-10-18T12:37:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 -- start start 2023-10-18T12:37:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702161f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b702154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b702146b70 con 0x55b702151650 2023-10-18T12:37:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b701f249a0 con 0x55b702156c60 2023-10-18T12:37:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.067+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70215be90 con 0x55b702157580 2023-10-18T12:37:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b702154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b702154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35018/0 (socket says 172.21.15.70:35018) 2023-10-18T12:37:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 -- 172.21.15.70:0/2459208642 learned_addr learned my addr 172.21.15.70:0/2459208642 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa961b9b700 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96239c700 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 -- 172.21.15.70:0/2459208642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 msgr2=0x55b70215d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 -- 172.21.15.70:0/2459208642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 msgr2=0x55b702161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702161f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 -- 172.21.15.70:0/2459208642 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b702171ee0 con 0x55b702156c60 2023-10-18T12:37:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa96139a700 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b702154e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005c10 tx=0x7fa94c019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6584b52d41141662 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.068+0000 7fa950bff700 1 -- 172.21.15.70:0/2459208642 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c02d040 con 0x55b702156c60 2023-10-18T12:37:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa950bff700 1 -- 172.21.15.70:0/2459208642 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa94c032900 con 0x55b702156c60 2023-10-18T12:37:53.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa950bff700 1 -- 172.21.15.70:0/2459208642 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0038e0 con 0x55b702156c60 2023-10-18T12:37:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2459208642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 msgr2=0x55b702154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b702154e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005c10 tx=0x7fa94c019040 comp rx=0 tx=0).stop 2023-10-18T12:37:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2459208642 shutdown_connections 2023-10-18T12:37:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702157580 0x55b70215d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b702161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/2459208642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702156c60 0x55b702154e40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2459208642 >> 172.21.15.70:0/2459208642 conn(0x55b7020c57a0 msgr2=0x55b702154730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2459208642 shutdown_connections 2023-10-18T12:37:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.069+0000 7fa9776e4b80 1 -- 172.21.15.70:0/2459208642 wait complete. 2023-10-18T12:37:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.070+0000 7fa9776e4b80 1 -- start start 2023-10-18T12:37:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.070+0000 7fa9776e4b80 1 Processor -- start 2023-10-18T12:37:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.070+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b7021606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9776e4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7021bee50 con 0x55b702151650 2023-10-18T12:37:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7021befc0 con 0x55b702157580 2023-10-18T12:37:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9776e4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7021bf180 con 0x55b702156c60 2023-10-18T12:37:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9776e4b80 10 client.0 _ll_register_callbacks cb 0x55b701f8ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b7021606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa96239c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa96239c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35034/0 (socket says 172.21.15.70:35034) 2023-10-18T12:37:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa96239c700 1 -- 172.21.15.70:0/3757212031 learned_addr learned my addr 172.21.15.70:0/3757212031 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa96139a700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 -- 172.21.15.70:0/3757212031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 msgr2=0x55b702160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 -- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 msgr2=0x55b702161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 -- 172.21.15.70:0/3757212031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c01c070 con 0x55b702151650 2023-10-18T12:37:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa96239c700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa961b9b700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b7021606d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa958005170 tx=0x7fa958009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74902ab37787f232 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.071+0000 7fa9457fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9457fa700 1 -- 172.21.15.70:0/3757212031 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ed10 con 0x55b702151650 2023-10-18T12:37:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa958019c50 con 0x55b702151650 2023-10-18T12:37:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9457fa700 1 -- 172.21.15.70:0/3757212031 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa958000e40 con 0x55b702151650 2023-10-18T12:37:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9457fa700 1 -- 172.21.15.70:0/3757212031 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa95800ee70 con 0x55b702151650 2023-10-18T12:37:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 10 client.17199 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b701ecee60 con 0x55b702151650 2023-10-18T12:37:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9457fa700 1 -- 172.21.15.70:0/3757212031 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958019c50 con 0x55b702151650 2023-10-18T12:37:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9457fa700 1 -- 172.21.15.70:0/3757212031 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa958011790 con 0x55b702151650 2023-10-18T12:37:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 10 client.17199 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 20 client.17199 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.072+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b7021c0550 0x55b7021c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.073+0000 7fa9776e4b80 4 client.17199 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:37:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.073+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b70209ced0 con 0x55b7021c0550 2023-10-18T12:37:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.073+0000 7fa96139a700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b7021c0550 0x55b7021c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.073+0000 7fa9457fa700 10 client.17199 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.073+0000 7fa96139a700 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b7021c0550 0x55b7021c2940 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94f7774fb1cd637 server_cookie=58a0a81654757b83 in_seq=0 out_seq=0 2023-10-18T12:37:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9457fa700 1 -- 172.21.15.70:0/3757212031 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b70209ced0 con 0x55b7021c0550 2023-10-18T12:37:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9457fa700 10 client.17199 handle_command_reply: tid=0 2023-10-18T12:37:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 client.17199 shutdown 2023-10-18T12:37:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 msgr2=0x55b7021606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b7021606d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa958005170 tx=0x7fa958009f70 comp rx=0 tx=0).stop 2023-10-18T12:37:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 shutdown_connections 2023-10-18T12:37:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55b7021c0550 0x55b7021c2940 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b702157580 0x55b702161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b702151650 0x55b7021606d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9457fa700 0 client.17199 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 --2- 172.21.15.70:0/3757212031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b702156c60 0x55b702160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 >> 172.21.15.70:0/3757212031 conn(0x55b7020c57a0 msgr2=0x55b702154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 shutdown_connections 2023-10-18T12:37:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 1 -- 172.21.15.70:0/3757212031 wait complete. 2023-10-18T12:37:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.075+0000 7fa9776e4b80 20 client.17199 trim_cache size 0 max 16384 2023-10-18T12:37:53.091 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:53.091 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:37:53.092 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "06e2a868-3f76-48c6-a4ac-d3dc0a6d85f3", 2023-10-18T12:37:53.092 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:37:53.092 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9480a15f0 msgr2=0x7fa9480a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9480a15f0 0x7fa9480a3ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa95c19e7a0 tx=0x7fa94c005e20 comp rx=0 tx=0).stop 2023-10-18T12:37:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 msgr2=0x7fa95c1a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 0x7fa95c1a1b30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ef90 tx=0x7fa95400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 shutdown_connections 2023-10-18T12:37:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa95c111720 0x7fa95c1a1b30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa95c072b90 0x7fa95c19d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa9480a15f0 0x7fa9480a3ab0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 --2- 172.21.15.70:0/1201784954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95c0734b0 0x7fa95c19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 >> 172.21.15.70:0/1201784954 conn(0x7fa95c0fc200 msgr2=0x7fa95c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 shutdown_connections 2023-10-18T12:37:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.076+0000 7fa963dff700 1 -- 172.21.15.70:0/1201784954 wait complete. 2023-10-18T12:37:53.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:37:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 -- 172.21.15.70:0/1380167896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c1010e0 msgr2=0x7f6d8c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 --2- 172.21.15.70:0/1380167896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c1010e0 0x7f6d8c10d5f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b0a0 tx=0x7f6d8400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 -- 172.21.15.70:0/1380167896 shutdown_connections 2023-10-18T12:37:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 --2- 172.21.15.70:0/1380167896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c1010e0 0x7f6d8c10d5f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 --2- 172.21.15.70:0/1380167896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c106730 0x7f6d8c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 --2- 172.21.15.70:0/1380167896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8c100720 0x7f6d8c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 -- 172.21.15.70:0/1380167896 >> 172.21.15.70:0/1380167896 conn(0x7f6d8c0fc1b0 msgr2=0x7f6d8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 -- 172.21.15.70:0/1380167896 shutdown_connections 2023-10-18T12:37:53.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.314+0000 7f6d937ff700 1 -- 172.21.15.70:0/1380167896 wait complete. 2023-10-18T12:37:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 Processor -- start 2023-10-18T12:37:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 -- start start 2023-10-18T12:37:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 0x7f6d8c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8c1010e0 0x7f6d8c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c106730 0x7f6d8c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19dc90 con 0x7f6d8c100720 2023-10-18T12:37:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19de00 con 0x7f6d8c106730 2023-10-18T12:37:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.315+0000 7f6d937ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19dfc0 con 0x7f6d8c1010e0 2023-10-18T12:37:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 0x7f6d8c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 0x7f6d8c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35982/0 (socket says 172.21.15.70:35982) 2023-10-18T12:37:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 -- 172.21.15.70:0/3068029501 learned_addr learned my addr 172.21.15.70:0/3068029501 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c106730 0x7f6d8c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8c1010e0 0x7f6d8c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 -- 172.21.15.70:0/3068029501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8c1010e0 msgr2=0x7f6d8c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8c1010e0 0x7f6d8c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 -- 172.21.15.70:0/3068029501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c106730 msgr2=0x7f6d8c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c106730 0x7f6d8c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 -- 172.21.15.70:0/3068029501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84009d00 con 0x7f6d8c100720 2023-10-18T12:37:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 0x7f6d8c198d70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e9a0 tx=0x7f6d8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d827fc700 1 -- 172.21.15.70:0/3068029501 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88009790 con 0x7f6d8c100720 2023-10-18T12:37:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d827fc700 1 -- 172.21.15.70:0/3068029501 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d88010430 con 0x7f6d8c100720 2023-10-18T12:37:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c19e2a0 con 0x7f6d8c100720 2023-10-18T12:37:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.316+0000 7f6d827fc700 1 -- 172.21.15.70:0/3068029501 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88018680 con 0x7f6d8c100720 2023-10-18T12:37:53.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.317+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c1a5ae0 con 0x7f6d8c100720 2023-10-18T12:37:53.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.320+0000 7f6d827fc700 1 -- 172.21.15.70:0/3068029501 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6d880105a0 con 0x7f6d8c100720 2023-10-18T12:37:53.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.321+0000 7f6d827fc700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6d780a15f0 0x7f6d780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.321+0000 7f6da70cab80 1 Processor -- start 2023-10-18T12:37:53.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d827fc700 1 -- 172.21.15.70:0/3068029501 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f6d88014070 con 0x7f6d8c100720 2023-10-18T12:37:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 -- start start 2023-10-18T12:37:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6d780a15f0 0x7f6d780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07103b6e0 con 0x55e0710ce500 2023-10-18T12:37:53.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e071021c60 con 0x55e0710c85d0 2023-10-18T12:37:53.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6da70cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07101f580 con 0x55e0710cdbe0 2023-10-18T12:37:53.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41968/0 (socket says 172.21.15.70:41968) 2023-10-18T12:37:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d9159b700 1 -- 172.21.15.70:0/3233106965 learned_addr learned my addr 172.21.15.70:0/3233106965 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.322+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d9159b700 1 -- 172.21.15.70:0/3233106965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 msgr2=0x55e0710c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d9159b700 1 -- 172.21.15.70:0/3233106965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 msgr2=0x55e0710c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d9159b700 1 -- 172.21.15.70:0/3233106965 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e070fd83f0 con 0x55e0710cdbe0 2023-10-18T12:37:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6d780a15f0 0x7f6d780a3ab0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c19a2c0 tx=0x7f6d7c011720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710cdfc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d88096cb0 tx=0x7f6d88099c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c78022fa3cf36764 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d777fe700 1 -- 172.21.15.70:0/3233106965 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8809e390 con 0x55e0710cdbe0 2023-10-18T12:37:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d777fe700 1 -- 172.21.15.70:0/3233106965 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d880a25d0 con 0x55e0710cdbe0 2023-10-18T12:37:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.323+0000 7f6d777fe700 1 -- 172.21.15.70:0/3233106965 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8809a4a0 con 0x55e0710cdbe0 2023-10-18T12:37:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 -- 172.21.15.70:0/3233106965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 msgr2=0x55e0710cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710cdfc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d88096cb0 tx=0x7f6d88099c90 comp rx=0 tx=0).stop 2023-10-18T12:37:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 -- 172.21.15.70:0/3233106965 shutdown_connections 2023-10-18T12:37:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710cdfc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3233106965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 -- 172.21.15.70:0/3233106965 >> 172.21.15.70:0/3233106965 conn(0x55e07103c720 msgr2=0x55e0710c2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 -- 172.21.15.70:0/3233106965 shutdown_connections 2023-10-18T12:37:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 -- 172.21.15.70:0/3233106965 wait complete. 2023-10-18T12:37:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.324+0000 7f6da70cab80 1 -- start start 2023-10-18T12:37:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 Processor -- start 2023-10-18T12:37:53.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07101ae70 con 0x55e0710ce500 2023-10-18T12:37:53.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e070fb7710 con 0x55e0710c85d0 2023-10-18T12:37:53.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e070fb73b0 con 0x55e0710cdbe0 2023-10-18T12:37:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6da70cab80 10 client.0 _ll_register_callbacks cb 0x55e070f01a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6d90d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.325+0000 7f6d91d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35072/0 (socket says 172.21.15.70:35072) 2023-10-18T12:37:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 -- 172.21.15.70:0/3534223761 learned_addr learned my addr 172.21.15.70:0/3534223761 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 -- 172.21.15.70:0/3534223761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 msgr2=0x55e0710d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 -- 172.21.15.70:0/3534223761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 msgr2=0x55e0710d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 -- 172.21.15.70:0/3534223761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c1a5ae0 con 0x55e0710c85d0 2023-10-18T12:37:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710d1840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d88098590 tx=0x7f6d880ca9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9133dec5847964 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3534223761 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8809e410 con 0x55e0710c85d0 2023-10-18T12:37:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e07101fef0 con 0x55e0710c85d0 2023-10-18T12:37:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.326+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3534223761 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f6d880b82a0 con 0x55e0710c85d0 2023-10-18T12:37:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3534223761 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d880b8460 con 0x55e0710c85d0 2023-10-18T12:37:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 10 client.27263 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:53.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e070e45de0 con 0x55e0710c85d0 2023-10-18T12:37:53.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3534223761 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e07101fef0 con 0x55e0710c85d0 2023-10-18T12:37:53.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3534223761 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6d880a2140 con 0x55e0710c85d0 2023-10-18T12:37:53.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 10 client.27263 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:53.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:53.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 20 client.27263 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:53.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e0710e1b10 0x55e0710e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 4 client.27263 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:53.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0710978b0 con 0x55e0710e1b10 2023-10-18T12:37:53.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.327+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e0710e1b10 0x55e0710e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.328+0000 7f6d74ff9700 10 client.27263 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.328+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e0710e1b10 0x55e0710e3f00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce3cbe52b312931 server_cookie=ed9a1e2e8c024acc in_seq=0 out_seq=0 2023-10-18T12:37:53.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3534223761 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e0710978b0 con 0x55e0710e1b10 2023-10-18T12:37:53.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6d74ff9700 10 client.27263 handle_command_reply: tid=0 2023-10-18T12:37:53.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 client.27263 shutdown 2023-10-18T12:37:53.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 msgr2=0x55e0710d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710d1840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d88098590 tx=0x7f6d880ca9c0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 shutdown_connections 2023-10-18T12:37:53.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e0710e1b10 0x55e0710e3f00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710ce500 0x55e0710d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710c85d0 0x55e0710d1840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3534223761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6d74ff9700 0 client.27263 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 >> 172.21.15.70:0/3534223761 conn(0x55e07103c720 msgr2=0x55e0710c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 shutdown_connections 2023-10-18T12:37:53.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 1 -- 172.21.15.70:0/3534223761 wait complete. 2023-10-18T12:37:53.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.329+0000 7f6da70cab80 20 client.27263 trim_cache size 0 max 16384 2023-10-18T12:37:53.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 Processor -- start 2023-10-18T12:37:53.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 -- start start 2023-10-18T12:37:53.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e0710cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0710bdaf0 con 0x55e0710c85d0 2023-10-18T12:37:53.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e070e9b920 con 0x55e0710ce500 2023-10-18T12:37:53.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.348+0000 7f6da70cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0710c43b0 con 0x55e0710cdbe0 2023-10-18T12:37:53.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d9159b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e0710cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d90d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e0710cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35082/0 (socket says 172.21.15.70:35082) 2023-10-18T12:37:53.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 -- 172.21.15.70:0/179731264 learned_addr learned my addr 172.21.15.70:0/179731264 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 -- 172.21.15.70:0/179731264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 msgr2=0x55e0710d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 -- 172.21.15.70:0/179731264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 msgr2=0x55e0710d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 -- 172.21.15.70:0/179731264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e070e2ec60 con 0x55e0710ce500 2023-10-18T12:37:53.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d90d9a700 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d9159b700 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:53.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e0710cb6b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d84015000 tx=0x7f6d8400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd919cb74f4c032f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d77fff700 1 -- 172.21.15.70:0/179731264 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400e040 con 0x55e0710ce500 2023-10-18T12:37:53.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d77fff700 1 -- 172.21.15.70:0/179731264 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d84007e90 con 0x55e0710ce500 2023-10-18T12:37:53.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.349+0000 7f6d77fff700 1 -- 172.21.15.70:0/179731264 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84005330 con 0x55e0710ce500 2023-10-18T12:37:53.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 -- 172.21.15.70:0/179731264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 msgr2=0x55e0710cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e0710cb6b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d84015000 tx=0x7f6d8400bbc0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 -- 172.21.15.70:0/179731264 shutdown_connections 2023-10-18T12:37:53.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e0710cb6b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 --2- 172.21.15.70:0/179731264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 -- 172.21.15.70:0/179731264 >> 172.21.15.70:0/179731264 conn(0x55e07103c720 msgr2=0x55e0710c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 -- 172.21.15.70:0/179731264 shutdown_connections 2023-10-18T12:37:53.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 -- 172.21.15.70:0/179731264 wait complete. 2023-10-18T12:37:53.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.350+0000 7f6da70cab80 1 -- start start 2023-10-18T12:37:53.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 Processor -- start 2023-10-18T12:37:53.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e07112e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07112ef20 con 0x55e0710c85d0 2023-10-18T12:37:53.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07112f090 con 0x55e0710ce500 2023-10-18T12:37:53.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07112f250 con 0x55e0710cdbe0 2023-10-18T12:37:53.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6da70cab80 10 client.0 _ll_register_callbacks cb 0x55e070f01a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:53.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6d91d9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e07112e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6d90d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6d90d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:41988/0 (socket says 172.21.15.70:41988) 2023-10-18T12:37:53.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6d90d9a700 1 -- 172.21.15.70:0/3704504813 learned_addr learned my addr 172.21.15.70:0/3704504813 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.351+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d91d9c700 1 -- 172.21.15.70:0/3704504813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 msgr2=0x55e0710d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d91d9c700 1 -- 172.21.15.70:0/3704504813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 msgr2=0x55e0710d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d91d9c700 1 -- 172.21.15.70:0/3704504813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e070e45de0 con 0x55e0710ce500 2023-10-18T12:37:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d91d9c700 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e07112e950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b780 tx=0x7f6d8400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71211e99a63d2128 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3704504813 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400e070 con 0x55e0710ce500 2023-10-18T12:37:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0710c9040 con 0x55e0710ce500 2023-10-18T12:37:53.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3704504813 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f6d84005240 con 0x55e0710ce500 2023-10-18T12:37:53.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3704504813 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d84005400 con 0x55e0710ce500 2023-10-18T12:37:53.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6da70cab80 10 client.27269 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:53.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.352+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f6d7c01ebe0 con 0x55e0710ce500 2023-10-18T12:37:53.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3704504813 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0710c9040 con 0x55e0710ce500 2023-10-18T12:37:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3704504813 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6d8400b0d0 con 0x55e0710ce500 2023-10-18T12:37:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6da70cab80 10 client.27269 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6da70cab80 10 client.27269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6da70cab80 20 client.27269 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e071136830 0x55e071138c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6da70cab80 4 client.27269 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:37:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f6d880a25d0 con 0x55e071136830 2023-10-18T12:37:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.353+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e071136830 0x55e071138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.354+0000 7f6d9159b700 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e071136830 0x55e071138c50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4670aae8040a81c server_cookie=2a64ac16c529f0c9 in_seq=0 out_seq=0 2023-10-18T12:37:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.354+0000 7f6d74ff9700 10 client.27269 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6d74ff9700 1 -- 172.21.15.70:0/3704504813 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f6d880a25d0 con 0x55e071136830 2023-10-18T12:37:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6d74ff9700 10 client.27269 handle_command_reply: tid=0 2023-10-18T12:37:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 client.27269 shutdown 2023-10-18T12:37:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 msgr2=0x55e07112e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e07112e950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b780 tx=0x7f6d8400bf90 comp rx=0 tx=0).stop 2023-10-18T12:37:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 shutdown_connections 2023-10-18T12:37:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55e071136830 0x55e071138c50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e0710ce500 0x55e07112e950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e0710c85d0 0x55e0710d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 --2- 172.21.15.70:0/3704504813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e0710cdbe0 0x55e0710d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6d74ff9700 0 client.27269 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 >> 172.21.15.70:0/3704504813 conn(0x55e07103c720 msgr2=0x55e0710cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 shutdown_connections 2023-10-18T12:37:53.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.355+0000 7f6da70cab80 1 -- 172.21.15.70:0/3704504813 wait complete. 2023-10-18T12:37:53.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6da70cab80 20 client.27269 trim_cache size 0 max 16384 2023-10-18T12:37:53.372 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:53.372 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:37:53.373 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:37:53.373 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6d780a15f0 msgr2=0x7f6d780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6d780a15f0 0x7f6d780a3ab0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c19a2c0 tx=0x7f6d7c011720 comp rx=0 tx=0).stop 2023-10-18T12:37:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 msgr2=0x7f6d8c198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 0x7f6d8c198d70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e9a0 tx=0x7f6d8800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 shutdown_connections 2023-10-18T12:37:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6d780a15f0 0x7f6d780a3ab0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c106730 0x7f6d8c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d8c100720 0x7f6d8c198d70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 --2- 172.21.15.70:0/3068029501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8c1010e0 0x7f6d8c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 >> 172.21.15.70:0/3068029501 conn(0x7f6d8c0fc1b0 msgr2=0x7f6d8c10c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 shutdown_connections 2023-10-18T12:37:53.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.356+0000 7f6d937ff700 1 -- 172.21.15.70:0/3068029501 wait complete. 2023-10-18T12:37:53.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:37:53.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:37:53.439 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:53 smithi070 ceph-mon[98639]: pgmap v680: 129 pgs: 129 active+clean; 618 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2023-10-18T12:37:53.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 -- 172.21.15.70:0/2405863129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b4101060 msgr2=0x7f96b410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2405863129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b4101060 0x7f96b410d570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00b0a0 tx=0x7f96ac00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 -- 172.21.15.70:0/2405863129 shutdown_connections 2023-10-18T12:37:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2405863129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b4101060 0x7f96b410d570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2405863129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b41066b0 0x7f96b4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2405863129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b41006a0 0x7f96b4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.578+0000 7f96bbbff700 1 -- 172.21.15.70:0/2405863129 >> 172.21.15.70:0/2405863129 conn(0x7f96b40fc170 msgr2=0x7f96b40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.579+0000 7f96bbbff700 1 -- 172.21.15.70:0/2405863129 shutdown_connections 2023-10-18T12:37:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.579+0000 7f96bbbff700 1 -- 172.21.15.70:0/2405863129 wait complete. 2023-10-18T12:37:53.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 Processor -- start 2023-10-18T12:37:53.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 -- start start 2023-10-18T12:37:53.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b41006a0 0x7f96b419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b4101060 0x7f96b419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b41066b0 0x7f96b4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4199260 con 0x7f96b4101060 2023-10-18T12:37:53.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b41993d0 con 0x7f96b41066b0 2023-10-18T12:37:53.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96bbbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4199590 con 0x7f96b41006a0 2023-10-18T12:37:53.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96b919a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b4101060 0x7f96b419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96ba19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b41066b0 0x7f96b4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96b919a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b4101060 0x7f96b419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36046/0 (socket says 172.21.15.70:36046) 2023-10-18T12:37:53.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.580+0000 7f96b919a700 1 -- 172.21.15.70:0/2601813580 learned_addr learned my addr 172.21.15.70:0/2601813580 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:53.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96b999b700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b41006a0 0x7f96b419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96ba19c700 1 -- 172.21.15.70:0/2601813580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b41006a0 msgr2=0x7f96b419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96ba19c700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b41006a0 0x7f96b419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96ba19c700 1 -- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b4101060 msgr2=0x7f96b419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96ba19c700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b4101060 0x7f96b419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96ba19c700 1 -- 172.21.15.70:0/2601813580 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0009520 con 0x7f96b41066b0 2023-10-18T12:37:53.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96ba19c700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b41066b0 0x7f96b4198a90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00b070 tx=0x7f96ac009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00e040 con 0x7f96b41066b0 2023-10-18T12:37:53.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96ac007990 con 0x7f96b41066b0 2023-10-18T12:37:53.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac005040 con 0x7f96b41066b0 2023-10-18T12:37:53.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac009d00 con 0x7f96b41066b0 2023-10-18T12:37:53.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.581+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b41a5790 con 0x7f96b41066b0 2023-10-18T12:37:53.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.582+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b4101de0 con 0x7f96b41066b0 2023-10-18T12:37:53.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.584+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f96ac01d470 con 0x7f96b41066b0 2023-10-18T12:37:53.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.586+0000 7f96aaffd700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f96a00a15f0 0x7f96a00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:53.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.586+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f96ac0cee90 con 0x7f96b41066b0 2023-10-18T12:37:53.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.586+0000 7f96b999b700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f96a00a15f0 0x7f96a00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:53.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.586+0000 7f96b999b700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f96a00a15f0 0x7f96a00a3ab0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009d70 tx=0x7f96b0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:53.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.588+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f96ac097e80 con 0x7f96b41066b0 2023-10-18T12:37:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:53 smithi152 ceph-mon[134767]: pgmap v680: 129 pgs: 129 active+clean; 618 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2023-10-18T12:37:53.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:53 smithi186 ceph-mon[135253]: pgmap v680: 129 pgs: 129 active+clean; 618 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 200 op/s 2023-10-18T12:37:53.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.846+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96b4067530 con 0x7f96b41066b0 2023-10-18T12:37:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.847+0000 7f96aaffd700 1 -- 172.21.15.70:0/2601813580 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f96ac09b4a0 con 0x7f96b41066b0 2023-10-18T12:37:53.850 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:37:53.850 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:37:53.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f96a00a15f0 msgr2=0x7f96a00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f96a00a15f0 0x7f96a00a3ab0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009d70 tx=0x7f96b0009260 comp rx=0 tx=0).stop 2023-10-18T12:37:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b41066b0 msgr2=0x7f96b4198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b41066b0 0x7f96b4198a90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00b070 tx=0x7f96ac009c60 comp rx=0 tx=0).stop 2023-10-18T12:37:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 shutdown_connections 2023-10-18T12:37:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f96a00a15f0 0x7f96a00a3ab0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b41066b0 0x7f96b4198a90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b41006a0 0x7f96b419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 --2- 172.21.15.70:0/2601813580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b4101060 0x7f96b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 >> 172.21.15.70:0/2601813580 conn(0x7f96b40fc170 msgr2=0x7f96b410b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:53.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 shutdown_connections 2023-10-18T12:37:53.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:53.849+0000 7f96bbbff700 1 -- 172.21.15.70:0/2601813580 wait complete. 2023-10-18T12:37:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:37:53.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:37:53.884 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1336079 11.09 MB/sec execute 502 sec latency 32.051 ms 2023-10-18T12:37:54.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 -- 172.21.15.70:0/368105818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb690100730 msgr2=0x7fb690100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 --2- 172.21.15.70:0/368105818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb690100730 0x7fb690100bb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 -- 172.21.15.70:0/368105818 shutdown_connections 2023-10-18T12:37:54.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 --2- 172.21.15.70:0/368105818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6901010f0 0x7fb69010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 --2- 172.21.15.70:0/368105818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690106740 0x7fb690106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 --2- 172.21.15.70:0/368105818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb690100730 0x7fb690100bb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 -- 172.21.15.70:0/368105818 >> 172.21.15.70:0/368105818 conn(0x7fb6900fc200 msgr2=0x7fb6900fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 -- 172.21.15.70:0/368105818 shutdown_connections 2023-10-18T12:37:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.086+0000 7fb696bff700 1 -- 172.21.15.70:0/368105818 wait complete. 2023-10-18T12:37:54.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.087+0000 7fb696bff700 1 Processor -- start 2023-10-18T12:37:54.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 -- start start 2023-10-18T12:37:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 0x7fb690072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6901010f0 0x7fb690073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb690106740 0x7fb690073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a4f20 con 0x7fb690100730 2023-10-18T12:37:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a5090 con 0x7fb6901010f0 2023-10-18T12:37:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb696bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a5250 con 0x7fb690106740 2023-10-18T12:37:54.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb69499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 0x7fb690072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6901010f0 0x7fb690073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb69499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 0x7fb690072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36050/0 (socket says 172.21.15.70:36050) 2023-10-18T12:37:54.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.088+0000 7fb69499b700 1 -- 172.21.15.70:0/3651893911 learned_addr learned my addr 172.21.15.70:0/3651893911 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:54.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb69519c700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb690106740 0x7fb690073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68ffff700 1 -- 172.21.15.70:0/3651893911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb690106740 msgr2=0x7fb690073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68ffff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb690106740 0x7fb690073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68ffff700 1 -- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 msgr2=0x7fb690072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68ffff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 0x7fb690072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68ffff700 1 -- 172.21.15.70:0/3651893911 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0095f0 con 0x7fb6901010f0 2023-10-18T12:37:54.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb69499b700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 0x7fb690072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:54.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68ffff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6901010f0 0x7fb690073030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0098e0 tx=0x7fb67c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:54.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68dffb700 1 -- 172.21.15.70:0/3651893911 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c01d070 con 0x7fb6901010f0 2023-10-18T12:37:54.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb68dffb700 1 -- 172.21.15.70:0/3651893911 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb67c022470 con 0x7fb6901010f0 2023-10-18T12:37:54.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.089+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6901a54d0 con 0x7fb6901010f0 2023-10-18T12:37:54.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.090+0000 7fb68dffb700 1 -- 172.21.15.70:0/3651893911 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c017650 con 0x7fb6901010f0 2023-10-18T12:37:54.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.090+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6901a5a00 con 0x7fb6901010f0 2023-10-18T12:37:54.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.093+0000 7fb68dffb700 1 -- 172.21.15.70:0/3651893911 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb67c0225e0 con 0x7fb6901010f0 2023-10-18T12:37:54.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 Processor -- start 2023-10-18T12:37:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 -- start start 2023-10-18T12:37:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb68dffb700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6800a1670 0x7fb6800a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb68dffb700 1 -- 172.21.15.70:0/3651893911 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fb67c0ce6f0 con 0x7fb6901010f0 2023-10-18T12:37:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04056fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04051090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04051a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03fc46e0 con 0x55ef04056be0 2023-10-18T12:37:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03faac60 con 0x55ef040515d0 2023-10-18T12:37:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03fa8580 con 0x55ef04057500 2023-10-18T12:37:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.094+0000 7fb69519c700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6800a1670 0x7fb6800a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04051090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb69499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04056fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04051090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42030/0 (socket says 172.21.15.70:42030) 2023-10-18T12:37:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 -- 172.21.15.70:0/2940620634 learned_addr learned my addr 172.21.15.70:0/2940620634 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb69519c700 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04051a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 -- 172.21.15.70:0/2940620634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 msgr2=0x55ef04051a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04051a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 -- 172.21.15.70:0/2940620634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 msgr2=0x55ef04056fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04056fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 -- 172.21.15.70:0/2940620634 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef03f613f0 con 0x55ef04057500 2023-10-18T12:37:54.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb68ffff700 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04051090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb67c09b0a0 tx=0x7fb67c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83cb890fbffadfb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:54.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb69519c700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6800a1670 0x7fb6800a3b30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb68800f870 tx=0x7fb68800fc30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:54.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb67affd700 1 -- 172.21.15.70:0/2940620634 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c09ad00 con 0x55ef04057500 2023-10-18T12:37:54.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.095+0000 7fb67affd700 1 -- 172.21.15.70:0/2940620634 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb67c09baa0 con 0x55ef04057500 2023-10-18T12:37:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb67affd700 1 -- 172.21.15.70:0/2940620634 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0228c0 con 0x55ef04057500 2023-10-18T12:37:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/2940620634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 msgr2=0x55ef04051090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04051090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb67c09b0a0 tx=0x7fb67c009580 comp rx=0 tx=0).stop 2023-10-18T12:37:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/2940620634 shutdown_connections 2023-10-18T12:37:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04051a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04056fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/2940620634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04051090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/2940620634 >> 172.21.15.70:0/2940620634 conn(0x55ef03fc5720 msgr2=0x55ef0404bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/2940620634 shutdown_connections 2023-10-18T12:37:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.096+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/2940620634 wait complete. 2023-10-18T12:37:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.097+0000 7fb6aa4feb80 1 -- start start 2023-10-18T12:37:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.097+0000 7fb6aa4feb80 1 Processor -- start 2023-10-18T12:37:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef0405d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04056be0 0x55ef0405a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04057500 0x55ef0405add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03fa3e70 con 0x55ef04057500 2023-10-18T12:37:54.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03f40710 con 0x55ef040515d0 2023-10-18T12:37:54.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03f403b0 con 0x55ef04056be0 2023-10-18T12:37:54.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb6aa4feb80 10 client.0 _ll_register_callbacks cb 0x55ef03e8aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:54.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef0405d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef0405d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35152/0 (socket says 172.21.15.70:35152) 2023-10-18T12:37:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 -- 172.21.15.70:0/585947471 learned_addr learned my addr 172.21.15.70:0/585947471 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69519c700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04057500 0x55ef0405add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb68ffff700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04056be0 0x55ef0405a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 -- 172.21.15.70:0/585947471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04056be0 msgr2=0x55ef0405a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04056be0 0x55ef0405a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 -- 172.21.15.70:0/585947471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04057500 msgr2=0x55ef0405add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04057500 0x55ef0405add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.098+0000 7fb69499b700 1 -- 172.21.15.70:0/585947471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6901a5a00 con 0x55ef040515d0 2023-10-18T12:37:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb69499b700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef0405d480 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb684011d10 tx=0x7fb68400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99eaf0f5b765492 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb66ffff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb66ffff700 1 -- 172.21.15.70:0/585947471 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684016070 con 0x55ef040515d0 2023-10-18T12:37:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef03fa8ef0 con 0x55ef040515d0 2023-10-18T12:37:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb66ffff700 1 -- 172.21.15.70:0/585947471 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fb68400fd90 con 0x55ef040515d0 2023-10-18T12:37:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb66ffff700 1 -- 172.21.15.70:0/585947471 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb684004830 con 0x55ef040515d0 2023-10-18T12:37:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb6aa4feb80 10 client.27287 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ef03dcede0 con 0x55ef040515d0 2023-10-18T12:37:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb66ffff700 1 -- 172.21.15.70:0/585947471 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef03fa8ef0 con 0x55ef040515d0 2023-10-18T12:37:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.099+0000 7fb66ffff700 1 -- 172.21.15.70:0/585947471 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb684019550 con 0x55ef040515d0 2023-10-18T12:37:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb6aa4feb80 10 client.27287 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb6aa4feb80 10 client.27287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb6aa4feb80 20 client.27287 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:54.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef0406acd0 0x55ef0406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb6aa4feb80 4 client.27287 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:54.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef040208b0 con 0x55ef0406acd0 2023-10-18T12:37:54.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb68ffff700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef0406acd0 0x55ef0406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb68ffff700 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef0406acd0 0x55ef0406d0f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aecefe6e741e13a server_cookie=47ca8028601ed3cc in_seq=0 out_seq=0 2023-10-18T12:37:54.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.100+0000 7fb66ffff700 10 client.27287 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.101+0000 7fb66ffff700 1 -- 172.21.15.70:0/585947471 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ef040208b0 con 0x55ef0406acd0 2023-10-18T12:37:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.101+0000 7fb66ffff700 10 client.27287 handle_command_reply: tid=0 2023-10-18T12:37:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.101+0000 7fb6aa4feb80 1 client.27287 shutdown 2023-10-18T12:37:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 msgr2=0x55ef0405d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef0405d480 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb684011d10 tx=0x7fb68400f6a0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 shutdown_connections 2023-10-18T12:37:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef0406acd0 0x55ef0406d0f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04057500 0x55ef0405add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef0405d480 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb66ffff700 0 client.27287 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/585947471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04056be0 0x55ef0405a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 >> 172.21.15.70:0/585947471 conn(0x55ef03fc5720 msgr2=0x55ef0404bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 shutdown_connections 2023-10-18T12:37:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/585947471 wait complete. 2023-10-18T12:37:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.102+0000 7fb6aa4feb80 20 client.27287 trim_cache size 0 max 16384 2023-10-18T12:37:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.115+0000 7fb6aa4feb80 1 Processor -- start 2023-10-18T12:37:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.115+0000 7fb6aa4feb80 1 -- start start 2023-10-18T12:37:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef040515d0 0x55ef04051090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 0x55ef040465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04059040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef03e24920 con 0x55ef040515d0 2023-10-18T12:37:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef0404d5e0 con 0x55ef04056be0 2023-10-18T12:37:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef04057330 con 0x55ef04057500 2023-10-18T12:37:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb69499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef040515d0 0x55ef04051090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 0x55ef040465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 0x55ef040465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35166/0 (socket says 172.21.15.70:35166) 2023-10-18T12:37:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 -- 172.21.15.70:0/76977376 learned_addr learned my addr 172.21.15.70:0/76977376 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb69519c700 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04059040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 -- 172.21.15.70:0/76977376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 msgr2=0x55ef04059040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04059040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 -- 172.21.15.70:0/76977376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef040515d0 msgr2=0x55ef04051090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef040515d0 0x55ef04051090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb68ffff700 1 -- 172.21.15.70:0/76977376 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef03db7c60 con 0x55ef04056be0 2023-10-18T12:37:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.116+0000 7fb69499b700 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef040515d0 0x55ef04051090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb68ffff700 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 0x55ef040465d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb67c005f50 tx=0x7fb67c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b945afa9370b4ebb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb67b7fe700 1 -- 172.21.15.70:0/76977376 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0a36a0 con 0x55ef04056be0 2023-10-18T12:37:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb67b7fe700 1 -- 172.21.15.70:0/76977376 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb67c09f4f0 con 0x55ef04056be0 2023-10-18T12:37:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb67b7fe700 1 -- 172.21.15.70:0/76977376 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c022990 con 0x55ef04056be0 2023-10-18T12:37:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/76977376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 msgr2=0x55ef040465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 0x55ef040465d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb67c005f50 tx=0x7fb67c01a040 comp rx=0 tx=0).stop 2023-10-18T12:37:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/76977376 shutdown_connections 2023-10-18T12:37:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04059040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef040515d0 0x55ef04051090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/76977376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef04056be0 0x55ef040465d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/76977376 >> 172.21.15.70:0/76977376 conn(0x55ef03fc5720 msgr2=0x55ef04054970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/76977376 shutdown_connections 2023-10-18T12:37:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.117+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/76977376 wait complete. 2023-10-18T12:37:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 -- start start 2023-10-18T12:37:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 Processor -- start 2023-10-18T12:37:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04053c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04052710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04052c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef040541a0 con 0x55ef04056be0 2023-10-18T12:37:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef04054310 con 0x55ef040515d0 2023-10-18T12:37:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef040544d0 con 0x55ef04057500 2023-10-18T12:37:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.118+0000 7fb6aa4feb80 10 client.0 _ll_register_callbacks cb 0x55ef03e8aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04053c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04052710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04053c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35174/0 (socket says 172.21.15.70:35174) 2023-10-18T12:37:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 -- 172.21.15.70:0/3040265000 learned_addr learned my addr 172.21.15.70:0/3040265000 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69519c700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04052c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 -- 172.21.15.70:0/3040265000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 msgr2=0x55ef04052c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04052c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 -- 172.21.15.70:0/3040265000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 msgr2=0x55ef04052710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04052710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 -- 172.21.15.70:0/3040265000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef03dcede0 con 0x55ef040515d0 2023-10-18T12:37:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69519c700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04052c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:37:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb69499b700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04053c20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb68401c000 tx=0x7fb684005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1462bff26cffe55 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb66ffff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb66ffff700 1 -- 172.21.15.70:0/3040265000 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684016040 con 0x55ef040515d0 2023-10-18T12:37:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb66ffff700 1 -- 172.21.15.70:0/3040265000 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fb684019cf0 con 0x55ef040515d0 2023-10-18T12:37:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.119+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb67c09baa0 con 0x55ef040515d0 2023-10-18T12:37:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb66ffff700 1 -- 172.21.15.70:0/3040265000 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6840050b0 con 0x55ef040515d0 2023-10-18T12:37:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 10 client.27293 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fb67c0b1040 con 0x55ef040515d0 2023-10-18T12:37:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb66ffff700 1 -- 172.21.15.70:0/3040265000 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb67c09baa0 con 0x55ef040515d0 2023-10-18T12:37:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb66ffff700 1 -- 172.21.15.70:0/3040265000 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb6840039f0 con 0x55ef040515d0 2023-10-18T12:37:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 10 client.27293 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 10 client.27293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 20 client.27293 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef040bfad0 0x55ef040c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 4 client.27293 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:37:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ef03f9ce50 con 0x55ef040bfad0 2023-10-18T12:37:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.120+0000 7fb68ffff700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef040bfad0 0x55ef040c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.121+0000 7fb66ffff700 10 client.27293 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.121+0000 7fb68ffff700 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef040bfad0 0x55ef040c1ef0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334b3ecfd4e5500a server_cookie=3d07c02d1b95978e in_seq=0 out_seq=0 2023-10-18T12:37:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb66ffff700 1 -- 172.21.15.70:0/3040265000 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ef03f9ce50 con 0x55ef040bfad0 2023-10-18T12:37:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb66ffff700 10 client.27293 handle_command_reply: tid=0 2023-10-18T12:37:54.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 client.27293 shutdown 2023-10-18T12:37:54.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 msgr2=0x55ef04053c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04053c20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb68401c000 tx=0x7fb684005dc0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 shutdown_connections 2023-10-18T12:37:54.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ef040bfad0 0x55ef040c1ef0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef04057500 0x55ef04052c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef040515d0 0x55ef04053c20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 --2- 172.21.15.70:0/3040265000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef04056be0 0x55ef04052710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb66ffff700 0 client.27293 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:54.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 >> 172.21.15.70:0/3040265000 conn(0x55ef03fc5720 msgr2=0x55ef040454b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:54.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 shutdown_connections 2023-10-18T12:37:54.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.122+0000 7fb6aa4feb80 1 -- 172.21.15.70:0/3040265000 wait complete. 2023-10-18T12:37:54.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb6aa4feb80 20 client.27293 trim_cache size 0 max 16384 2023-10-18T12:37:54.145 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:37:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6800a1670 msgr2=0x7fb6800a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6800a1670 0x7fb6800a3b30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb68800f870 tx=0x7fb68800fc30 comp rx=0 tx=0).stop 2023-10-18T12:37:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6901010f0 msgr2=0x7fb690073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6901010f0 0x7fb690073030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0098e0 tx=0x7fb67c005740 comp rx=0 tx=0).stop 2023-10-18T12:37:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 shutdown_connections 2023-10-18T12:37:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb690106740 0x7fb690073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb690100730 0x7fb690072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb6800a1670 0x7fb6800a3b30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 --2- 172.21.15.70:0/3651893911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6901010f0 0x7fb690073030 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 >> 172.21.15.70:0/3651893911 conn(0x7fb6900fc200 msgr2=0x7fb6900fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:54.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 shutdown_connections 2023-10-18T12:37:54.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:54.123+0000 7fb696bff700 1 -- 172.21.15.70:0/3651893911 wait complete. 2023-10-18T12:37:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:54 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:37:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:54 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:54 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:37:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:54 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:54.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:54 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:37:54.607 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:54 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2601813580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:54.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:54 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:37:54.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:54 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:54.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:54 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:37:54.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:54 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:54.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:54 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:37:54.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:54 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2601813580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:54.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:54 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:37:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:54 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:54 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:37:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:54 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:54 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:37:54.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:54 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2601813580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:37:54.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1338439 11.10 MB/sec execute 503 sec latency 46.135 ms 2023-10-18T12:37:55.150 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:37:55.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:37:55.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 -- 172.21.15.70:0/3903509142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 msgr2=0x7f8f4010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3903509142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 0x7f8f4010d5f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800b0a0 tx=0x7f8f3800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 -- 172.21.15.70:0/3903509142 shutdown_connections 2023-10-18T12:37:55.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3903509142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 0x7f8f4010d5f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3903509142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40106730 0x7f8f40106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3903509142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f40100720 0x7f8f40100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 -- 172.21.15.70:0/3903509142 >> 172.21.15.70:0/3903509142 conn(0x7f8f400fc1b0 msgr2=0x7f8f400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 -- 172.21.15.70:0/3903509142 shutdown_connections 2023-10-18T12:37:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.357+0000 7f8f465ff700 1 -- 172.21.15.70:0/3903509142 wait complete. 2023-10-18T12:37:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.358+0000 7f8f465ff700 1 Processor -- start 2023-10-18T12:37:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.358+0000 7f8f465ff700 1 -- start start 2023-10-18T12:37:55.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f465ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 0x7f8f40198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f465ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 0x7f8f401992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f465ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f40106730 0x7f8f4019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f465ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019dcd0 con 0x7f8f40106730 2023-10-18T12:37:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f465ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019de40 con 0x7f8f401010e0 2023-10-18T12:37:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f465ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019e000 con 0x7f8f40100720 2023-10-18T12:37:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 0x7f8f40198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 0x7f8f40198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42086/0 (socket says 172.21.15.70:42086) 2023-10-18T12:37:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f3ffff700 1 -- 172.21.15.70:0/3431867584 learned_addr learned my addr 172.21.15.70:0/3431867584 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 0x7f8f401992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.359+0000 7f8f44b9c700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f40106730 0x7f8f4019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3ffff700 1 -- 172.21.15.70:0/3431867584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 msgr2=0x7f8f401992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 0x7f8f401992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3ffff700 1 -- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f40106730 msgr2=0x7f8f4019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f40106730 0x7f8f4019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3ffff700 1 -- 172.21.15.70:0/3431867584 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009520 con 0x7f8f40100720 2023-10-18T12:37:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 0x7f8f40198db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e9a0 tx=0x7f8f3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3d7fa700 1 -- 172.21.15.70:0/3431867584 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30009790 con 0x7f8f40100720 2023-10-18T12:37:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f38009d00 con 0x7f8f40100720 2023-10-18T12:37:55.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3d7fa700 1 -- 172.21.15.70:0/3431867584 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f30010430 con 0x7f8f40100720 2023-10-18T12:37:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f3d7fa700 1 -- 172.21.15.70:0/3431867584 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30018680 con 0x7f8f40100720 2023-10-18T12:37:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.360+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4019e4d0 con 0x7f8f40100720 2023-10-18T12:37:55.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.365+0000 7f8f3d7fa700 1 -- 172.21.15.70:0/3431867584 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8f300105a0 con 0x7f8f40100720 2023-10-18T12:37:55.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f59ee3b80 1 Processor -- start 2023-10-18T12:37:55.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f59ee3b80 1 -- start start 2023-10-18T12:37:55.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f3d7fa700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f280a15f0 0x7f8f280a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 0x55f1520c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f3d7fa700 1 -- 172.21.15.70:0/3431867584 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f8f30014070 con 0x7f8f40100720 2023-10-18T12:37:55.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.366+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f15202d760 con 0x55f1520c0580 2023-10-18T12:37:55.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f152013ce0 con 0x55f1520bfc60 2023-10-18T12:37:55.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f152011600 con 0x55f1520ba650 2023-10-18T12:37:55.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 0x55f1520c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f44b9c700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f280a15f0 0x7f8f280a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 0x55f1520c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35210/0 (socket says 172.21.15.70:35210) 2023-10-18T12:37:55.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 -- 172.21.15.70:0/805343257 learned_addr learned my addr 172.21.15.70:0/805343257 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f44b9c700 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 -- 172.21.15.70:0/805343257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 msgr2=0x55f1520baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520baad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 -- 172.21.15.70:0/805343257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 msgr2=0x55f1520ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 -- 172.21.15.70:0/805343257 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f151fca470 con 0x55f1520bfc60 2023-10-18T12:37:55.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 0x55f1520c0040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f3009a1d0 tx=0x7f8f30099ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ddf5016088a214e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.367+0000 7f8f44b9c700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f280a15f0 0x7f8f280a3ab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009610 tx=0x7f8f38009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f267fc700 1 -- 172.21.15.70:0/805343257 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3009e390 con 0x55f1520bfc60 2023-10-18T12:37:55.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f267fc700 1 -- 172.21.15.70:0/805343257 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f300a25d0 con 0x55f1520bfc60 2023-10-18T12:37:55.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f267fc700 1 -- 172.21.15.70:0/805343257 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3009a700 con 0x55f1520bfc60 2023-10-18T12:37:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/805343257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 msgr2=0x55f1520c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 0x55f1520c0040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f3009a1d0 tx=0x7f8f30099ca0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/805343257 shutdown_connections 2023-10-18T12:37:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520bfc60 0x55f1520c0040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/805343257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/805343257 >> 172.21.15.70:0/805343257 conn(0x55f15202e7a0 msgr2=0x55f1520b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.368+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/805343257 shutdown_connections 2023-10-18T12:37:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.369+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/805343257 wait complete. 2023-10-18T12:37:55.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.369+0000 7f8f59ee3b80 1 -- start start 2023-10-18T12:37:55.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.369+0000 7f8f59ee3b80 1 Processor -- start 2023-10-18T12:37:55.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520c3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f15200cef0 con 0x55f1520c0580 2023-10-18T12:37:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f151fa9790 con 0x55f1520ba650 2023-10-18T12:37:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f151fa9430 con 0x55f1520bfc60 2023-10-18T12:37:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f59ee3b80 10 client.0 _ll_register_callbacks cb 0x55f151ef3b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35224/0 (socket says 172.21.15.70:35224) 2023-10-18T12:37:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.370+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1394684375 learned_addr learned my addr 172.21.15.70:0/1394684375 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f44b9c700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1394684375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 msgr2=0x55f1520c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1394684375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 msgr2=0x55f1520c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1394684375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4019e4d0 con 0x55f1520ba650 2023-10-18T12:37:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c6560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f30096cb0 tx=0x7f8f3000c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634fdfa294887a96 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1394684375 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30099730 con 0x55f1520ba650 2023-10-18T12:37:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1394684375 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f8f30099890 con 0x55f1520ba650 2023-10-18T12:37:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f152011f70 con 0x55f1520ba650 2023-10-18T12:37:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1394684375 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3009a4a0 con 0x55f1520ba650 2023-10-18T12:37:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f59ee3b80 10 client.27305 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:55.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.371+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55f151e37e60 con 0x55f1520ba650 2023-10-18T12:37:55.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1394684375 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f152011f70 con 0x55f1520ba650 2023-10-18T12:37:55.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1394684375 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8f30096030 con 0x55f1520ba650 2023-10-18T12:37:55.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f59ee3b80 10 client.27305 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:55.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f59ee3b80 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:55.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f59ee3b80 20 client.27305 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:55.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f1520d3a80 0x55f1520d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f59ee3b80 4 client.27305 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:55.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f152089930 con 0x55f1520d3a80 2023-10-18T12:37:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.372+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f1520d3a80 0x55f1520d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.373+0000 7f8f1f7fe700 10 client.27305 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.373+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f1520d3a80 0x55f1520d5e70 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab46c89d3b211142 server_cookie=300c090b0c1bf2c3 in_seq=0 out_seq=0 2023-10-18T12:37:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1394684375 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f152089930 con 0x55f1520d3a80 2023-10-18T12:37:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f1f7fe700 10 client.27305 handle_command_reply: tid=0 2023-10-18T12:37:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 client.27305 shutdown 2023-10-18T12:37:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 msgr2=0x55f1520c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c6560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f30096cb0 tx=0x7f8f3000c4c0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 shutdown_connections 2023-10-18T12:37:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f1520d3a80 0x55f1520d5e70 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f1520c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c6560 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f1f7fe700 0 client.27305 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1394684375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 >> 172.21.15.70:0/1394684375 conn(0x55f15202e7a0 msgr2=0x55f1520b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 shutdown_connections 2023-10-18T12:37:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1394684375 wait complete. 2023-10-18T12:37:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.374+0000 7f8f59ee3b80 20 client.27305 trim_cache size 0 max 16384 2023-10-18T12:37:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.392+0000 7f8f59ee3b80 1 Processor -- start 2023-10-18T12:37:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.392+0000 7f8f59ee3b80 1 -- start start 2023-10-18T12:37:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.392+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.392+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 0x55f1520c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.392+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520c0580 0x55f1520c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.392+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f151e8d9a0 con 0x55f1520bfc60 2023-10-18T12:37:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1520cb4d0 con 0x55f1520c0580 2023-10-18T12:37:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1520d64a0 con 0x55f1520ba650 2023-10-18T12:37:55.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 0x55f1520c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 0x55f1520c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36118/0 (socket says 172.21.15.70:36118) 2023-10-18T12:37:55.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f44b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520c0580 0x55f1520c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3f7fe700 1 -- 172.21.15.70:0/2048224888 learned_addr learned my addr 172.21.15.70:0/2048224888 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 -- 172.21.15.70:0/2048224888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520c0580 msgr2=0x55f1520c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520c0580 0x55f1520c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 -- 172.21.15.70:0/2048224888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 msgr2=0x55f1520c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 0x55f1520c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 -- 172.21.15.70:0/2048224888 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f151e20ce0 con 0x55f1520ba650 2023-10-18T12:37:55.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 0x55f1520c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:55.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.393+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520caf90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f3009e550 tx=0x7f8f300caa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba76054c4ad67a11 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f26ffd700 1 -- 172.21.15.70:0/2048224888 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3009a9c0 con 0x55f1520ba650 2023-10-18T12:37:55.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f26ffd700 1 -- 172.21.15.70:0/2048224888 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f300992c0 con 0x55f1520ba650 2023-10-18T12:37:55.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f26ffd700 1 -- 172.21.15.70:0/2048224888 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30099730 con 0x55f1520ba650 2023-10-18T12:37:55.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/2048224888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 msgr2=0x55f1520caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520caf90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f3009e550 tx=0x7f8f300caa80 comp rx=0 tx=0).stop 2023-10-18T12:37:55.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/2048224888 shutdown_connections 2023-10-18T12:37:55.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520c0580 0x55f1520c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520ba650 0x55f1520caf90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/2048224888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520bfc60 0x55f1520c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/2048224888 >> 172.21.15.70:0/2048224888 conn(0x55f15202e7a0 msgr2=0x55f1520b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/2048224888 shutdown_connections 2023-10-18T12:37:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.394+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/2048224888 wait complete. 2023-10-18T12:37:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 -- start start 2023-10-18T12:37:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 Processor -- start 2023-10-18T12:37:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f152127820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f152127d60 con 0x55f1520c0580 2023-10-18T12:37:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f152127ea0 con 0x55f1520ba650 2023-10-18T12:37:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f152127fe0 con 0x55f1520bfc60 2023-10-18T12:37:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.395+0000 7f8f59ee3b80 10 client.0 _ll_register_callbacks cb 0x55f151ef3b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f44b9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f152127820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35242/0 (socket says 172.21.15.70:35242) 2023-10-18T12:37:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1866319250 learned_addr learned my addr 172.21.15.70:0/1866319250 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1866319250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 msgr2=0x55f1520ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1866319250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 msgr2=0x55f152127820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f152127820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 -- 172.21.15.70:0/1866319250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f34009bf0 con 0x55f1520ba650 2023-10-18T12:37:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f3ffff700 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c9d30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f30098590 tx=0x7f8f30010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4371814813f88297 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1866319250 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3009a9c0 con 0x55f1520ba650 2023-10-18T12:37:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f300a25d0 con 0x55f1520ba650 2023-10-18T12:37:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.396+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1866319250 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f8f30099520 con 0x55f1520ba650 2023-10-18T12:37:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1866319250 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3009a4a0 con 0x55f1520ba650 2023-10-18T12:37:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 10 client.27311 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55f151e37e60 con 0x55f1520ba650 2023-10-18T12:37:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1866319250 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f300a25d0 con 0x55f1520ba650 2023-10-18T12:37:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1866319250 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8f3009f240 con 0x55f1520ba650 2023-10-18T12:37:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 10 client.27311 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 10 client.27311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 20 client.27311 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f152129270 0x55f15212b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 4 client.27311 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:37:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f152005ed0 con 0x55f152129270 2023-10-18T12:37:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.397+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f152129270 0x55f15212b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.398+0000 7f8f1f7fe700 10 client.27311 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.398+0000 7f8f3f7fe700 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f152129270 0x55f15212b660 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd880788053028a0 server_cookie=2c29b40a9b301b22 in_seq=0 out_seq=0 2023-10-18T12:37:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f1f7fe700 1 -- 172.21.15.70:0/1866319250 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f152005ed0 con 0x55f152129270 2023-10-18T12:37:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f1f7fe700 10 client.27311 handle_command_reply: tid=0 2023-10-18T12:37:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 client.27311 shutdown 2023-10-18T12:37:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 msgr2=0x55f1520c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c9d30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f30098590 tx=0x7f8f30010ab0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 shutdown_connections 2023-10-18T12:37:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55f152129270 0x55f15212b660 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f1520c0580 0x55f152127820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1520ba650 0x55f1520c9d30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 --2- 172.21.15.70:0/1866319250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1520bfc60 0x55f1520ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f1f7fe700 0 client.27311 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 >> 172.21.15.70:0/1866319250 conn(0x55f15202e7a0 msgr2=0x55f1520bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 shutdown_connections 2023-10-18T12:37:55.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.399+0000 7f8f59ee3b80 1 -- 172.21.15.70:0/1866319250 wait complete. 2023-10-18T12:37:55.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f59ee3b80 20 client.27311 trim_cache size 0 max 16384 2023-10-18T12:37:55.429 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:37:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "784b0e5f-a531-45be-876f-1b583aa9f6ac", 2023-10-18T12:37:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:37:55.430 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:55.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f280a15f0 msgr2=0x7f8f280a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f280a15f0 0x7f8f280a3ab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009610 tx=0x7f8f38009560 comp rx=0 tx=0).stop 2023-10-18T12:37:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 msgr2=0x7f8f40198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 0x7f8f40198db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e9a0 tx=0x7f8f3000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 shutdown_connections 2023-10-18T12:37:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f40106730 0x7f8f4019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f40100720 0x7f8f40198db0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f8f280a15f0 0x7f8f280a3ab0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 --2- 172.21.15.70:0/3431867584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f401010e0 0x7f8f401992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 >> 172.21.15.70:0/3431867584 conn(0x7f8f400fc1b0 msgr2=0x7f8f400fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 shutdown_connections 2023-10-18T12:37:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.400+0000 7f8f465ff700 1 -- 172.21.15.70:0/3431867584 wait complete. 2023-10-18T12:37:55.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:37:55.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:55 smithi070 ceph-mon[98639]: pgmap v681: 129 pgs: 129 active+clean; 611 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 195 op/s 2023-10-18T12:37:55.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 -- 172.21.15.70:0/1510545831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17810abb0 msgr2=0x7fb17810d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 --2- 172.21.15.70:0/1510545831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17810abb0 0x7fb17810d070 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb170009f90 tx=0x7fb17000b330 comp rx=0 tx=0).stop 2023-10-18T12:37:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 -- 172.21.15.70:0/1510545831 shutdown_connections 2023-10-18T12:37:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 --2- 172.21.15.70:0/1510545831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17810abb0 0x7fb17810d070 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 --2- 172.21.15.70:0/1510545831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 --2- 172.21.15.70:0/1510545831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1781020b0 0x7fb17810a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 -- 172.21.15.70:0/1510545831 >> 172.21.15.70:0/1510545831 conn(0x7fb1780fafb0 msgr2=0x7fb1780fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.637+0000 7fb17759e700 1 -- 172.21.15.70:0/1510545831 shutdown_connections 2023-10-18T12:37:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.638+0000 7fb17759e700 1 -- 172.21.15.70:0/1510545831 wait complete. 2023-10-18T12:37:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 Processor -- start 2023-10-18T12:37:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 -- start start 2023-10-18T12:37:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781020b0 0x7fb1781993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb17810abb0 0x7fb17819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17819dd60 con 0x7fb1781020b0 2023-10-18T12:37:55.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17819ded0 con 0x7fb178101790 2023-10-18T12:37:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17759e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17819e090 con 0x7fb17810abb0 2023-10-18T12:37:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.639+0000 7fb17659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35254/0 (socket says 172.21.15.70:35254) 2023-10-18T12:37:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 -- 172.21.15.70:0/2220244605 learned_addr learned my addr 172.21.15.70:0/2220244605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb176d9d700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb17810abb0 0x7fb17819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb175d9b700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781020b0 0x7fb1781993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 -- 172.21.15.70:0/2220244605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb17810abb0 msgr2=0x7fb17819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb17810abb0 0x7fb17819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 -- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781020b0 msgr2=0x7fb1781993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781020b0 0x7fb1781993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 -- 172.21.15.70:0/2220244605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c0095f0 con 0x7fb178101790 2023-10-18T12:37:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.640+0000 7fb17659c700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178198e90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e9a0 tx=0x7fb16800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.641+0000 7fb1677fe700 1 -- 172.21.15.70:0/2220244605 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168009790 con 0x7fb178101790 2023-10-18T12:37:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.641+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb170009c40 con 0x7fb178101790 2023-10-18T12:37:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.641+0000 7fb1677fe700 1 -- 172.21.15.70:0/2220244605 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb168010430 con 0x7fb178101790 2023-10-18T12:37:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.641+0000 7fb1677fe700 1 -- 172.21.15.70:0/2220244605 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168018680 con 0x7fb178101790 2023-10-18T12:37:55.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.641+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1781a59d0 con 0x7fb178101790 2023-10-18T12:37:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.644+0000 7fb1677fe700 1 -- 172.21.15.70:0/2220244605 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb1680105a0 con 0x7fb178101790 2023-10-18T12:37:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.646+0000 7fb1677fe700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a15f0 0x7fb1600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.646+0000 7fb1677fe700 1 -- 172.21.15.70:0/2220244605 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fb168014070 con 0x7fb178101790 2023-10-18T12:37:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.646+0000 7fb175d9b700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a15f0 0x7fb1600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.646+0000 7fb19076eb80 1 Processor -- start 2023-10-18T12:37:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.646+0000 7fb19076eb80 1 -- start start 2023-10-18T12:37:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c052d60 0x56538c053140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 0x56538c04d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb175d9b700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a15f0 0x7fb1600a3ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb17819a390 tx=0x7fb16c009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c04dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb19076eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538bfc06e0 con 0x56538c052d60 2023-10-18T12:37:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb19076eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538bfa6c60 con 0x56538c04d820 2023-10-18T12:37:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb19076eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538bfa4580 con 0x56538c053680 2023-10-18T12:37:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 0x56538c04d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb17659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c052d60 0x56538c053140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 0x56538c04d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42152/0 (socket says 172.21.15.70:42152) 2023-10-18T12:37:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb17659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c052d60 0x56538c053140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36158/0 (socket says 172.21.15.70:36158) 2023-10-18T12:37:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb175d9b700 1 -- 172.21.15.70:0/3764528430 learned_addr learned my addr 172.21.15.70:0/3764528430 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.647+0000 7fb176d9d700 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c04dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb175d9b700 1 -- 172.21.15.70:0/3764528430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 msgr2=0x56538c04dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c04dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb175d9b700 1 -- 172.21.15.70:0/3764528430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c052d60 msgr2=0x56538c053140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c052d60 0x56538c053140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb175d9b700 1 -- 172.21.15.70:0/3764528430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56538bf5d3f0 con 0x56538c053680 2023-10-18T12:37:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 0x56538c04d2e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb16c004360 tx=0x7fb16c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e77cd015c9d060 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb164ff9700 1 -- 172.21.15.70:0/3764528430 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c003d10 con 0x56538c053680 2023-10-18T12:37:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb164ff9700 1 -- 172.21.15.70:0/3764528430 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16c003e70 con 0x56538c053680 2023-10-18T12:37:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.648+0000 7fb164ff9700 1 -- 172.21.15.70:0/3764528430 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c01d5f0 con 0x56538c053680 2023-10-18T12:37:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 -- 172.21.15.70:0/3764528430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 msgr2=0x56538c04d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 0x56538c04d2e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb16c004360 tx=0x7fb16c014000 comp rx=0 tx=0).stop 2023-10-18T12:37:55.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 -- 172.21.15.70:0/3764528430 shutdown_connections 2023-10-18T12:37:55.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c04dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c052d60 0x56538c053140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3764528430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c053680 0x56538c04d2e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 -- 172.21.15.70:0/3764528430 >> 172.21.15.70:0/3764528430 conn(0x56538bfc1720 msgr2=0x56538c047cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 -- 172.21.15.70:0/3764528430 shutdown_connections 2023-10-18T12:37:55.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.649+0000 7fb19076eb80 1 -- 172.21.15.70:0/3764528430 wait complete. 2023-10-18T12:37:55.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 -- start start 2023-10-18T12:37:55.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 Processor -- start 2023-10-18T12:37:55.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c0596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c0570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538bf9fe70 con 0x56538c04d820 2023-10-18T12:37:55.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538bf3c710 con 0x56538c053680 2023-10-18T12:37:55.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.650+0000 7fb19076eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538bf3c3b0 con 0x56538c052d60 2023-10-18T12:37:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb19076eb80 10 client.0 _ll_register_callbacks cb 0x56538be86a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c0596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c0596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36172/0 (socket says 172.21.15.70:36172) 2023-10-18T12:37:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 -- 172.21.15.70:0/3664593596 learned_addr learned my addr 172.21.15.70:0/3664593596 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb176d9d700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c0570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 -- 172.21.15.70:0/3664593596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 msgr2=0x56538c0569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 -- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 msgr2=0x56538c0570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c0570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 -- 172.21.15.70:0/3664593596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c017070 con 0x56538c04d820 2023-10-18T12:37:55.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:55.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb17659c700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c0596a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb16809a220 tx=0x7fb16809aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69c03d8ff13dfd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb15dffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:37:55.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.651+0000 7fb15dffb700 1 -- 172.21.15.70:0/3664593596 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168094ea0 con 0x56538c04d820 2023-10-18T12:37:55.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb15dffb700 1 -- 172.21.15.70:0/3664593596 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fb1680a2620 con 0x56538c04d820 2023-10-18T12:37:55.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56538bfa4ef0 con 0x56538c04d820 2023-10-18T12:37:55.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb15dffb700 1 -- 172.21.15.70:0/3664593596 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1680cd240 con 0x56538c04d820 2023-10-18T12:37:55.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 10 client.17271 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:55.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56538bdcade0 con 0x56538c04d820 2023-10-18T12:37:55.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb15dffb700 1 -- 172.21.15.70:0/3664593596 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb168064070 con 0x56538c04d820 2023-10-18T12:37:55.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb15dffb700 1 -- 172.21.15.70:0/3664593596 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb1680a2200 con 0x56538c04d820 2023-10-18T12:37:55.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 10 client.17271 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:55.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:55.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 20 client.17271 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:55.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c067080 0x56538c069470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 4 client.17271 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:37:55.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.652+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56538c01c8b0 con 0x56538c067080 2023-10-18T12:37:55.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.653+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c067080 0x56538c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.653+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c067080 0x56538c069470 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ee9e471ff8ba9a server_cookie=19e87e09f5b3e91e in_seq=0 out_seq=0 2023-10-18T12:37:55.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.653+0000 7fb15dffb700 10 client.17271 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:55 smithi152 ceph-mon[134767]: pgmap v681: 129 pgs: 129 active+clean; 611 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 195 op/s 2023-10-18T12:37:55.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb15dffb700 1 -- 172.21.15.70:0/3664593596 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56538c01c8b0 con 0x56538c067080 2023-10-18T12:37:55.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb15dffb700 10 client.17271 handle_command_reply: tid=0 2023-10-18T12:37:55.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 client.17271 shutdown 2023-10-18T12:37:55.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 msgr2=0x56538c0596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c0596a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb16809a220 tx=0x7fb16809aae0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 shutdown_connections 2023-10-18T12:37:55.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c067080 0x56538c069470 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c0570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c0596a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb15dffb700 0 client.17271 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3664593596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 >> 172.21.15.70:0/3664593596 conn(0x56538bfc1720 msgr2=0x56538c04c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 shutdown_connections 2023-10-18T12:37:55.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 1 -- 172.21.15.70:0/3664593596 wait complete. 2023-10-18T12:37:55.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.658+0000 7fb19076eb80 20 client.17271 trim_cache size 0 max 16384 2023-10-18T12:37:55.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 Processor -- start 2023-10-18T12:37:55.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 -- start start 2023-10-18T12:37:55.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c05e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c053680 0x56538c056270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538be20920 con 0x56538c053680 2023-10-18T12:37:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538c066ed0 con 0x56538c04d820 2023-10-18T12:37:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.675+0000 7fb19076eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538c064800 con 0x56538c052d60 2023-10-18T12:37:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c05e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c05e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35300/0 (socket says 172.21.15.70:35300) 2023-10-18T12:37:55.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb176d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c053680 0x56538c056270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42170/0 (socket says 172.21.15.70:42170) 2023-10-18T12:37:55.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 -- 172.21.15.70:0/3762759785 learned_addr learned my addr 172.21.15.70:0/3762759785 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 -- 172.21.15.70:0/3762759785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 msgr2=0x56538c0591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 -- 172.21.15.70:0/3762759785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c053680 msgr2=0x56538c056270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c053680 0x56538c056270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 -- 172.21.15.70:0/3762759785 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56538bdb3c60 con 0x56538c04d820 2023-10-18T12:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb176d9d700 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c053680 0x56538c056270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb175d9b700 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:55.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb17659c700 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c05e090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb168099f90 tx=0x7fb168010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd6438f9a85bb0d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.676+0000 7fb1657fa700 1 -- 172.21.15.70:0/3762759785 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168094ea0 con 0x56538c04d820 2023-10-18T12:37:55.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb1657fa700 1 -- 172.21.15.70:0/3762759785 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb168064070 con 0x56538c04d820 2023-10-18T12:37:55.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb1657fa700 1 -- 172.21.15.70:0/3762759785 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680a2190 con 0x56538c04d820 2023-10-18T12:37:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 -- 172.21.15.70:0/3762759785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 msgr2=0x56538c05e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c05e090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb168099f90 tx=0x7fb168010a90 comp rx=0 tx=0).stop 2023-10-18T12:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 -- 172.21.15.70:0/3762759785 shutdown_connections 2023-10-18T12:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c053680 0x56538c056270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c04d820 0x56538c05e090 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 --2- 172.21.15.70:0/3762759785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c0591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 -- 172.21.15.70:0/3762759785 >> 172.21.15.70:0/3762759785 conn(0x56538bfc1720 msgr2=0x56538c049580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 -- 172.21.15.70:0/3762759785 shutdown_connections 2023-10-18T12:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.677+0000 7fb19076eb80 1 -- 172.21.15.70:0/3762759785 wait complete. 2023-10-18T12:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 -- start start 2023-10-18T12:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 Processor -- start 2023-10-18T12:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c05dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c05c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c05c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538c05d070 con 0x56538c04d820 2023-10-18T12:37:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538c05d1e0 con 0x56538c053680 2023-10-18T12:37:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.678+0000 7fb19076eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538c0bae60 con 0x56538c052d60 2023-10-18T12:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb19076eb80 10 client.0 _ll_register_callbacks cb 0x56538be86a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c05c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c05c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c05c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35312/0 (socket says 172.21.15.70:35312) 2023-10-18T12:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb175d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c05c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:42174/0 (socket says 172.21.15.70:42174) 2023-10-18T12:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 -- 172.21.15.70:0/2954465947 learned_addr learned my addr 172.21.15.70:0/2954465947 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb17659c700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c05dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 -- 172.21.15.70:0/2954465947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 msgr2=0x56538c05c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c05c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 -- 172.21.15.70:0/2954465947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 msgr2=0x56538c05dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c05dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 -- 172.21.15.70:0/2954465947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56538bf5d3f0 con 0x56538c053680 2023-10-18T12:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb175d9b700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c05c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb17659c700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c05dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb176d9d700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c05c9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb170003dc0 tx=0x7fb170003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7019816174f04b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.679+0000 7fb15dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb15dffb700 1 -- 172.21.15.70:0/2954465947 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000e040 con 0x56538c053680 2023-10-18T12:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb16c003e70 con 0x56538c053680 2023-10-18T12:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb15dffb700 1 -- 172.21.15.70:0/2954465947 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fb1700079d0 con 0x56538c053680 2023-10-18T12:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb15dffb700 1 -- 172.21.15.70:0/2954465947 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb170007b90 con 0x56538c053680 2023-10-18T12:37:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 10 client.27329 fetch_fsmap learned FSMap version 20 2023-10-18T12:37:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fb17810dd30 con 0x56538c053680 2023-10-18T12:37:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb15dffb700 1 -- 172.21.15.70:0/2954465947 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16c003e70 con 0x56538c053680 2023-10-18T12:37:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb15dffb700 1 -- 172.21.15.70:0/2954465947 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb17000b050 con 0x56538c053680 2023-10-18T12:37:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 10 client.27329 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:37:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 10 client.27329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:37:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 20 client.27329 populate_metadata read hostname 'smithi070' 2023-10-18T12:37:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c0bc5a0 0x56538c0be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:37:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 4 client.27329 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:37:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.680+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56538bf98e50 con 0x56538c0bc5a0 2023-10-18T12:37:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb17659c700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c0bc5a0 0x56538c0be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:37:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb17659c700 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c0bc5a0 0x56538c0be9c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24b3c0449790f00 server_cookie=47d6f0ccc15b90e6 in_seq=0 out_seq=0 2023-10-18T12:37:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb15dffb700 10 client.27329 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb15dffb700 1 -- 172.21.15.70:0/2954465947 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56538bf98e50 con 0x56538c0bc5a0 2023-10-18T12:37:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb15dffb700 10 client.27329 handle_command_reply: tid=0 2023-10-18T12:37:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 client.27329 shutdown 2023-10-18T12:37:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 msgr2=0x56538c05c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c05c9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb170003dc0 tx=0x7fb170003df0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 shutdown_connections 2023-10-18T12:37:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x56538c0bc5a0 0x56538c0be9c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56538c053680 0x56538c05c9e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb15dffb700 0 client.27329 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:37:55.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56538c04d820 0x56538c05dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 --2- 172.21.15.70:0/2954465947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56538c052d60 0x56538c05c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.681+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 >> 172.21.15.70:0/2954465947 conn(0x56538bfc1720 msgr2=0x56538c0508d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 shutdown_connections 2023-10-18T12:37:55.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb19076eb80 1 -- 172.21.15.70:0/2954465947 wait complete. 2023-10-18T12:37:55.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb19076eb80 20 client.27329 trim_cache size 0 max 16384 2023-10-18T12:37:55.701 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:37:55.701 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:37:55.702 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:37:55.702 INFO:teuthology.orchestra.run.smithi070.stdout: "784b0e5f-a531-45be-876f-1b583aa9f6ac": { 2023-10-18T12:37:55.702 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:37:55.702 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "784b0e5f-a531-45be-876f-1b583aa9f6ac", 2023-10-18T12:37:55.702 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:37:55.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:37:55.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:37:55.703 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:37:55.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a15f0 msgr2=0x7fb1600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb17759e700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a15f0 0x7fb1600a3ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb17819a390 tx=0x7fb16c009310 comp rx=0 tx=0).stop 2023-10-18T12:37:55.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 msgr2=0x7fb178198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:37:55.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb17759e700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178198e90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e9a0 tx=0x7fb16800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.682+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 shutdown_connections 2023-10-18T12:37:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb17810abb0 0x7fb17819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb1600a15f0 0x7fb1600a3ab0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb178101790 0x7fb178198e90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 --2- 172.21.15.70:0/2220244605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1781020b0 0x7fb1781993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:37:55.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 >> 172.21.15.70:0/2220244605 conn(0x7fb1780fafb0 msgr2=0x7fb1781004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:37:55.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 shutdown_connections 2023-10-18T12:37:55.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:37:55.683+0000 7fb17759e700 1 -- 172.21.15.70:0/2220244605 wait complete. 2023-10-18T12:37:55.707 INFO:tasks.cephfs.filesystem:scrub status for tag:784b0e5f-a531-45be-876f-1b583aa9f6ac - {'path': '/', 'tag': '784b0e5f-a531-45be-876f-1b583aa9f6ac', 'options': 'recursive,force'} 2023-10-18T12:37:55.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:55 smithi186 ceph-mon[135253]: pgmap v681: 129 pgs: 129 active+clean; 611 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 195 op/s 2023-10-18T12:37:55.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1339928 11.09 MB/sec execute 504 sec latency 16.103 ms 2023-10-18T12:37:56.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1342113 11.09 MB/sec execute 505 sec latency 85.100 ms 2023-10-18T12:37:57.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:57 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:37:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:57 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:57 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:37:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:57 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:57.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:57 smithi070 ceph-mon[98639]: pgmap v682: 129 pgs: 129 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 148 op/s 2023-10-18T12:37:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:57 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:37:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:57 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:57 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:37:57.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:57 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:57.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:57 smithi152 ceph-mon[134767]: pgmap v682: 129 pgs: 129 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 148 op/s 2023-10-18T12:37:57.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:57 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:37:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:57 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:57 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:37:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:57 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:37:57.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:57 smithi186 ceph-mon[135253]: pgmap v682: 129 pgs: 129 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 148 op/s 2023-10-18T12:37:57.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1343245 11.08 MB/sec execute 506 sec latency 64.919 ms 2023-10-18T12:37:58.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1345996 11.09 MB/sec execute 507 sec latency 13.552 ms 2023-10-18T12:37:59.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:59 smithi070 ceph-mon[98639]: pgmap v683: 129 pgs: 129 active+clean; 571 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 176 op/s 2023-10-18T12:37:59.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:37:59 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:37:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:59 smithi152 ceph-mon[134767]: pgmap v683: 129 pgs: 129 active+clean; 571 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 176 op/s 2023-10-18T12:37:59.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:37:59 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:37:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:59 smithi186 ceph-mon[135253]: pgmap v683: 129 pgs: 129 active+clean; 571 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 176 op/s 2023-10-18T12:37:59.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:37:59 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:37:59.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1347559 11.08 MB/sec execute 508 sec latency 15.620 ms 2023-10-18T12:38:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:00 smithi152 ceph-mon[134767]: pgmap v684: 129 pgs: 129 active+clean; 571 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:38:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:00 smithi186 ceph-mon[135253]: pgmap v684: 129 pgs: 129 active+clean; 571 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:38:00.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:00 smithi070 ceph-mon[98639]: pgmap v684: 129 pgs: 129 active+clean; 571 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 199 op/s 2023-10-18T12:38:00.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1349624 11.09 MB/sec execute 509 sec latency 44.866 ms 2023-10-18T12:38:01.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1349664 11.07 MB/sec execute 510 sec latency 150.455 ms 2023-10-18T12:38:02.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1350963 11.06 MB/sec execute 511 sec latency 40.908 ms 2023-10-18T12:38:03.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:02 smithi152 ceph-mon[134767]: pgmap v685: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-18T12:38:03.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:02 smithi186 ceph-mon[135253]: pgmap v685: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-18T12:38:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:02 smithi070 ceph-mon[98639]: pgmap v685: 129 pgs: 129 active+clean; 574 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2023-10-18T12:38:03.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1353197 11.06 MB/sec execute 512 sec latency 16.057 ms 2023-10-18T12:38:04.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1354650 11.05 MB/sec execute 513 sec latency 40.319 ms 2023-10-18T12:38:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:04 smithi152 ceph-mon[134767]: pgmap v686: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-10-18T12:38:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:04 smithi152 ceph-mon[134767]: osdmap e104: 12 total, 12 up, 12 in 2023-10-18T12:38:05.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:04 smithi186 ceph-mon[135253]: pgmap v686: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-10-18T12:38:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:04 smithi186 ceph-mon[135253]: osdmap e104: 12 total, 12 up, 12 in 2023-10-18T12:38:05.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:04 smithi070 ceph-mon[98639]: pgmap v686: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-10-18T12:38:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:04 smithi070 ceph-mon[98639]: osdmap e104: 12 total, 12 up, 12 in 2023-10-18T12:38:05.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1356942 11.06 MB/sec execute 514 sec latency 13.740 ms 2023-10-18T12:38:06.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1358307 11.04 MB/sec execute 515 sec latency 132.142 ms 2023-10-18T12:38:07.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:06 smithi152 ceph-mon[134767]: pgmap v688: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2023-10-18T12:38:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:06 smithi186 ceph-mon[135253]: pgmap v688: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2023-10-18T12:38:07.357 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:06 smithi070 ceph-mon[98639]: pgmap v688: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2023-10-18T12:38:07.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1360234 11.05 MB/sec execute 516 sec latency 37.003 ms 2023-10-18T12:38:08.885 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1361741 11.04 MB/sec execute 517 sec latency 24.434 ms 2023-10-18T12:38:09.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:08 smithi152 ceph-mon[134767]: pgmap v689: 129 pgs: 129 active+clean; 592 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 238 op/s 2023-10-18T12:38:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:08 smithi186 ceph-mon[135253]: pgmap v689: 129 pgs: 129 active+clean; 592 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 238 op/s 2023-10-18T12:38:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:08 smithi070 ceph-mon[98639]: pgmap v689: 129 pgs: 129 active+clean; 592 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 238 op/s 2023-10-18T12:38:09.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1364073 11.04 MB/sec execute 518 sec latency 15.438 ms 2023-10-18T12:38:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:10 smithi152 ceph-mon[134767]: pgmap v690: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-10-18T12:38:10.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:10 smithi186 ceph-mon[135253]: pgmap v690: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-10-18T12:38:10.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:10 smithi070 ceph-mon[98639]: pgmap v690: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-10-18T12:38:10.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1366087 11.04 MB/sec execute 519 sec latency 17.151 ms 2023-10-18T12:38:11.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1367737 11.04 MB/sec execute 520 sec latency 35.586 ms 2023-10-18T12:38:12.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1368994 11.03 MB/sec execute 521 sec latency 49.380 ms 2023-10-18T12:38:13.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:12 smithi152 ceph-mon[134767]: pgmap v691: 129 pgs: 129 active+clean; 590 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 194 op/s 2023-10-18T12:38:13.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:12 smithi186 ceph-mon[135253]: pgmap v691: 129 pgs: 129 active+clean; 590 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 194 op/s 2023-10-18T12:38:13.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:12 smithi070 ceph-mon[98639]: pgmap v691: 129 pgs: 129 active+clean; 590 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 194 op/s 2023-10-18T12:38:13.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1371070 11.03 MB/sec execute 522 sec latency 19.485 ms 2023-10-18T12:38:14.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1372904 11.02 MB/sec execute 523 sec latency 35.646 ms 2023-10-18T12:38:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:14 smithi152 ceph-mon[134767]: pgmap v692: 129 pgs: 129 active+clean; 600 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 194 op/s 2023-10-18T12:38:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:14 smithi152 ceph-mon[134767]: osdmap e105: 12 total, 12 up, 12 in 2023-10-18T12:38:15.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:14 smithi186 ceph-mon[135253]: pgmap v692: 129 pgs: 129 active+clean; 600 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 194 op/s 2023-10-18T12:38:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:14 smithi186 ceph-mon[135253]: osdmap e105: 12 total, 12 up, 12 in 2023-10-18T12:38:15.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:14 smithi070 ceph-mon[98639]: pgmap v692: 129 pgs: 129 active+clean; 600 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 194 op/s 2023-10-18T12:38:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:14 smithi070 ceph-mon[98639]: osdmap e105: 12 total, 12 up, 12 in 2023-10-18T12:38:15.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1374831 11.03 MB/sec execute 524 sec latency 15.037 ms 2023-10-18T12:38:16.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1375826 11.05 MB/sec execute 525 sec latency 60.886 ms 2023-10-18T12:38:17.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:16 smithi152 ceph-mon[134767]: pgmap v694: 129 pgs: 129 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 217 op/s 2023-10-18T12:38:17.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:16 smithi186 ceph-mon[135253]: pgmap v694: 129 pgs: 129 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 217 op/s 2023-10-18T12:38:17.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:16 smithi070 ceph-mon[98639]: pgmap v694: 129 pgs: 129 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 217 op/s 2023-10-18T12:38:17.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1377448 11.04 MB/sec execute 526 sec latency 73.050 ms 2023-10-18T12:38:18.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1379324 11.04 MB/sec execute 527 sec latency 26.188 ms 2023-10-18T12:38:19.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:18 smithi152 ceph-mon[134767]: pgmap v695: 129 pgs: 129 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 188 op/s 2023-10-18T12:38:19.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:18 smithi186 ceph-mon[135253]: pgmap v695: 129 pgs: 129 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 188 op/s 2023-10-18T12:38:19.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:18 smithi070 ceph-mon[98639]: pgmap v695: 129 pgs: 129 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 188 op/s 2023-10-18T12:38:19.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1381799 11.05 MB/sec execute 528 sec latency 37.769 ms 2023-10-18T12:38:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:20 smithi152 ceph-mon[134767]: pgmap v696: 129 pgs: 129 active+clean; 624 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 240 op/s 2023-10-18T12:38:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:20 smithi186 ceph-mon[135253]: pgmap v696: 129 pgs: 129 active+clean; 624 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 240 op/s 2023-10-18T12:38:20.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:20 smithi070 ceph-mon[98639]: pgmap v696: 129 pgs: 129 active+clean; 624 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 240 op/s 2023-10-18T12:38:20.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1383559 11.05 MB/sec execute 529 sec latency 14.253 ms 2023-10-18T12:38:21.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1385957 11.05 MB/sec execute 530 sec latency 14.911 ms 2023-10-18T12:38:22.886 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1387134 11.04 MB/sec execute 531 sec latency 60.745 ms 2023-10-18T12:38:23.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:22 smithi152 ceph-mon[134767]: pgmap v697: 129 pgs: 129 active+clean; 628 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 252 op/s 2023-10-18T12:38:23.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:22 smithi186 ceph-mon[135253]: pgmap v697: 129 pgs: 129 active+clean; 628 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 252 op/s 2023-10-18T12:38:23.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:22 smithi070 ceph-mon[98639]: pgmap v697: 129 pgs: 129 active+clean; 628 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 252 op/s 2023-10-18T12:38:23.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1389439 11.04 MB/sec execute 532 sec latency 30.831 ms 2023-10-18T12:38:24.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1390836 11.03 MB/sec execute 533 sec latency 65.505 ms 2023-10-18T12:38:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:24 smithi152 ceph-mon[134767]: pgmap v698: 129 pgs: 129 active+clean; 617 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 226 op/s 2023-10-18T12:38:25.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:24 smithi186 ceph-mon[135253]: pgmap v698: 129 pgs: 129 active+clean; 617 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 226 op/s 2023-10-18T12:38:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:24 smithi070 ceph-mon[98639]: pgmap v698: 129 pgs: 129 active+clean; 617 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 226 op/s 2023-10-18T12:38:25.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:38:25.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1393058 11.04 MB/sec execute 534 sec latency 42.614 ms 2023-10-18T12:38:25.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 -- 172.21.15.70:0/2905641454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed640734e0 msgr2=0x7fed6410ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 --2- 172.21.15.70:0/2905641454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed640734e0 0x7fed6410ca60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed50009990 tx=0x7fed50009ca0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 -- 172.21.15.70:0/2905641454 shutdown_connections 2023-10-18T12:38:25.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 --2- 172.21.15.70:0/2905641454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6410cfa0 0x7fed6410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 --2- 172.21.15.70:0/2905641454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 --2- 172.21.15.70:0/2905641454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed640734e0 0x7fed6410ca60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 -- 172.21.15.70:0/2905641454 >> 172.21.15.70:0/2905641454 conn(0x7fed640fc1d0 msgr2=0x7fed640fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:25.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 -- 172.21.15.70:0/2905641454 shutdown_connections 2023-10-18T12:38:25.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.922+0000 7fed6adff700 1 -- 172.21.15.70:0/2905641454 wait complete. 2023-10-18T12:38:25.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.923+0000 7fed6adff700 1 Processor -- start 2023-10-18T12:38:25.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.923+0000 7fed6adff700 1 -- start start 2023-10-18T12:38:25.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.923+0000 7fed6adff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.923+0000 7fed6adff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed640734e0 0x7fed641993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed6adff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed6410cfa0 0x7fed6419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed6adff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6419dd80 con 0x7fed640734e0 2023-10-18T12:38:25.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed6adff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6419def0 con 0x7fed6410cfa0 2023-10-18T12:38:25.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed6adff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6419e0b0 con 0x7fed64072bc0 2023-10-18T12:38:25.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed6939c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed6410cfa0 0x7fed6419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed63fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed640734e0 0x7fed641993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:44896/0 (socket says 172.21.15.70:44896) 2023-10-18T12:38:25.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 -- 172.21.15.70:0/1120351191 learned_addr learned my addr 172.21.15.70:0/1120351191 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:25.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 -- 172.21.15.70:0/1120351191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed6410cfa0 msgr2=0x7fed6419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed6410cfa0 0x7fed6419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 -- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed640734e0 msgr2=0x7fed641993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed640734e0 0x7fed641993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 -- 172.21.15.70:0/1120351191 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500095f0 con 0x7fed64072bc0 2023-10-18T12:38:25.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed6939c700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed6410cfa0 0x7fed6419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:25.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed63fff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed640734e0 0x7fed641993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:25.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.924+0000 7fed68b9b700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64198e60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed5800d980 tx=0x7fed5800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:25.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.925+0000 7fed61ffb700 1 -- 172.21.15.70:0/1120351191 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58004220 con 0x7fed64072bc0 2023-10-18T12:38:25.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.925+0000 7fed61ffb700 1 -- 172.21.15.70:0/1120351191 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed58007640 con 0x7fed64072bc0 2023-10-18T12:38:25.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.925+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6419e390 con 0x7fed64072bc0 2023-10-18T12:38:25.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.925+0000 7fed61ffb700 1 -- 172.21.15.70:0/1120351191 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58005690 con 0x7fed64072bc0 2023-10-18T12:38:25.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.925+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed641a5bd0 con 0x7fed64072bc0 2023-10-18T12:38:25.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.928+0000 7fed61ffb700 1 -- 172.21.15.70:0/1120351191 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fed58004750 con 0x7fed64072bc0 2023-10-18T12:38:25.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.929+0000 7fed7e7dfb80 1 Processor -- start 2023-10-18T12:38:25.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.929+0000 7fed7e7dfb80 1 -- start start 2023-10-18T12:38:25.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.929+0000 7fed61ffb700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed540a1670 0x7fed540a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed61ffb700 1 -- 172.21.15.70:0/1120351191 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fed580cd170 con 0x7fed64072bc0 2023-10-18T12:38:25.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f53be0 0x55a478f53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f4e5d0 0x55a478f4ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478ec16e0 con 0x55a478f54500 2023-10-18T12:38:25.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478ea7c60 con 0x55a478f53be0 2023-10-18T12:38:25.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478ea5580 con 0x55a478f4e5d0 2023-10-18T12:38:25.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed6939c700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed540a1670 0x7fed540a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed63fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed6939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f4e5d0 0x55a478f4ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed63fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f4e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56260/0 (socket says 172.21.15.70:56260) 2023-10-18T12:38:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.930+0000 7fed63fff700 1 -- 172.21.15.70:0/1830135605 learned_addr learned my addr 172.21.15.70:0/1830135605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed63fff700 1 -- 172.21.15.70:0/1830135605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f4e5d0 msgr2=0x55a478f4ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed68b9b700 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f53be0 0x55a478f53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed63fff700 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f4e5d0 0x55a478f4ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed63fff700 1 -- 172.21.15.70:0/1830135605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f53be0 msgr2=0x55a478f53fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed63fff700 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f53be0 0x55a478f53fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed63fff700 1 -- 172.21.15.70:0/1830135605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a478e5e3f0 con 0x55a478f54500 2023-10-18T12:38:25.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed68b9b700 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f53be0 0x55a478f53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:38:25.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed6939c700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed540a1670 0x7fed540a3b30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009ec0 tx=0x7fed5c003cf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:25.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed63fff700 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f4e090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fed50015000 tx=0x7fed50005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b046fd51483f82d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:25.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed4effd700 1 -- 172.21.15.70:0/1830135605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5001d070 con 0x55a478f54500 2023-10-18T12:38:25.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed4effd700 1 -- 172.21.15.70:0/1830135605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5001e040 con 0x55a478f54500 2023-10-18T12:38:25.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.931+0000 7fed4effd700 1 -- 172.21.15.70:0/1830135605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50004ce0 con 0x55a478f54500 2023-10-18T12:38:25.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/1830135605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 msgr2=0x55a478f4e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f4e090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fed50015000 tx=0x7fed50005f00 comp rx=0 tx=0).stop 2023-10-18T12:38:25.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/1830135605 shutdown_connections 2023-10-18T12:38:25.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f4e5d0 0x55a478f4ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f53be0 0x55a478f53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/1830135605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f4e090 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/1830135605 >> 172.21.15.70:0/1830135605 conn(0x55a478ec2720 msgr2=0x55a478f48b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:25.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/1830135605 shutdown_connections 2023-10-18T12:38:25.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.932+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/1830135605 wait complete. 2023-10-18T12:38:25.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.933+0000 7fed7e7dfb80 1 -- start start 2023-10-18T12:38:25.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.933+0000 7fed7e7dfb80 1 Processor -- start 2023-10-18T12:38:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 0x55a478f5a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f54500 0x55a478f57df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478ea0e70 con 0x55a478f4e5d0 2023-10-18T12:38:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478e3d710 con 0x55a478f54500 2023-10-18T12:38:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478e3d3b0 con 0x55a478f53be0 2023-10-18T12:38:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed7e7dfb80 10 client.0 _ll_register_callbacks cb 0x55a478d87a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 0x55a478f5a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 0x55a478f5a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56270/0 (socket says 172.21.15.70:56270) 2023-10-18T12:38:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed6939c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f54500 0x55a478f57df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 -- 172.21.15.70:0/644168810 learned_addr learned my addr 172.21.15.70:0/644168810 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed63fff700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 -- 172.21.15.70:0/644168810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 msgr2=0x55a478f577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 -- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f54500 msgr2=0x55a478f57df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f54500 0x55a478f57df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed68b9b700 1 -- 172.21.15.70:0/644168810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed641100e0 con 0x55a478f4e5d0 2023-10-18T12:38:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.934+0000 7fed6939c700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f54500 0x55a478f57df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed68b9b700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 0x55a478f5a4f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fed58007e80 tx=0x7fed5809a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367fe30bb22780ac server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed43fff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:38:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed43fff700 1 -- 172.21.15.70:0/644168810 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5809a250 con 0x55a478f4e5d0 2023-10-18T12:38:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed43fff700 1 -- 172.21.15.70:0/644168810 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fed5809df40 con 0x55a478f4e5d0 2023-10-18T12:38:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a478ea5ef0 con 0x55a478f4e5d0 2023-10-18T12:38:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed43fff700 1 -- 172.21.15.70:0/644168810 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed580a2180 con 0x55a478f4e5d0 2023-10-18T12:38:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed7e7dfb80 10 client.17289 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a478ccbde0 con 0x55a478f4e5d0 2023-10-18T12:38:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.935+0000 7fed43fff700 1 -- 172.21.15.70:0/644168810 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a478ea5ef0 con 0x55a478f4e5d0 2023-10-18T12:38:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed43fff700 1 -- 172.21.15.70:0/644168810 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed580a1cf0 con 0x55a478f4e5d0 2023-10-18T12:38:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed7e7dfb80 10 client.17289 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed7e7dfb80 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed7e7dfb80 20 client.17289 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478f67d50 0x55a478f6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed7e7dfb80 4 client.17289 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a478f1d8b0 con 0x55a478f67d50 2023-10-18T12:38:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed63fff700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478f67d50 0x55a478f6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed43fff700 10 client.17289 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.936+0000 7fed63fff700 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478f67d50 0x55a478f6a140 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e4e3804a94ec24 server_cookie=58e2a7f1ad76c3ba in_seq=0 out_seq=0 2023-10-18T12:38:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.937+0000 7fed43fff700 1 -- 172.21.15.70:0/644168810 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a478f1d8b0 con 0x55a478f67d50 2023-10-18T12:38:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.937+0000 7fed43fff700 10 client.17289 handle_command_reply: tid=0 2023-10-18T12:38:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.937+0000 7fed7e7dfb80 1 client.17289 shutdown 2023-10-18T12:38:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.937+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 msgr2=0x55a478f5a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.937+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 0x55a478f5a4f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fed58007e80 tx=0x7fed5809a4f0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 shutdown_connections 2023-10-18T12:38:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478f67d50 0x55a478f6a140 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f54500 0x55a478f57df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f4e5d0 0x55a478f5a4f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed43fff700 0 client.17289 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/644168810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 >> 172.21.15.70:0/644168810 conn(0x55a478ec2720 msgr2=0x55a478f4d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 shutdown_connections 2023-10-18T12:38:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/644168810 wait complete. 2023-10-18T12:38:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.938+0000 7fed7e7dfb80 20 client.17289 trim_cache size 0 max 16384 2023-10-18T12:38:25.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.957+0000 7fed7e7dfb80 1 Processor -- start 2023-10-18T12:38:25.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.957+0000 7fed7e7dfb80 1 -- start start 2023-10-18T12:38:25.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f4d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f57040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478d21920 con 0x55a478f54500 2023-10-18T12:38:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478f490f0 con 0x55a478f4e5d0 2023-10-18T12:38:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478f4a430 con 0x55a478f53be0 2023-10-18T12:38:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f4d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f4d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:44934/0 (socket says 172.21.15.70:44934) 2023-10-18T12:38:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 -- 172.21.15.70:0/2064215359 learned_addr learned my addr 172.21.15.70:0/2064215359 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed6939c700 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f57040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 -- 172.21.15.70:0/2064215359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 msgr2=0x55a478f5eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 -- 172.21.15.70:0/2064215359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 msgr2=0x55a478f57040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f57040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 -- 172.21.15.70:0/2064215359 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a478cb4c60 con 0x55a478f53be0 2023-10-18T12:38:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed6939c700 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f57040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:38:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.958+0000 7fed63fff700 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f4d930 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fed50009990 tx=0x7fed50000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=588a10363e35f4ac server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed4f7fe700 1 -- 172.21.15.70:0/2064215359 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50032040 con 0x55a478f53be0 2023-10-18T12:38:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed4f7fe700 1 -- 172.21.15.70:0/2064215359 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed500344c0 con 0x55a478f53be0 2023-10-18T12:38:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed4f7fe700 1 -- 172.21.15.70:0/2064215359 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5001d040 con 0x55a478f53be0 2023-10-18T12:38:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/2064215359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 msgr2=0x55a478f4d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f4d930 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fed50009990 tx=0x7fed50000bc0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/2064215359 shutdown_connections 2023-10-18T12:38:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478f57040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/2064215359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f4d930 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.959+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/2064215359 >> 172.21.15.70:0/2064215359 conn(0x55a478ec2720 msgr2=0x55a478f4d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.960+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/2064215359 shutdown_connections 2023-10-18T12:38:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.960+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/2064215359 wait complete. 2023-10-18T12:38:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.960+0000 7fed7e7dfb80 1 -- start start 2023-10-18T12:38:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.960+0000 7fed7e7dfb80 1 Processor -- start 2023-10-18T12:38:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f5e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478fbab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478f5ef50 con 0x55a478f54500 2023-10-18T12:38:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478fbb060 con 0x55a478f4e5d0 2023-10-18T12:38:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a478fbb1a0 con 0x55a478f53be0 2023-10-18T12:38:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed7e7dfb80 10 client.0 _ll_register_callbacks cb 0x55a478d87a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46710/0 (socket says 172.21.15.70:46710) 2023-10-18T12:38:25.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 -- 172.21.15.70:0/3948485144 learned_addr learned my addr 172.21.15.70:0/3948485144 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:25.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed6939c700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478fbab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed63fff700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f5e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 -- 172.21.15.70:0/3948485144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 msgr2=0x55a478f5e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f5e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 -- 172.21.15.70:0/3948485144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 msgr2=0x55a478fbab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478fbab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 -- 172.21.15.70:0/3948485144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a478e5e3f0 con 0x55a478f4e5d0 2023-10-18T12:38:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed6939c700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478fbab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:38:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed68b9b700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5e0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed580cab60 tx=0x7fed580a1f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c68446d6127f57fa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.961+0000 7fed43fff700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:38:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed43fff700 1 -- 172.21.15.70:0/3948485144 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58014040 con 0x55a478f4e5d0 2023-10-18T12:38:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5001e040 con 0x55a478f4e5d0 2023-10-18T12:38:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed43fff700 1 -- 172.21.15.70:0/3948485144 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fed58098fc0 con 0x55a478f4e5d0 2023-10-18T12:38:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed43fff700 1 -- 172.21.15.70:0/3948485144 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed58098d50 con 0x55a478f4e5d0 2023-10-18T12:38:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 10 client.27344 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a478cf2250 con 0x55a478f4e5d0 2023-10-18T12:38:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed43fff700 1 -- 172.21.15.70:0/3948485144 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5001e040 con 0x55a478f4e5d0 2023-10-18T12:38:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed43fff700 1 -- 172.21.15.70:0/3948485144 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed580adbd0 con 0x55a478f4e5d0 2023-10-18T12:38:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 10 client.27344 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 10 client.27344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 20 client.27344 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478fbc5b0 0x55a478fbe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 4 client.27344 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:38:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.962+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a478e99e50 con 0x55a478fbc5b0 2023-10-18T12:38:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.963+0000 7fed63fff700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478fbc5b0 0x55a478fbe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.963+0000 7fed63fff700 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478fbc5b0 0x55a478fbe9d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed33c5588ef71a87 server_cookie=8e37a55015b0168c in_seq=0 out_seq=0 2023-10-18T12:38:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.963+0000 7fed43fff700 10 client.27344 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.963+0000 7fed43fff700 1 -- 172.21.15.70:0/3948485144 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a478e99e50 con 0x55a478fbc5b0 2023-10-18T12:38:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.963+0000 7fed43fff700 10 client.27344 handle_command_reply: tid=0 2023-10-18T12:38:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.963+0000 7fed7e7dfb80 1 client.27344 shutdown 2023-10-18T12:38:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 msgr2=0x55a478f5e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5e0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed580cab60 tx=0x7fed580a1f00 comp rx=0 tx=0).stop 2023-10-18T12:38:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 shutdown_connections 2023-10-18T12:38:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55a478fbc5b0 0x55a478fbe9d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a478f54500 0x55a478fbab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a478f4e5d0 0x55a478f5e0d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 --2- 172.21.15.70:0/3948485144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a478f53be0 0x55a478f5e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 >> 172.21.15.70:0/3948485144 conn(0x55a478ec2720 msgr2=0x55a478f63600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed43fff700 0 client.27344 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 shutdown_connections 2023-10-18T12:38:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 1 -- 172.21.15.70:0/3948485144 wait complete. 2023-10-18T12:38:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.964+0000 7fed7e7dfb80 20 client.27344 trim_cache size 0 max 16384 2023-10-18T12:38:25.981 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:38:25.982 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:38:25.982 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:38:25.982 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:38:25.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed540a1670 msgr2=0x7fed540a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed540a1670 0x7fed540a3b30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009ec0 tx=0x7fed5c003cf0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 msgr2=0x7fed64198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:25.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64198e60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed5800d980 tx=0x7fed5800dc90 comp rx=0 tx=0).stop 2023-10-18T12:38:25.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 shutdown_connections 2023-10-18T12:38:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed540a1670 0x7fed540a3b30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed6410cfa0 0x7fed6419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed64072bc0 0x7fed64198e60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 --2- 172.21.15.70:0/1120351191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed640734e0 0x7fed641993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 >> 172.21.15.70:0/1120351191 conn(0x7fed640fc1d0 msgr2=0x7fed640fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 shutdown_connections 2023-10-18T12:38:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:25.965+0000 7fed6adff700 1 -- 172.21.15.70:0/1120351191 wait complete. 2023-10-18T12:38:25.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:38:25.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:38:26.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 -- 172.21.15.70:0/4177458924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11341004c0 msgr2=0x7f1134100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 --2- 172.21.15.70:0/4177458924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11341004c0 0x7f1134100940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1128009990 tx=0x7f1128009ca0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 -- 172.21.15.70:0/4177458924 shutdown_connections 2023-10-18T12:38:26.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 --2- 172.21.15.70:0/4177458924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f113410d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 --2- 172.21.15.70:0/4177458924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1134106580 0x7f1134106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 --2- 172.21.15.70:0/4177458924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11341004c0 0x7f1134100940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 -- 172.21.15.70:0/4177458924 >> 172.21.15.70:0/4177458924 conn(0x7f11340fbfb0 msgr2=0x7f11340fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 -- 172.21.15.70:0/4177458924 shutdown_connections 2023-10-18T12:38:26.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.208+0000 7f113959d700 1 -- 172.21.15.70:0/4177458924 wait complete. 2023-10-18T12:38:26.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.209+0000 7f113959d700 1 Processor -- start 2023-10-18T12:38:26.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.209+0000 7f113959d700 1 -- start start 2023-10-18T12:38:26.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f113959d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11341004c0 0x7f1134072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f113959d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f1134073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f113959d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1134106580 0x7f1134073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f113959d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a4c80 con 0x7f11341004c0 2023-10-18T12:38:26.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f113959d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a4df0 con 0x7f1134100e80 2023-10-18T12:38:26.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f113959d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a4fb0 con 0x7f1134106580 2023-10-18T12:38:26.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f11337fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f1134073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f1133fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11341004c0 0x7f1134072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f11337fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f1134073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46742/0 (socket says 172.21.15.70:46742) 2023-10-18T12:38:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f11337fe700 1 -- 172.21.15.70:0/2583670951 learned_addr learned my addr 172.21.15.70:0/2583670951 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.210+0000 7f1138d9c700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1134106580 0x7f1134073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.211+0000 7f11337fe700 1 -- 172.21.15.70:0/2583670951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1134106580 msgr2=0x7f1134073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.211+0000 7f11337fe700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1134106580 0x7f1134073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.211+0000 7f11337fe700 1 -- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11341004c0 msgr2=0x7f1134072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.211+0000 7f11337fe700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11341004c0 0x7f1134072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.211+0000 7f11337fe700 1 -- 172.21.15.70:0/2583670951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280095f0 con 0x7f1134100e80 2023-10-18T12:38:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.211+0000 7f11337fe700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f1134073030 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1128005910 tx=0x7f1128004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.212+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112801d070 con 0x7f1134100e80 2023-10-18T12:38:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.212+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1128022470 con 0x7f1134100e80 2023-10-18T12:38:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.212+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11341a5230 con 0x7f1134100e80 2023-10-18T12:38:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.212+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128017650 con 0x7f1134100e80 2023-10-18T12:38:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.212+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11341a56b0 con 0x7f1134100e80 2023-10-18T12:38:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.213+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134108a20 con 0x7f1134100e80 2023-10-18T12:38:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.215+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f11280225e0 con 0x7f1134100e80 2023-10-18T12:38:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.216+0000 7f11317fa700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f111c0a1670 0x7f111c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.217+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f11280ce670 con 0x7f1134100e80 2023-10-18T12:38:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.217+0000 7f1133fff700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f111c0a1670 0x7f111c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.217+0000 7f1133fff700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f111c0a1670 0x7f111c0a3b30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1124009c60 tx=0x7f1124009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.222+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1128097660 con 0x7f1134100e80 2023-10-18T12:38:26.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.529+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1134101be0 con 0x7f1134100e80 2023-10-18T12:38:26.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.530+0000 7f11317fa700 1 -- 172.21.15.70:0/2583670951 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f112809ac80 con 0x7f1134100e80 2023-10-18T12:38:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:38:26.533 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:38:26.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f111c0a1670 msgr2=0x7f111c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f111c0a1670 0x7f111c0a3b30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1124009c60 tx=0x7f1124009260 comp rx=0 tx=0).stop 2023-10-18T12:38:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 msgr2=0x7f1134073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f1134073030 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1128005910 tx=0x7f1128004730 comp rx=0 tx=0).stop 2023-10-18T12:38:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 shutdown_connections 2023-10-18T12:38:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1134106580 0x7f1134073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11341004c0 0x7f1134072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f111c0a1670 0x7f111c0a3b30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 --2- 172.21.15.70:0/2583670951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1134100e80 0x7f1134073030 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 >> 172.21.15.70:0/2583670951 conn(0x7f11340fbfb0 msgr2=0x7f11340fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 shutdown_connections 2023-10-18T12:38:26.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.533+0000 7f113959d700 1 -- 172.21.15.70:0/2583670951 wait complete. 2023-10-18T12:38:26.539 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:38:26.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:38:26.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 -- 172.21.15.70:0/2094850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 msgr2=0x7f07b4106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2094850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4106b10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f07a400b0a0 tx=0x7f07a400b3b0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 -- 172.21.15.70:0/2094850026 shutdown_connections 2023-10-18T12:38:26.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2094850026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b41010e0 0x7f07b410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2094850026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4106b10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2094850026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b4100720 0x7f07b4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 -- 172.21.15.70:0/2094850026 >> 172.21.15.70:0/2094850026 conn(0x7f07b40fc1b0 msgr2=0x7f07b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 -- 172.21.15.70:0/2094850026 shutdown_connections 2023-10-18T12:38:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.771+0000 7f07b8dff700 1 -- 172.21.15.70:0/2094850026 wait complete. 2023-10-18T12:38:26.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.772+0000 7f07b8dff700 1 Processor -- start 2023-10-18T12:38:26.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.772+0000 7f07b8dff700 1 -- start start 2023-10-18T12:38:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.772+0000 7f07b8dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 0x7f07b419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.772+0000 7f07b8dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 0x7f07b419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.772+0000 7f07b8dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b8dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b4199200 con 0x7f07b4106730 2023-10-18T12:38:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b8dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b4199370 con 0x7f07b41010e0 2023-10-18T12:38:26.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b8dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b4199530 con 0x7f07b4100720 2023-10-18T12:38:26.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b1d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 0x7f07b419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 0x7f07b419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b2d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 0x7f07b419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:44986/0 (socket says 172.21.15.70:44986) 2023-10-18T12:38:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b1d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 0x7f07b419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46762/0 (socket says 172.21.15.70:46762) 2023-10-18T12:38:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 -- 172.21.15.70:0/2866446341 learned_addr learned my addr 172.21.15.70:0/2866446341 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 -- 172.21.15.70:0/2866446341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 msgr2=0x7f07b419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 0x7f07b419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 -- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 msgr2=0x7f07b4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b259c700 1 -- 172.21.15.70:0/2866446341 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4009d00 con 0x7f07b4100720 2023-10-18T12:38:26.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 0x7f07b419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:26.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.773+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:26.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.774+0000 7f07b259c700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 0x7f07b419ea70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07a400b070 tx=0x7f07a40075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.774+0000 7f07a37fe700 1 -- 172.21.15.70:0/2866446341 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400e040 con 0x7f07b4100720 2023-10-18T12:38:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.774+0000 7f07a37fe700 1 -- 172.21.15.70:0/2866446341 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a4007980 con 0x7f07b4100720 2023-10-18T12:38:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.774+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b41997b0 con 0x7f07b4100720 2023-10-18T12:38:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.774+0000 7f07a37fe700 1 -- 172.21.15.70:0/2866446341 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a401b6c0 con 0x7f07b4100720 2023-10-18T12:38:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.774+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b41a5a60 con 0x7f07b4100720 2023-10-18T12:38:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.777+0000 7f07a37fe700 1 -- 172.21.15.70:0/2866446341 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f07a401d470 con 0x7f07b4100720 2023-10-18T12:38:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 Processor -- start 2023-10-18T12:38:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07a37fe700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f079c0a1670 0x7f079c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 -- start start 2023-10-18T12:38:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07a37fe700 1 -- 172.21.15.70:0/2866446341 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f07a40ce460 con 0x7f07b4100720 2023-10-18T12:38:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f079c0a1670 0x7f079c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 0x55587c691fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c68ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c5ff6e0 con 0x55587c691be0 2023-10-18T12:38:26.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c5e5c60 con 0x55587c68c5d0 2023-10-18T12:38:26.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.780+0000 7f07cc70db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c5e3580 con 0x55587c692500 2023-10-18T12:38:26.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 0x55587c691fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b2d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c68ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 0x55587c691fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56346/0 (socket says 172.21.15.70:56346) 2023-10-18T12:38:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 -- 172.21.15.70:0/3223355704 learned_addr learned my addr 172.21.15.70:0/3223355704 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 -- 172.21.15.70:0/3223355704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 msgr2=0x55587c68c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 -- 172.21.15.70:0/3223355704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 msgr2=0x55587c68ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c68ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 -- 172.21.15.70:0/3223355704 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55587c59c3f0 con 0x55587c691be0 2023-10-18T12:38:26.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f079c0a1670 0x7f079c0a3b30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00bfd0 tx=0x7f07ac014c50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07b259c700 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 0x55587c691fc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f07a40a2ee0 tx=0x7f07a409aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220126886b10f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.781+0000 7f07a0ff9700 1 -- 172.21.15.70:0/3223355704 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4003c00 con 0x55587c691be0 2023-10-18T12:38:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07a0ff9700 1 -- 172.21.15.70:0/3223355704 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a409b810 con 0x55587c691be0 2023-10-18T12:38:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07a0ff9700 1 -- 172.21.15.70:0/3223355704 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a401d720 con 0x55587c691be0 2023-10-18T12:38:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 -- 172.21.15.70:0/3223355704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 msgr2=0x55587c691fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 0x55587c691fc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f07a40a2ee0 tx=0x7f07a409aaf0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 -- 172.21.15.70:0/3223355704 shutdown_connections 2023-10-18T12:38:26.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c68ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c691be0 0x55587c691fc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 --2- 172.21.15.70:0/3223355704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 -- 172.21.15.70:0/3223355704 >> 172.21.15.70:0/3223355704 conn(0x55587c600720 msgr2=0x55587c686b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.782+0000 7f07cc70db80 1 -- 172.21.15.70:0/3223355704 shutdown_connections 2023-10-18T12:38:26.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.783+0000 7f07cc70db80 1 -- 172.21.15.70:0/3223355704 wait complete. 2023-10-18T12:38:26.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.783+0000 7f07cc70db80 1 -- start start 2023-10-18T12:38:26.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.783+0000 7f07cc70db80 1 Processor -- start 2023-10-18T12:38:26.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c695790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c691be0 0x55587c695cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c692500 0x55587c696210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c5dee70 con 0x55587c692500 2023-10-18T12:38:26.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c57b710 con 0x55587c68c5d0 2023-10-18T12:38:26.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c57b3b0 con 0x55587c691be0 2023-10-18T12:38:26.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07cc70db80 10 client.0 _ll_register_callbacks cb 0x55587c4c5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:26.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c695790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c695790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46790/0 (socket says 172.21.15.70:46790) 2023-10-18T12:38:26.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 -- 172.21.15.70:0/4253909350 learned_addr learned my addr 172.21.15.70:0/4253909350 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:26.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c692500 0x55587c696210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c691be0 0x55587c695cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 -- 172.21.15.70:0/4253909350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c691be0 msgr2=0x55587c695cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c691be0 0x55587c695cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 -- 172.21.15.70:0/4253909350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c692500 msgr2=0x55587c696210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c692500 0x55587c696210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b259c700 1 -- 172.21.15.70:0/4253909350 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b41a5a60 con 0x55587c68c5d0 2023-10-18T12:38:26.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.784+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c692500 0x55587c696210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:26.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f07b259c700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c695790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07a4097b80 tx=0x7f07a40ce330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd64f10766c20ba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f0799ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:38:26.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f0799ffb700 1 -- 172.21.15.70:0/4253909350 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a40cbc80 con 0x55587c68c5d0 2023-10-18T12:38:26.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f0799ffb700 1 -- 172.21.15.70:0/4253909350 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f07a409b110 con 0x55587c68c5d0 2023-10-18T12:38:26.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55587c5e3ef0 con 0x55587c68c5d0 2023-10-18T12:38:26.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f0799ffb700 1 -- 172.21.15.70:0/4253909350 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a40a3410 con 0x55587c68c5d0 2023-10-18T12:38:26.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f07cc70db80 10 client.27362 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:26.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55587c409de0 con 0x55587c68c5d0 2023-10-18T12:38:26.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f0799ffb700 1 -- 172.21.15.70:0/4253909350 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a407d080 con 0x55587c68c5d0 2023-10-18T12:38:26.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f0799ffb700 1 -- 172.21.15.70:0/4253909350 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07a401ac70 con 0x55587c68c5d0 2023-10-18T12:38:26.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f07cc70db80 10 client.27362 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:26.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.785+0000 7f07cc70db80 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:26.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f07cc70db80 20 client.27362 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:26.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f07cc70db80 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6a57f0 0x55587c6a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f07cc70db80 4 client.27362 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:26.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55587c65b8b0 con 0x55587c6a57f0 2023-10-18T12:38:26.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6a57f0 0x55587c6a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6a57f0 0x55587c6a7be0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a79d1087d33022 server_cookie=1e30f1ec45f2547c in_seq=0 out_seq=0 2023-10-18T12:38:26.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.786+0000 7f0799ffb700 10 client.27362 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:26.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.787+0000 7f0799ffb700 1 -- 172.21.15.70:0/4253909350 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55587c65b8b0 con 0x55587c6a57f0 2023-10-18T12:38:26.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.787+0000 7f0799ffb700 10 client.27362 handle_command_reply: tid=0 2023-10-18T12:38:26.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.787+0000 7f07cc70db80 1 client.27362 shutdown 2023-10-18T12:38:26.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 msgr2=0x55587c695790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c695790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07a4097b80 tx=0x7f07a40ce330 comp rx=0 tx=0).stop 2023-10-18T12:38:26.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 shutdown_connections 2023-10-18T12:38:26.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6a57f0 0x55587c6a7be0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c692500 0x55587c696210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c68c5d0 0x55587c695790 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 --2- 172.21.15.70:0/4253909350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c691be0 0x55587c695cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f0799ffb700 0 client.27362 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:26.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 >> 172.21.15.70:0/4253909350 conn(0x55587c600720 msgr2=0x55587c68b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 shutdown_connections 2023-10-18T12:38:26.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 1 -- 172.21.15.70:0/4253909350 wait complete. 2023-10-18T12:38:26.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.788+0000 7f07cc70db80 20 client.27362 trim_cache size 0 max 16384 2023-10-18T12:38:26.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 Processor -- start 2023-10-18T12:38:26.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 -- start start 2023-10-18T12:38:26.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c6815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c69ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c45f920 con 0x55587c698040 2023-10-18T12:38:26.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c690370 con 0x55587c691be0 2023-10-18T12:38:26.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.802+0000 7f07cc70db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c696e00 con 0x55587c692500 2023-10-18T12:38:26.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c6815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c6815d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:46796/0 (socket says 172.21.15.70:46796) 2023-10-18T12:38:26.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 -- 172.21.15.70:0/1419775606 learned_addr learned my addr 172.21.15.70:0/1419775606 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:26.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c69ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 -- 172.21.15.70:0/1419775606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 msgr2=0x55587c68fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 -- 172.21.15.70:0/1419775606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 msgr2=0x55587c69ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c69ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 -- 172.21.15.70:0/1419775606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55587c3f2c60 con 0x55587c691be0 2023-10-18T12:38:26.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07b259c700 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c6815d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f07a409b0a0 tx=0x7f07a40074e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786ee2f80e344777 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07a17fa700 1 -- 172.21.15.70:0/1419775606 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a40a3410 con 0x55587c691be0 2023-10-18T12:38:26.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07a17fa700 1 -- 172.21.15.70:0/1419775606 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55587c5e3ef0 con 0x55587c691be0 2023-10-18T12:38:26.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.803+0000 7f07a17fa700 1 -- 172.21.15.70:0/1419775606 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a401ac70 con 0x55587c691be0 2023-10-18T12:38:26.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 -- 172.21.15.70:0/1419775606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 msgr2=0x55587c6815d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c6815d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f07a409b0a0 tx=0x7f07a40074e0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 -- 172.21.15.70:0/1419775606 shutdown_connections 2023-10-18T12:38:26.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c69ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c6815d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 --2- 172.21.15.70:0/1419775606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c68fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 -- 172.21.15.70:0/1419775606 >> 172.21.15.70:0/1419775606 conn(0x55587c600720 msgr2=0x55587c68f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 -- 172.21.15.70:0/1419775606 shutdown_connections 2023-10-18T12:38:26.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.804+0000 7f07cc70db80 1 -- 172.21.15.70:0/1419775606 wait complete. 2023-10-18T12:38:26.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 -- start start 2023-10-18T12:38:26.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 Processor -- start 2023-10-18T12:38:26.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c69ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c69b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c6f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c6f8b00 con 0x55587c698040 2023-10-18T12:38:26.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c6f8c40 con 0x55587c691be0 2023-10-18T12:38:26.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587c6f8dd0 con 0x55587c692500 2023-10-18T12:38:26.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07cc70db80 10 client.0 _ll_register_callbacks cb 0x55587c4c5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:26.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07b259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c69ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.805+0000 7f07b1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c69b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c69b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:45018/0 (socket says 172.21.15.70:45018) 2023-10-18T12:38:26.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 -- 172.21.15.70:0/2766547260 learned_addr learned my addr 172.21.15.70:0/2766547260 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:26.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b2d9d700 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c6f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 -- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 msgr2=0x55587c69ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c69ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 -- 172.21.15.70:0/2766547260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 msgr2=0x55587c6f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:26.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c6f85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 -- 172.21.15.70:0/2766547260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55587c409de0 con 0x55587c692500 2023-10-18T12:38:26.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f07b1d9b700 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c69b190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f07b419a240 tx=0x7f07a8020d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b445c507d03e5f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:26.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.806+0000 7f0799ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:38:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f0799ffb700 1 -- 172.21.15.70:0/2766547260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8017070 con 0x55587c692500 2023-10-18T12:38:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07a409b810 con 0x55587c692500 2023-10-18T12:38:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f0799ffb700 1 -- 172.21.15.70:0/2766547260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f07a8004a30 con 0x55587c692500 2023-10-18T12:38:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f0799ffb700 1 -- 172.21.15.70:0/2766547260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a8020e70 con 0x55587c692500 2023-10-18T12:38:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f07cc70db80 10 client.27352 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55587c430250 con 0x55587c692500 2023-10-18T12:38:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f0799ffb700 1 -- 172.21.15.70:0/2766547260 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a409b810 con 0x55587c692500 2023-10-18T12:38:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.807+0000 7f0799ffb700 1 -- 172.21.15.70:0/2766547260 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07a80107d0 con 0x55587c692500 2023-10-18T12:38:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07cc70db80 10 client.27352 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07cc70db80 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:26.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07cc70db80 20 client.27352 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:26.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07cc70db80 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6fa640 0x55587c6fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:26.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07cc70db80 4 client.27352 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:38:26.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55587c5d7e50 con 0x55587c6fa640 2023-10-18T12:38:26.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07b259c700 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6fa640 0x55587c6fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:26.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f07b259c700 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6fa640 0x55587c6fca60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d6834d7f365bd2 server_cookie=61435612928404dc in_seq=0 out_seq=0 2023-10-18T12:38:26.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.808+0000 7f0799ffb700 10 client.27352 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:26.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.809+0000 7f0799ffb700 1 -- 172.21.15.70:0/2766547260 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55587c5d7e50 con 0x55587c6fa640 2023-10-18T12:38:26.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.809+0000 7f0799ffb700 10 client.27352 handle_command_reply: tid=0 2023-10-18T12:38:26.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.809+0000 7f07cc70db80 1 client.27352 shutdown 2023-10-18T12:38:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 msgr2=0x55587c69b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c69b190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f07b419a240 tx=0x7f07a8020d40 comp rx=0 tx=0).stop 2023-10-18T12:38:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 shutdown_connections 2023-10-18T12:38:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55587c6fa640 0x55587c6fca60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55587c698040 0x55587c6f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55587c691be0 0x55587c69ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 --2- 172.21.15.70:0/2766547260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55587c692500 0x55587c69b190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f0799ffb700 0 client.27352 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 >> 172.21.15.70:0/2766547260 conn(0x55587c600720 msgr2=0x55587c6960b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 shutdown_connections 2023-10-18T12:38:26.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 1 -- 172.21.15.70:0/2766547260 wait complete. 2023-10-18T12:38:26.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07cc70db80 20 client.27352 trim_cache size 0 max 16384 2023-10-18T12:38:26.829 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:38:26.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f079c0a1670 msgr2=0x7f079c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f079c0a1670 0x7f079c0a3b30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00bfd0 tx=0x7f07ac014c50 comp rx=0 tx=0).stop 2023-10-18T12:38:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 msgr2=0x7f07b419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.810+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 0x7f07b419ea70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07a400b070 tx=0x7f07a40075c0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 shutdown_connections 2023-10-18T12:38:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b4106730 0x7f07b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07b4100720 0x7f07b419ea70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f079c0a1670 0x7f079c0a3b30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 --2- 172.21.15.70:0/2866446341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07b41010e0 0x7f07b419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 >> 172.21.15.70:0/2866446341 conn(0x7f07b40fc1b0 msgr2=0x7f07b410bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 shutdown_connections 2023-10-18T12:38:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:26.811+0000 7f07b8dff700 1 -- 172.21.15.70:0/2866446341 wait complete. 2023-10-18T12:38:26.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1394756 11.03 MB/sec execute 535 sec latency 15.155 ms 2023-10-18T12:38:27.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:26 smithi070 ceph-mon[98639]: pgmap v699: 129 pgs: 129 active+clean; 597 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 193 op/s 2023-10-18T12:38:27.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:26 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2583670951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:38:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:26 smithi152 ceph-mon[134767]: pgmap v699: 129 pgs: 129 active+clean; 597 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 193 op/s 2023-10-18T12:38:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:26 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2583670951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:38:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:26 smithi186 ceph-mon[135253]: pgmap v699: 129 pgs: 129 active+clean; 597 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 193 op/s 2023-10-18T12:38:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:26 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2583670951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:38:27.835 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:38:27.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:38:27.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1396504 11.03 MB/sec execute 536 sec latency 64.112 ms 2023-10-18T12:38:28.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 -- 172.21.15.70:0/2200558301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48381010e0 msgr2=0x7f483810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 --2- 172.21.15.70:0/2200558301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48381010e0 0x7f483810d5f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f483000b0a0 tx=0x7f483000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 -- 172.21.15.70:0/2200558301 shutdown_connections 2023-10-18T12:38:28.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 --2- 172.21.15.70:0/2200558301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48381010e0 0x7f483810d5f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 --2- 172.21.15.70:0/2200558301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4838106730 0x7f4838106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 --2- 172.21.15.70:0/2200558301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 -- 172.21.15.70:0/2200558301 >> 172.21.15.70:0/2200558301 conn(0x7f48380fc1b0 msgr2=0x7f48380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 -- 172.21.15.70:0/2200558301 shutdown_connections 2023-10-18T12:38:28.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.050+0000 7f483f3ff700 1 -- 172.21.15.70:0/2200558301 wait complete. 2023-10-18T12:38:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.051+0000 7f483f3ff700 1 Processor -- start 2023-10-18T12:38:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.051+0000 7f483f3ff700 1 -- start start 2023-10-18T12:38:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483f3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483f3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48381010e0 0x7f4838073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483f3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4838106730 0x7f4838073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483f3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838074ac0 con 0x7f4838100720 2023-10-18T12:38:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483f3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a9530 con 0x7f4838106730 2023-10-18T12:38:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483f3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a96f0 con 0x7f48381010e0 2023-10-18T12:38:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34798/0 (socket says 172.21.15.70:34798) 2023-10-18T12:38:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d19b700 1 -- 172.21.15.70:0/3703123740 learned_addr learned my addr 172.21.15.70:0/3703123740 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d99c700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4838106730 0x7f4838073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483c99a700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48381010e0 0x7f4838073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d99c700 1 -- 172.21.15.70:0/3703123740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48381010e0 msgr2=0x7f4838073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d99c700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48381010e0 0x7f4838073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d99c700 1 -- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 msgr2=0x7f4838072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d99c700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.052+0000 7f483d99c700 1 -- 172.21.15.70:0/3703123740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830009d00 con 0x7f4838106730 2023-10-18T12:38:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f483d19b700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f483d99c700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4838106730 0x7f4838073570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f483000b070 tx=0x7f48300077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f482e7fc700 1 -- 172.21.15.70:0/3703123740 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000e040 con 0x7f4838106730 2023-10-18T12:38:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f482e7fc700 1 -- 172.21.15.70:0/3703123740 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4830004e60 con 0x7f4838106730 2023-10-18T12:38:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48381a9970 con 0x7f4838106730 2023-10-18T12:38:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f482e7fc700 1 -- 172.21.15.70:0/3703123740 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830005040 con 0x7f4838106730 2023-10-18T12:38:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.053+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48381a9df0 con 0x7f4838106730 2023-10-18T12:38:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.056+0000 7f482e7fc700 1 -- 172.21.15.70:0/3703123740 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f483001d470 con 0x7f4838106730 2023-10-18T12:38:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.058+0000 7f482e7fc700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48240a15f0 0x7f48240a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.058+0000 7f482e7fc700 1 -- 172.21.15.70:0/3703123740 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f48300ce920 con 0x7f4838106730 2023-10-18T12:38:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.058+0000 7f483d19b700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48240a15f0 0x7f48240a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.058+0000 7f4852d8fb80 1 Processor -- start 2023-10-18T12:38:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 -- start start 2023-10-18T12:38:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483d19b700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48240a15f0 0x7f48240a3ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4834011550 tx=0x7f4834009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081aac60 0x5628081ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x5628081a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562808118760 con 0x5628081aac60 2023-10-18T12:38:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628080fece0 con 0x5628081a5650 2023-10-18T12:38:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628080fc600 con 0x5628081ab580 2023-10-18T12:38:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081aac60 0x5628081ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483d99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483d99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081a5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51514/0 (socket says 172.21.15.70:51514) 2023-10-18T12:38:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483d99c700 1 -- 172.21.15.70:0/423743838 learned_addr learned my addr 172.21.15.70:0/423743838 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483c99a700 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x5628081a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.059+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081aac60 0x5628081ab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34814/0 (socket says 172.21.15.70:34814) 2023-10-18T12:38:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f483d99c700 1 -- 172.21.15.70:0/423743838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 msgr2=0x5628081a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f483d99c700 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x5628081a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f483d99c700 1 -- 172.21.15.70:0/423743838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081aac60 msgr2=0x5628081ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f483d99c700 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081aac60 0x5628081ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f483d99c700 1 -- 172.21.15.70:0/423743838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628080b5470 con 0x5628081a5650 2023-10-18T12:38:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f483d99c700 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081a5ad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f48300a33a0 tx=0x7f483009b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f10582ad7d7def2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f48237fe700 1 -- 172.21.15.70:0/423743838 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830009450 con 0x5628081a5650 2023-10-18T12:38:28.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f48237fe700 1 -- 172.21.15.70:0/423743838 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4830011e50 con 0x5628081a5650 2023-10-18T12:38:28.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.060+0000 7f48237fe700 1 -- 172.21.15.70:0/423743838 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300cbf30 con 0x5628081a5650 2023-10-18T12:38:28.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 -- 172.21.15.70:0/423743838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 msgr2=0x5628081a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081a5ad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f48300a33a0 tx=0x7f483009b3a0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 -- 172.21.15.70:0/423743838 shutdown_connections 2023-10-18T12:38:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081a5ad0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081aac60 0x5628081ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/423743838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x5628081a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 -- 172.21.15.70:0/423743838 >> 172.21.15.70:0/423743838 conn(0x5628081197a0 msgr2=0x56280819fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 -- 172.21.15.70:0/423743838 shutdown_connections 2023-10-18T12:38:28.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.061+0000 7f4852d8fb80 1 -- 172.21.15.70:0/423743838 wait complete. 2023-10-18T12:38:28.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 -- start start 2023-10-18T12:38:28.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 Processor -- start 2023-10-18T12:38:28.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x5628081aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081ab580 0x5628081af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628080f7ef0 con 0x5628081ab580 2023-10-18T12:38:28.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562808094790 con 0x5628081a5650 2023-10-18T12:38:28.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.062+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562808094430 con 0x5628081aac60 2023-10-18T12:38:28.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f4852d8fb80 10 client.0 _ll_register_callbacks cb 0x562807fdeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:28.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51530/0 (socket says 172.21.15.70:51530) 2023-10-18T12:38:28.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081ab580 0x5628081af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 -- 172.21.15.70:0/2042069502 learned_addr learned my addr 172.21.15.70:0/2042069502 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483c99a700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x5628081aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 -- 172.21.15.70:0/2042069502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 msgr2=0x5628081aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x5628081aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 -- 172.21.15.70:0/2042069502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081ab580 msgr2=0x5628081af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081ab580 0x5628081af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 -- 172.21.15.70:0/2042069502 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48381a9df0 con 0x5628081a5650 2023-10-18T12:38:28.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d99c700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081ab580 0x5628081af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f483d19b700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081ae970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f483400edf0 tx=0x7f48340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e5d20bc2eeabe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.063+0000 7f4820ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:38:28.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4820ff9700 1 -- 172.21.15.70:0/2042069502 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400ee20 con 0x5628081a5650 2023-10-18T12:38:28.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628080fcf70 con 0x5628081a5650 2023-10-18T12:38:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4820ff9700 1 -- 172.21.15.70:0/2042069502 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f483401fdb0 con 0x5628081a5650 2023-10-18T12:38:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4820ff9700 1 -- 172.21.15.70:0/2042069502 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4834028440 con 0x5628081a5650 2023-10-18T12:38:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4852d8fb80 10 client.27380 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562807f22e60 con 0x5628081a5650 2023-10-18T12:38:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4820ff9700 1 -- 172.21.15.70:0/2042069502 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628080fcf70 con 0x5628081a5650 2023-10-18T12:38:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4820ff9700 1 -- 172.21.15.70:0/2042069502 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4834027640 con 0x5628081a5650 2023-10-18T12:38:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4852d8fb80 10 client.27380 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4852d8fb80 10 client.27380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.064+0000 7f4852d8fb80 20 client.27380 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.065+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5628081bef70 0x5628081c1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.065+0000 7f4852d8fb80 4 client.27380 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:28.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.065+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562808174930 con 0x5628081bef70 2023-10-18T12:38:28.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.065+0000 7f483c99a700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5628081bef70 0x5628081c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.065+0000 7f483c99a700 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5628081bef70 0x5628081c1390 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b3933b9242cd5f server_cookie=a26b25f4c767c9f7 in_seq=0 out_seq=0 2023-10-18T12:38:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.065+0000 7f4820ff9700 10 client.27380 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4820ff9700 1 -- 172.21.15.70:0/2042069502 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562808174930 con 0x5628081bef70 2023-10-18T12:38:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4820ff9700 10 client.27380 handle_command_reply: tid=0 2023-10-18T12:38:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 client.27380 shutdown 2023-10-18T12:38:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 msgr2=0x5628081ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081ae970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f483400edf0 tx=0x7f48340047c0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 shutdown_connections 2023-10-18T12:38:28.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5628081bef70 0x5628081c1390 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081ab580 0x5628081af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081a5650 0x5628081ae970 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/2042069502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x5628081aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4820ff9700 0 client.27380 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 >> 172.21.15.70:0/2042069502 conn(0x5628081197a0 msgr2=0x5628081a4100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 shutdown_connections 2023-10-18T12:38:28.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.066+0000 7f4852d8fb80 1 -- 172.21.15.70:0/2042069502 wait complete. 2023-10-18T12:38:28.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.067+0000 7f4852d8fb80 20 client.27380 trim_cache size 0 max 16384 2023-10-18T12:38:28.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 Processor -- start 2023-10-18T12:38:28.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 -- start start 2023-10-18T12:38:28.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x5628081b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x56280819f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 0x5628081a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562807f789a0 con 0x5628081a5650 2023-10-18T12:38:28.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628081a1430 con 0x5628081ab580 2023-10-18T12:38:28.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.087+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628081aff70 con 0x5628081aac60 2023-10-18T12:38:28.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x5628081b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 0x5628081a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x5628081b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34824/0 (socket says 172.21.15.70:34824) 2023-10-18T12:38:28.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d19b700 1 -- 172.21.15.70:0/960586250 learned_addr learned my addr 172.21.15.70:0/960586250 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483c99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x56280819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 0x5628081a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51546/0 (socket says 172.21.15.70:51546) 2023-10-18T12:38:28.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 -- 172.21.15.70:0/960586250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 msgr2=0x56280819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x56280819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 -- 172.21.15.70:0/960586250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 msgr2=0x5628081b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x5628081b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 -- 172.21.15.70:0/960586250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628081c5f80 con 0x5628081ab580 2023-10-18T12:38:28.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d19b700 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x5628081b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f483d99c700 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 0x5628081a4b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f483009b790 tx=0x7f483000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d934eb655dc46b21 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f4823fff700 1 -- 172.21.15.70:0/960586250 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830009450 con 0x5628081ab580 2023-10-18T12:38:28.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f4823fff700 1 -- 172.21.15.70:0/960586250 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f483009b5d0 con 0x5628081ab580 2023-10-18T12:38:28.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.088+0000 7f4823fff700 1 -- 172.21.15.70:0/960586250 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830099a90 con 0x5628081ab580 2023-10-18T12:38:28.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 -- 172.21.15.70:0/960586250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 msgr2=0x5628081a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 0x5628081a4b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f483009b790 tx=0x7f483000b3b0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 -- 172.21.15.70:0/960586250 shutdown_connections 2023-10-18T12:38:28.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081ab580 0x5628081a4b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x5628081b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/960586250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081aac60 0x56280819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 -- 172.21.15.70:0/960586250 >> 172.21.15.70:0/960586250 conn(0x5628081197a0 msgr2=0x5628081af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 -- 172.21.15.70:0/960586250 shutdown_connections 2023-10-18T12:38:28.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.089+0000 7f4852d8fb80 1 -- 172.21.15.70:0/960586250 wait complete. 2023-10-18T12:38:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.090+0000 7f4852d8fb80 1 -- start start 2023-10-18T12:38:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.090+0000 7f4852d8fb80 1 Processor -- start 2023-10-18T12:38:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x56280820c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081aac60 0x56280820cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x562808206cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628082072c0 con 0x5628081a5650 2023-10-18T12:38:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562808207430 con 0x5628081aac60 2023-10-18T12:38:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628082075f0 con 0x5628081ab580 2023-10-18T12:38:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f4852d8fb80 10 client.0 _ll_register_callbacks cb 0x562807fdeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x56280820c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483c99a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081aac60 0x56280820cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x56280820c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34828/0 (socket says 172.21.15.70:34828) 2023-10-18T12:38:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 -- 172.21.15.70:0/845543352 learned_addr learned my addr 172.21.15.70:0/845543352 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d99c700 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x562808206cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 -- 172.21.15.70:0/845543352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 msgr2=0x562808206cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x562808206cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 -- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081aac60 msgr2=0x56280820cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081aac60 0x56280820cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.091+0000 7f483d19b700 1 -- 172.21.15.70:0/845543352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562807f22e60 con 0x5628081a5650 2023-10-18T12:38:28.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f483d19b700 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x56280820c610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4834027af0 tx=0x7f483400e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b41dbd5b4887be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4820ff9700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:38:28.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4820ff9700 1 -- 172.21.15.70:0/845543352 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834004830 con 0x5628081a5650 2023-10-18T12:38:28.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4820ff9700 1 -- 172.21.15.70:0/845543352 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f4834004990 con 0x5628081a5650 2023-10-18T12:38:28.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4830011e50 con 0x5628081a5650 2023-10-18T12:38:28.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4820ff9700 1 -- 172.21.15.70:0/845543352 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4834004b50 con 0x5628081a5650 2023-10-18T12:38:28.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4852d8fb80 10 client.17337 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:28.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5628080b5470 con 0x5628081a5650 2023-10-18T12:38:28.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.092+0000 7f4820ff9700 1 -- 172.21.15.70:0/845543352 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4830011e50 con 0x5628081a5650 2023-10-18T12:38:28.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4820ff9700 1 -- 172.21.15.70:0/845543352 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f483403f060 con 0x5628081a5650 2023-10-18T12:38:28.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4852d8fb80 10 client.17337 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:28.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4852d8fb80 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:28.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4852d8fb80 20 client.17337 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562808214610 0x562808216a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4852d8fb80 4 client.17337 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:38:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5628080f0ed0 con 0x562808214610 2023-10-18T12:38:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f483c99a700 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562808214610 0x562808216a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f483c99a700 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562808214610 0x562808216a30 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa742883b651bc8b server_cookie=39b312a1d04d0750 in_seq=0 out_seq=0 2023-10-18T12:38:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.093+0000 7f4820ff9700 10 client.17337 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.094+0000 7f4820ff9700 1 -- 172.21.15.70:0/845543352 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5628080f0ed0 con 0x562808214610 2023-10-18T12:38:28.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.094+0000 7f4820ff9700 10 client.17337 handle_command_reply: tid=0 2023-10-18T12:38:28.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.094+0000 7f4852d8fb80 1 client.17337 shutdown 2023-10-18T12:38:28.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 msgr2=0x56280820c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x56280820c610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4834027af0 tx=0x7f483400e4c0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 shutdown_connections 2023-10-18T12:38:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562808214610 0x562808216a30 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5628081ab580 0x562808206cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5628081a5650 0x56280820c610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 --2- 172.21.15.70:0/845543352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628081aac60 0x56280820cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 >> 172.21.15.70:0/845543352 conn(0x5628081197a0 msgr2=0x56280819f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4820ff9700 0 client.17337 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 shutdown_connections 2023-10-18T12:38:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 1 -- 172.21.15.70:0/845543352 wait complete. 2023-10-18T12:38:28.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.095+0000 7f4852d8fb80 20 client.17337 trim_cache size 0 max 16384 2023-10-18T12:38:28.112 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:38:28.112 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:38:28.112 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "a5b78874-6a1b-4f7c-a5c5-007555f60216", 2023-10-18T12:38:28.113 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:38:28.113 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:38:28.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48240a15f0 msgr2=0x7f48240a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48240a15f0 0x7f48240a3ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4834011550 tx=0x7f4834009260 comp rx=0 tx=0).stop 2023-10-18T12:38:28.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4838106730 msgr2=0x7f4838073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4838106730 0x7f4838073570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f483000b070 tx=0x7f48300077f0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 shutdown_connections 2023-10-18T12:38:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f48240a15f0 0x7f48240a3ab0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4838106730 0x7f4838073570 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4838100720 0x7f4838072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 --2- 172.21.15.70:0/3703123740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48381010e0 0x7f4838073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 >> 172.21.15.70:0/3703123740 conn(0x7f48380fc1b0 msgr2=0x7f48380fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 shutdown_connections 2023-10-18T12:38:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.096+0000 7f483f3ff700 1 -- 172.21.15.70:0/3703123740 wait complete. 2023-10-18T12:38:28.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:38:28.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.340+0000 7f940cd9d700 1 -- 172.21.15.70:0/1870413894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 msgr2=0x7f940810d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.340+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1870413894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 0x7f940810d490 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9400009f90 tx=0x7f940000b330 comp rx=0 tx=0).stop 2023-10-18T12:38:28.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 -- 172.21.15.70:0/1870413894 shutdown_connections 2023-10-18T12:38:28.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1870413894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 0x7f940810d490 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1870413894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9408106660 0x7f9408106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1870413894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94081005c0 0x7f9408100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 -- 172.21.15.70:0/1870413894 >> 172.21.15.70:0/1870413894 conn(0x7f94080fc0f0 msgr2=0x7f94080fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 -- 172.21.15.70:0/1870413894 shutdown_connections 2023-10-18T12:38:28.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.341+0000 7f940cd9d700 1 -- 172.21.15.70:0/1870413894 wait complete. 2023-10-18T12:38:28.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.342+0000 7f940cd9d700 1 Processor -- start 2023-10-18T12:38:28.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.342+0000 7f940cd9d700 1 -- start start 2023-10-18T12:38:28.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f940cd9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 0x7f940819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f940cd9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 0x7f940819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f940cd9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408106660 0x7f94081966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f940cd9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408196ec0 con 0x7f94081005c0 2023-10-18T12:38:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f940cd9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408197030 con 0x7f9408100f80 2023-10-18T12:38:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f940cd9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94081971f0 con 0x7f9408106660 2023-10-18T12:38:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 0x7f940819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f9406ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 0x7f940819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 0x7f940819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34858/0 (socket says 172.21.15.70:34858) 2023-10-18T12:38:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f9407fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408106660 0x7f94081966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 -- 172.21.15.70:0/1079596048 learned_addr learned my addr 172.21.15.70:0/1079596048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 -- 172.21.15.70:0/1079596048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408106660 msgr2=0x7f94081966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408106660 0x7f94081966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 -- 172.21.15.70:0/1079596048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 msgr2=0x7f940819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 0x7f940819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.343+0000 7f94077fe700 1 -- 172.21.15.70:0/1079596048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80095f0 con 0x7f94081005c0 2023-10-18T12:38:28.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.344+0000 7f94077fe700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 0x7f940819c650 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f93f800ebf0 tx=0x7f93f800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.344+0000 7f9404ff9700 1 -- 172.21.15.70:0/1079596048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8009a10 con 0x7f94081005c0 2023-10-18T12:38:28.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.344+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9400009c40 con 0x7f94081005c0 2023-10-18T12:38:28.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.344+0000 7f9404ff9700 1 -- 172.21.15.70:0/1079596048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f800cc90 con 0x7f94081005c0 2023-10-18T12:38:28.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.344+0000 7f9404ff9700 1 -- 172.21.15.70:0/1079596048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8018750 con 0x7f94081005c0 2023-10-18T12:38:28.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.344+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94081a35d0 con 0x7f94081005c0 2023-10-18T12:38:28.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.347+0000 7f9404ff9700 1 -- 172.21.15.70:0/1079596048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f93f80188b0 con 0x7f94081005c0 2023-10-18T12:38:28.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.349+0000 7f9421935b80 1 Processor -- start 2023-10-18T12:38:28.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.349+0000 7f9421935b80 1 -- start start 2023-10-18T12:38:28.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9421935b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9421935b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9421935b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1edc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9404ff9700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93f00a15f0 0x7f93f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9421935b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d1606e0 con 0x55da4d1ed820 2023-10-18T12:38:28.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9421935b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d146c60 con 0x55da4d1f3680 2023-10-18T12:38:28.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9421935b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d144580 con 0x55da4d1f2d60 2023-10-18T12:38:28.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9404ff9700 1 -- 172.21.15.70:0/1079596048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f93f8014070 con 0x7f94081005c0 2023-10-18T12:38:28.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9407fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9407fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57012/0 (socket says 172.21.15.70:57012) 2023-10-18T12:38:28.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9407fff700 1 -- 172.21.15.70:0/252867062 learned_addr learned my addr 172.21.15.70:0/252867062 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9406ffd700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93f00a15f0 0x7f93f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9407fff700 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f94077fe700 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9407fff700 1 -- 172.21.15.70:0/252867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 msgr2=0x55da4d1ed2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.350+0000 7f9407fff700 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1ed2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f9407fff700 1 -- 172.21.15.70:0/252867062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 msgr2=0x55da4d1edc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f9407fff700 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1edc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f9407fff700 1 -- 172.21.15.70:0/252867062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da4d0fd3f0 con 0x55da4d1f2d60 2023-10-18T12:38:28.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f94077fe700 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f9406ffd700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93f00a15f0 0x7f93f00a3ab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9408197ed0 tx=0x7f93fc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f9407fff700 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1f3140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f94000060f0 tx=0x7f9400006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4526ea21670fde11 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f93edffb700 1 -- 172.21.15.70:0/252867062 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000e040 con 0x55da4d1f2d60 2023-10-18T12:38:28.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f93edffb700 1 -- 172.21.15.70:0/252867062 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9400004d30 con 0x55da4d1f2d60 2023-10-18T12:38:28.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.351+0000 7f93edffb700 1 -- 172.21.15.70:0/252867062 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400007680 con 0x55da4d1f2d60 2023-10-18T12:38:28.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 -- 172.21.15.70:0/252867062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 msgr2=0x55da4d1f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1f3140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f94000060f0 tx=0x7f9400006040 comp rx=0 tx=0).stop 2023-10-18T12:38:28.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 -- 172.21.15.70:0/252867062 shutdown_connections 2023-10-18T12:38:28.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1f3140 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 --2- 172.21.15.70:0/252867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 -- 172.21.15.70:0/252867062 >> 172.21.15.70:0/252867062 conn(0x55da4d161720 msgr2=0x55da4d1e7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 -- 172.21.15.70:0/252867062 shutdown_connections 2023-10-18T12:38:28.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 -- 172.21.15.70:0/252867062 wait complete. 2023-10-18T12:38:28.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.352+0000 7f9421935b80 1 -- start start 2023-10-18T12:38:28.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 Processor -- start 2023-10-18T12:38:28.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f2d60 0x55da4d1f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d13fe70 con 0x55da4d1f2d60 2023-10-18T12:38:28.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d0dc710 con 0x55da4d1f3680 2023-10-18T12:38:28.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d0dc3b0 con 0x55da4d1ed820 2023-10-18T12:38:28.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9421935b80 10 client.0 _ll_register_callbacks cb 0x55da4d026a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:28.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f94077fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f94077fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57018/0 (socket says 172.21.15.70:57018) 2023-10-18T12:38:28.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f94077fe700 1 -- 172.21.15.70:0/3707843854 learned_addr learned my addr 172.21.15.70:0/3707843854 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9407fff700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.353+0000 7f9406ffd700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f2d60 0x55da4d1f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f94077fe700 1 -- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 msgr2=0x55da4d1f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f94077fe700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f94077fe700 1 -- 172.21.15.70:0/3707843854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f2d60 msgr2=0x55da4d1f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f94077fe700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f2d60 0x55da4d1f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f94077fe700 1 -- 172.21.15.70:0/3707843854 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94081a35d0 con 0x55da4d1ed820 2023-10-18T12:38:28.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f94077fe700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f9690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93f8096420 tx=0x7f93f809a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8650ba76a2c2df12 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f93e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:38:28.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f93e6ffd700 1 -- 172.21.15.70:0/3707843854 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f80a9fd0 con 0x55da4d1ed820 2023-10-18T12:38:28.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da4d144ef0 con 0x55da4d1ed820 2023-10-18T12:38:28.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f93e6ffd700 1 -- 172.21.15.70:0/3707843854 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f93f80a38f0 con 0x55da4d1ed820 2023-10-18T12:38:28.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f93e6ffd700 1 -- 172.21.15.70:0/3707843854 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93f80ba070 con 0x55da4d1ed820 2023-10-18T12:38:28.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f9421935b80 10 client.27382 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:28.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.354+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f940810dd90 con 0x55da4d1ed820 2023-10-18T12:38:28.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f93e6ffd700 1 -- 172.21.15.70:0/3707843854 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da4d144ef0 con 0x55da4d1ed820 2023-10-18T12:38:28.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f93e6ffd700 1 -- 172.21.15.70:0/3707843854 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93f809a900 con 0x55da4d1ed820 2023-10-18T12:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9421935b80 10 client.27382 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9421935b80 10 client.27382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9421935b80 20 client.27382 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9421935b80 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d207040 0x55da4d209460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9421935b80 4 client.27382 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da4d1bc8b0 con 0x55da4d207040 2023-10-18T12:38:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.355+0000 7f9406ffd700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d207040 0x55da4d209460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.356+0000 7f9406ffd700 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d207040 0x55da4d209460 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bfde364c2cdbbd server_cookie=d76e6a0dc6333f84 in_seq=0 out_seq=0 2023-10-18T12:38:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.356+0000 7f93e6ffd700 10 client.27382 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.356+0000 7f93e6ffd700 1 -- 172.21.15.70:0/3707843854 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da4d1bc8b0 con 0x55da4d207040 2023-10-18T12:38:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.356+0000 7f93e6ffd700 10 client.27382 handle_command_reply: tid=0 2023-10-18T12:38:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 client.27382 shutdown 2023-10-18T12:38:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 msgr2=0x55da4d1f9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f9690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93f8096420 tx=0x7f93f809a1e0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 shutdown_connections 2023-10-18T12:38:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d207040 0x55da4d209460 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f93e6ffd700 0 client.27382 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f9690 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 --2- 172.21.15.70:0/3707843854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f2d60 0x55da4d1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 >> 172.21.15.70:0/3707843854 conn(0x55da4d161720 msgr2=0x55da4d1ec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 shutdown_connections 2023-10-18T12:38:28.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 1 -- 172.21.15.70:0/3707843854 wait complete. 2023-10-18T12:38:28.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.357+0000 7f9421935b80 20 client.27382 trim_cache size 0 max 16384 2023-10-18T12:38:28.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.375+0000 7f9421935b80 1 Processor -- start 2023-10-18T12:38:28.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.375+0000 7f9421935b80 1 -- start start 2023-10-18T12:38:28.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9421935b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9421935b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f2d60 0x55da4d1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9421935b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f3680 0x55da4d1fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9421935b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4cfc0920 con 0x55da4d1f3680 2023-10-18T12:38:28.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9421935b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d1fe570 con 0x55da4d1f2d60 2023-10-18T12:38:28.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9421935b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d1ed620 con 0x55da4d1ed820 2023-10-18T12:38:28.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9406ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f2d60 0x55da4d1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57022/0 (socket says 172.21.15.70:57022) 2023-10-18T12:38:28.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 -- 172.21.15.70:0/2409261579 learned_addr learned my addr 172.21.15.70:0/2409261579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f9407fff700 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f3680 0x55da4d1fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 -- 172.21.15.70:0/2409261579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f2d60 msgr2=0x55da4d1f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f2d60 0x55da4d1f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 -- 172.21.15.70:0/2409261579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f3680 msgr2=0x55da4d1fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f3680 0x55da4d1fe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.376+0000 7f94077fe700 1 -- 172.21.15.70:0/2409261579 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da4cf6ade0 con 0x55da4d1ed820 2023-10-18T12:38:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.377+0000 7f94077fe700 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f68b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93f80970c0 tx=0x7f93f8098c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2226d76bbf78b98 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.377+0000 7f93ee7fc700 1 -- 172.21.15.70:0/2409261579 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f809e7f0 con 0x55da4d1ed820 2023-10-18T12:38:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.377+0000 7f93ee7fc700 1 -- 172.21.15.70:0/2409261579 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f80a2a30 con 0x55da4d1ed820 2023-10-18T12:38:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.377+0000 7f93ee7fc700 1 -- 172.21.15.70:0/2409261579 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8099df0 con 0x55da4d1ed820 2023-10-18T12:38:28.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.377+0000 7f9421935b80 1 -- 172.21.15.70:0/2409261579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 msgr2=0x55da4d1f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.377+0000 7f9421935b80 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f68b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93f80970c0 tx=0x7f93f8098c50 comp rx=0 tx=0).stop 2023-10-18T12:38:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 -- 172.21.15.70:0/2409261579 shutdown_connections 2023-10-18T12:38:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1f3680 0x55da4d1fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1ed820 0x55da4d1f68b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 --2- 172.21.15.70:0/2409261579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f2d60 0x55da4d1f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 -- 172.21.15.70:0/2409261579 >> 172.21.15.70:0/2409261579 conn(0x55da4d161720 msgr2=0x55da4d1f61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 -- 172.21.15.70:0/2409261579 shutdown_connections 2023-10-18T12:38:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 -- 172.21.15.70:0/2409261579 wait complete. 2023-10-18T12:38:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.378+0000 7f9421935b80 1 -- start start 2023-10-18T12:38:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 Processor -- start 2023-10-18T12:38:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d25aea0 con 0x55da4d1ed820 2023-10-18T12:38:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d25afe0 con 0x55da4d1f3680 2023-10-18T12:38:28.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4d25b120 con 0x55da4d1f2d60 2023-10-18T12:38:28.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.379+0000 7f9421935b80 10 client.0 _ll_register_callbacks cb 0x55da4d026a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:28.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f9406ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1fc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34882/0 (socket says 172.21.15.70:34882) 2023-10-18T12:38:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 -- 172.21.15.70:0/2975539463 learned_addr learned my addr 172.21.15.70:0/2975539463 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f9407fff700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 -- 172.21.15.70:0/2975539463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 msgr2=0x55da4d1fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 -- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 msgr2=0x55da4d1fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 -- 172.21.15.70:0/2975539463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940810dd90 con 0x55da4d1ed820 2023-10-18T12:38:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f9406ffd700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f9407fff700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f94077fe700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1fc630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f93f80989f0 tx=0x7f93f8010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115e802c60e825d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f93e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:38:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.380+0000 7f93e6ffd700 1 -- 172.21.15.70:0/2975539463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f809a1e0 con 0x55da4d1ed820 2023-10-18T12:38:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9400004d30 con 0x55da4d1ed820 2023-10-18T12:38:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f93e6ffd700 1 -- 172.21.15.70:0/2975539463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f93f809a6a0 con 0x55da4d1ed820 2023-10-18T12:38:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f93e6ffd700 1 -- 172.21.15.70:0/2975539463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93f809e7f0 con 0x55da4d1ed820 2023-10-18T12:38:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 10 client.17355 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55da4cf53c60 con 0x55da4d1ed820 2023-10-18T12:38:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f93e6ffd700 1 -- 172.21.15.70:0/2975539463 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9400004d30 con 0x55da4d1ed820 2023-10-18T12:38:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f93e6ffd700 1 -- 172.21.15.70:0/2975539463 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93f8018b60 con 0x55da4d1ed820 2023-10-18T12:38:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 10 client.17355 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 20 client.17355 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d25c3c0 0x55da4d25e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 4 client.17355 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:38:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da4d138e50 con 0x55da4d25c3c0 2023-10-18T12:38:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.381+0000 7f9406ffd700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d25c3c0 0x55da4d25e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.382+0000 7f9406ffd700 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d25c3c0 0x55da4d25e7b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93832f70c534ec85 server_cookie=2239a06ee1c191d8 in_seq=0 out_seq=0 2023-10-18T12:38:28.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.382+0000 7f93e6ffd700 10 client.17355 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f93e6ffd700 1 -- 172.21.15.70:0/2975539463 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da4d138e50 con 0x55da4d25c3c0 2023-10-18T12:38:28.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f93e6ffd700 10 client.17355 handle_command_reply: tid=0 2023-10-18T12:38:28.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 client.17355 shutdown 2023-10-18T12:38:28.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 msgr2=0x55da4d1fc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1fc630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f93f80989f0 tx=0x7f93f8010d60 comp rx=0 tx=0).stop 2023-10-18T12:38:28.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 shutdown_connections 2023-10-18T12:38:28.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55da4d25c3c0 0x55da4d25e7b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55da4d1f3680 0x55da4d1fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da4d1ed820 0x55da4d1fc630 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 --2- 172.21.15.70:0/2975539463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da4d1f2d60 0x55da4d1fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f93e6ffd700 0 client.17355 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:28.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 >> 172.21.15.70:0/2975539463 conn(0x55da4d161720 msgr2=0x55da4d1e1700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 shutdown_connections 2023-10-18T12:38:28.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.384+0000 7f9421935b80 1 -- 172.21.15.70:0/2975539463 wait complete. 2023-10-18T12:38:28.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.385+0000 7f9421935b80 20 client.17355 trim_cache size 0 max 16384 2023-10-18T12:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:38:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:38:28.402 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:38:28.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.385+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93f00a15f0 msgr2=0x7f93f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.385+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93f00a15f0 0x7f93f00a3ab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9408197ed0 tx=0x7f93fc009260 comp rx=0 tx=0).stop 2023-10-18T12:38:28.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.385+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 msgr2=0x7f940819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.385+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 0x7f940819c650 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f93f800ebf0 tx=0x7f93f800ef00 comp rx=0 tx=0).stop 2023-10-18T12:38:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 shutdown_connections 2023-10-18T12:38:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9408106660 0x7f94081966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94081005c0 0x7f940819c650 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93f00a15f0 0x7f93f00a3ab0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 --2- 172.21.15.70:0/1079596048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9408100f80 0x7f940819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 >> 172.21.15.70:0/1079596048 conn(0x7f94080fc0f0 msgr2=0x7f94080fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 shutdown_connections 2023-10-18T12:38:28.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.386+0000 7f940cd9d700 1 -- 172.21.15.70:0/1079596048 wait complete. 2023-10-18T12:38:28.408 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:38:28.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:38:28.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.634+0000 7fe1311ff700 1 -- 172.21.15.70:0/2197042249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c072c20 msgr2=0x7fe12c073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.634+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2197042249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c072c20 0x7fe12c073000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009990 tx=0x7fe11c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 -- 172.21.15.70:0/2197042249 shutdown_connections 2023-10-18T12:38:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2197042249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 0x7fe12c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2197042249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c072c20 0x7fe12c073000 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2197042249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c073540 0x7fe12c10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 -- 172.21.15.70:0/2197042249 >> 172.21.15.70:0/2197042249 conn(0x7fe12c0fc170 msgr2=0x7fe12c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 -- 172.21.15.70:0/2197042249 shutdown_connections 2023-10-18T12:38:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.635+0000 7fe1311ff700 1 -- 172.21.15.70:0/2197042249 wait complete. 2023-10-18T12:38:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.636+0000 7fe1311ff700 1 Processor -- start 2023-10-18T12:38:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.636+0000 7fe1311ff700 1 -- start start 2023-10-18T12:38:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe1311ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 0x7fe12c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe1311ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c073540 0x7fe12c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe1311ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 0x7fe12c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe1311ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c19dcd0 con 0x7fe12c072c20 2023-10-18T12:38:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe1311ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c19de40 con 0x7fe12c073540 2023-10-18T12:38:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe1311ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c19e000 con 0x7fe12c10cee0 2023-10-18T12:38:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 0x7fe12c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 0x7fe12c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34898/0 (socket says 172.21.15.70:34898) 2023-10-18T12:38:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12b59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 0x7fe12c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12ad9d700 1 -- 172.21.15.70:0/2501666953 learned_addr learned my addr 172.21.15.70:0/2501666953 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12a59c700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c073540 0x7fe12c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12b59e700 1 -- 172.21.15.70:0/2501666953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c073540 msgr2=0x7fe12c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12b59e700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c073540 0x7fe12c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12b59e700 1 -- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 msgr2=0x7fe12c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12b59e700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 0x7fe12c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12b59e700 1 -- 172.21.15.70:0/2501666953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120009520 con 0x7fe12c10cee0 2023-10-18T12:38:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.637+0000 7fe12ad9d700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 0x7fe12c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.638+0000 7fe12b59e700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 0x7fe12c19d690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe12400efc0 tx=0x7fe12400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.638+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124018070 con 0x7fe12c10cee0 2023-10-18T12:38:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.638+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe12400f040 con 0x7fe12c10cee0 2023-10-18T12:38:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.638+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c0095f0 con 0x7fe12c10cee0 2023-10-18T12:38:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.638+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400cd40 con 0x7fe12c10cee0 2023-10-18T12:38:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.638+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c1a5940 con 0x7fe12c10cee0 2023-10-18T12:38:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.639+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c1080f0 con 0x7fe12c10cee0 2023-10-18T12:38:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.642+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe124003a00 con 0x7fe12c10cee0 2023-10-18T12:38:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.643+0000 7fe113fff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1140a15f0 0x7fe1140a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.643+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe1240cd0d0 con 0x7fe12c10cee0 2023-10-18T12:38:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.644+0000 7fe12ad9d700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1140a15f0 0x7fe1140a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.644+0000 7fe12ad9d700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1140a15f0 0x7fe1140a3ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe11c006010 tx=0x7fe11c00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.645+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe124096120 con 0x7fe12c10cee0 2023-10-18T12:38:28.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1398024 11.02 MB/sec execute 537 sec latency 15.193 ms 2023-10-18T12:38:28.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.929+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe12c0742d0 con 0x7fe12c10cee0 2023-10-18T12:38:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.929+0000 7fe113fff700 1 -- 172.21.15.70:0/2501666953 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fe124099740 con 0x7fe12c10cee0 2023-10-18T12:38:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:38:28.933 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:38:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1140a15f0 msgr2=0x7fe1140a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1140a15f0 0x7fe1140a3ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe11c006010 tx=0x7fe11c00b260 comp rx=0 tx=0).stop 2023-10-18T12:38:28.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 msgr2=0x7fe12c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:28.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 0x7fe12c19d690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe12400efc0 tx=0x7fe12400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 shutdown_connections 2023-10-18T12:38:28.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe12c10cee0 0x7fe12c19d690 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe12c072c20 0x7fe12c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe1140a15f0 0x7fe1140a3ab0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 --2- 172.21.15.70:0/2501666953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe12c073540 0x7fe12c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:28.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 >> 172.21.15.70:0/2501666953 conn(0x7fe12c0fc170 msgr2=0x7fe12c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:28.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 shutdown_connections 2023-10-18T12:38:28.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:28.932+0000 7fe1311ff700 1 -- 172.21.15.70:0/2501666953 wait complete. 2023-10-18T12:38:28.938 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:38:28.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:38:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:28 smithi152 ceph-mon[134767]: pgmap v700: 129 pgs: 129 active+clean; 582 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 222 op/s 2023-10-18T12:38:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:28 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:38:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:28 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:28 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:38:29.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:28 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:29.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:28 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:38:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.169+0000 7f5b307ff700 1 -- 172.21.15.70:0/2928225575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b28106730 msgr2=0x7f5b28106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.169+0000 7f5b307ff700 1 --2- 172.21.15.70:0/2928225575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b28106730 0x7f5b28106b10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009990 tx=0x7f5b24009ca0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 -- 172.21.15.70:0/2928225575 shutdown_connections 2023-10-18T12:38:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 --2- 172.21.15.70:0/2928225575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b281010e0 0x7f5b2810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 --2- 172.21.15.70:0/2928225575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b28106730 0x7f5b28106b10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 --2- 172.21.15.70:0/2928225575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28100720 0x7f5b28100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 -- 172.21.15.70:0/2928225575 >> 172.21.15.70:0/2928225575 conn(0x7f5b280fc1b0 msgr2=0x7f5b280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 -- 172.21.15.70:0/2928225575 shutdown_connections 2023-10-18T12:38:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.170+0000 7f5b307ff700 1 -- 172.21.15.70:0/2928225575 wait complete. 2023-10-18T12:38:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.171+0000 7f5b307ff700 1 Processor -- start 2023-10-18T12:38:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.171+0000 7f5b307ff700 1 -- start start 2023-10-18T12:38:29.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b307ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b28100720 0x7f5b28198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b307ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 0x7f5b281992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b307ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28106730 0x7f5b2819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b307ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2819dcc0 con 0x7f5b28106730 2023-10-18T12:38:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b307ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2819de30 con 0x7f5b281010e0 2023-10-18T12:38:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b307ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2819dff0 con 0x7f5b28100720 2023-10-18T12:38:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b2dd9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 0x7f5b281992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b2dd9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 0x7f5b281992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51634/0 (socket says 172.21.15.70:51634) 2023-10-18T12:38:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/1601921638 learned_addr learned my addr 172.21.15.70:0/1601921638 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28106730 0x7f5b2819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.172+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b28100720 0x7f5b28198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/1601921638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b28100720 msgr2=0x7f5b28198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b28100720 0x7f5b28198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28106730 msgr2=0x7f5b2819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28106730 0x7f5b2819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/1601921638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18009520 con 0x7f5b281010e0 2023-10-18T12:38:29.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 0x7f5b281992e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800eb30 tx=0x7f5b1800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b1f7fe700 1 -- 172.21.15.70:0/1601921638 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18009880 con 0x7f5b281010e0 2023-10-18T12:38:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b240095f0 con 0x7f5b281010e0 2023-10-18T12:38:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b1f7fe700 1 -- 172.21.15.70:0/1601921638 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b180044e0 con 0x7f5b281010e0 2023-10-18T12:38:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b1f7fe700 1 -- 172.21.15.70:0/1601921638 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18003d80 con 0x7f5b281010e0 2023-10-18T12:38:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.173+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b281a5930 con 0x7f5b281010e0 2023-10-18T12:38:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.178+0000 7f5b1f7fe700 1 -- 172.21.15.70:0/1601921638 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5b18003f20 con 0x7f5b281010e0 2023-10-18T12:38:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 Processor -- start 2023-10-18T12:38:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 -- start start 2023-10-18T12:38:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 0x557487974fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487975500 0x55748796f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55748796f5d0 0x55748796fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574878e26e0 con 0x55748796f5d0 2023-10-18T12:38:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574878c8c60 con 0x557487974be0 2023-10-18T12:38:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.179+0000 7f5b4413db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574878c6580 con 0x557487975500 2023-10-18T12:38:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2ed9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 0x557487974fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2ed9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 0x557487974fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51642/0 (socket says 172.21.15.70:51642) 2023-10-18T12:38:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b1f7fe700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5b140a15f0 0x7f5b140a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2ed9c700 1 -- 172.21.15.70:0/2648037202 learned_addr learned my addr 172.21.15.70:0/2648037202 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55748796f5d0 0x55748796fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487975500 0x55748796f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b1f7fe700 1 -- 172.21.15.70:0/1601921638 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5b18014070 con 0x7f5b281010e0 2023-10-18T12:38:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5b140a15f0 0x7f5b140a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/2648037202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487975500 msgr2=0x55748796f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487975500 0x55748796f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/2648037202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 msgr2=0x557487974fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 0x557487974fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/2648037202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55748787f3f0 con 0x55748796f5d0 2023-10-18T12:38:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487975500 0x55748796f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.180+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 0x557487974fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.181+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55748796f5d0 0x55748796fa50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5b180cad50 tx=0x7f5b18003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47892d765febd615 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.181+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5b140a15f0 0x7f5b140a3ab0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000fcb0 tx=0x7f5b20004b40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.181+0000 7f5b1cff9700 1 -- 172.21.15.70:0/2648037202 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18099f40 con 0x55748796f5d0 2023-10-18T12:38:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.181+0000 7f5b1cff9700 1 -- 172.21.15.70:0/2648037202 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b1809a5e0 con 0x55748796f5d0 2023-10-18T12:38:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.181+0000 7f5b1cff9700 1 -- 172.21.15.70:0/2648037202 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b180a2970 con 0x55748796f5d0 2023-10-18T12:38:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 -- 172.21.15.70:0/2648037202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55748796f5d0 msgr2=0x55748796fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55748796f5d0 0x55748796fa50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5b180cad50 tx=0x7f5b18003680 comp rx=0 tx=0).stop 2023-10-18T12:38:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 -- 172.21.15.70:0/2648037202 shutdown_connections 2023-10-18T12:38:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55748796f5d0 0x55748796fa50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487974be0 0x557487974fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2648037202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487975500 0x55748796f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 -- 172.21.15.70:0/2648037202 >> 172.21.15.70:0/2648037202 conn(0x5574878e3720 msgr2=0x557487969b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 -- 172.21.15.70:0/2648037202 shutdown_connections 2023-10-18T12:38:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.182+0000 7f5b4413db80 1 -- 172.21.15.70:0/2648037202 wait complete. 2023-10-18T12:38:29.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.183+0000 7f5b4413db80 1 -- start start 2023-10-18T12:38:29.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.183+0000 7f5b4413db80 1 Processor -- start 2023-10-18T12:38:29.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 0x557487978890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487974be0 0x557487978dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487975500 0x557487979310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574878c1e70 con 0x557487975500 2023-10-18T12:38:29.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55748785e710 con 0x55748796f5d0 2023-10-18T12:38:29.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55748785e3b0 con 0x557487974be0 2023-10-18T12:38:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b4413db80 10 client.0 _ll_register_callbacks cb 0x5574877a8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b2e59b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 0x557487978890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b2e59b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 0x557487978890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51644/0 (socket says 172.21.15.70:51644) 2023-10-18T12:38:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b2e59b700 1 -- 172.21.15.70:0/4058713487 learned_addr learned my addr 172.21.15.70:0/4058713487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487974be0 0x557487978dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.184+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487975500 0x557487979310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b2e59b700 1 -- 172.21.15.70:0/4058713487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487974be0 msgr2=0x557487978dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487974be0 0x557487978dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b2e59b700 1 -- 172.21.15.70:0/4058713487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487975500 msgr2=0x557487979310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487975500 0x557487979310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b2e59b700 1 -- 172.21.15.70:0/4058713487 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2810def0 con 0x55748796f5d0 2023-10-18T12:38:29.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 0x557487978890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009990 tx=0x7f5b24005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e0f507886b67b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:29.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:38:29.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.185+0000 7f5b11ffb700 1 -- 172.21.15.70:0/4058713487 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2401d070 con 0x55748796f5d0 2023-10-18T12:38:29.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b11ffb700 1 -- 172.21.15.70:0/4058713487 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5b24004500 con 0x55748796f5d0 2023-10-18T12:38:29.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574878c6ef0 con 0x55748796f5d0 2023-10-18T12:38:29.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b11ffb700 1 -- 172.21.15.70:0/4058713487 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b240046c0 con 0x55748796f5d0 2023-10-18T12:38:29.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b4413db80 10 client.27422 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:29.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f5b281a5930 con 0x55748796f5d0 2023-10-18T12:38:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b11ffb700 1 -- 172.21.15.70:0/4058713487 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574878c6ef0 con 0x55748796f5d0 2023-10-18T12:38:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.186+0000 7f5b11ffb700 1 -- 172.21.15.70:0/4058713487 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5b2400f800 con 0x55748796f5d0 2023-10-18T12:38:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b4413db80 10 client.27422 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b4413db80 10 client.27422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:29.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b4413db80 20 client.27422 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b4413db80 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557487988fe0 0x55748798b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b4413db80 4 client.27422 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55748793e8b0 con 0x557487988fe0 2023-10-18T12:38:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.187+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557487988fe0 0x55748798b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.188+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557487988fe0 0x55748798b400 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3e882573c0f82d server_cookie=9434bd6d54302db8 in_seq=0 out_seq=0 2023-10-18T12:38:29.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.188+0000 7f5b11ffb700 10 client.27422 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:29.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.188+0000 7f5b11ffb700 1 -- 172.21.15.70:0/4058713487 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55748793e8b0 con 0x557487988fe0 2023-10-18T12:38:29.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.188+0000 7f5b11ffb700 10 client.27422 handle_command_reply: tid=0 2023-10-18T12:38:29.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.188+0000 7f5b4413db80 1 client.27422 shutdown 2023-10-18T12:38:29.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 msgr2=0x557487978890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 0x557487978890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009990 tx=0x7f5b24005800 comp rx=0 tx=0).stop 2023-10-18T12:38:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 shutdown_connections 2023-10-18T12:38:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557487988fe0 0x55748798b400 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487975500 0x557487979310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55748796f5d0 0x557487978890 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b11ffb700 0 client.27422 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 --2- 172.21.15.70:0/4058713487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557487974be0 0x557487978dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 >> 172.21.15.70:0/4058713487 conn(0x5574878e3720 msgr2=0x55748796e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 shutdown_connections 2023-10-18T12:38:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 1 -- 172.21.15.70:0/4058713487 wait complete. 2023-10-18T12:38:29.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.189+0000 7f5b4413db80 20 client.27422 trim_cache size 0 max 16384 2023-10-18T12:38:29.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 Processor -- start 2023-10-18T12:38:29.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 -- start start 2023-10-18T12:38:29.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x55748797feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x557487984600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487742920 con 0x557487974be0 2023-10-18T12:38:29.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487986640 con 0x557487975500 2023-10-18T12:38:29.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b4413db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557487979e90 con 0x55748796f5d0 2023-10-18T12:38:29.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b2dd9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x55748797feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b2dd9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x55748797feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34950/0 (socket says 172.21.15.70:34950) 2023-10-18T12:38:29.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/3467766219 learned_addr learned my addr 172.21.15.70:0/3467766219 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:29.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.204+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x557487984600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/3467766219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 msgr2=0x5574879645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/3467766219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 msgr2=0x557487984600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x557487984600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2dd9a700 1 -- 172.21.15.70:0/3467766219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574876ecde0 con 0x557487974be0 2023-10-18T12:38:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x55748797feb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5b18096360 tx=0x7f5b180ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68316380643337c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b1d7fa700 1 -- 172.21.15.70:0/3467766219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1809ad60 con 0x557487974be0 2023-10-18T12:38:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b1d7fa700 1 -- 172.21.15.70:0/3467766219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b1800cbd0 con 0x557487974be0 2023-10-18T12:38:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.205+0000 7f5b1d7fa700 1 -- 172.21.15.70:0/3467766219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b180b8ae0 con 0x557487974be0 2023-10-18T12:38:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 -- 172.21.15.70:0/3467766219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 msgr2=0x55748797feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x55748797feb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5b18096360 tx=0x7f5b180ba040 comp rx=0 tx=0).stop 2023-10-18T12:38:29.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 -- 172.21.15.70:0/3467766219 shutdown_connections 2023-10-18T12:38:29.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x557487984600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 --2- 172.21.15.70:0/3467766219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x55748797feb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 -- 172.21.15.70:0/3467766219 >> 172.21.15.70:0/3467766219 conn(0x5574878e3720 msgr2=0x557487979170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:29.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 -- 172.21.15.70:0/3467766219 shutdown_connections 2023-10-18T12:38:29.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.206+0000 7f5b4413db80 1 -- 172.21.15.70:0/3467766219 wait complete. 2023-10-18T12:38:29.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 -- start start 2023-10-18T12:38:29.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 Processor -- start 2023-10-18T12:38:29.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x5574879d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x5574879d5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574879d6330 con 0x557487974be0 2023-10-18T12:38:29.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574879d6470 con 0x557487975500 2023-10-18T12:38:29.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574879d6630 con 0x55748796f5d0 2023-10-18T12:38:29.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b4413db80 10 client.0 _ll_register_callbacks cb 0x5574877a8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:29.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b2dd9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x5574879d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.207+0000 7f5b2ed9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x5574879d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x5574879d5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51662/0 (socket says 172.21.15.70:51662) 2023-10-18T12:38:29.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 -- 172.21.15.70:0/2013762394 learned_addr learned my addr 172.21.15.70:0/2013762394 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:29.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 -- 172.21.15.70:0/2013762394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 msgr2=0x5574879d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 -- 172.21.15.70:0/2013762394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 msgr2=0x5574879d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:29.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x5574879d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 -- 172.21.15.70:0/2013762394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b281a5930 con 0x557487975500 2023-10-18T12:38:29.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2dd9a700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x5574879d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:29.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b2ed9c700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x5574879d5df0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000f930 tx=0x7f5b20004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe3d1f9ec4f5b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:29.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.208+0000 7f5b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:38:29.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b11ffb700 1 -- 172.21.15.70:0/2013762394 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000e040 con 0x557487975500 2023-10-18T12:38:29.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b11ffb700 1 -- 172.21.15.70:0/2013762394 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5b20004bb0 con 0x557487975500 2023-10-18T12:38:29.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b1809a5e0 con 0x557487975500 2023-10-18T12:38:29.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b11ffb700 1 -- 172.21.15.70:0/2013762394 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b20004d70 con 0x557487975500 2023-10-18T12:38:29.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 10 client.27428 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:29.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55748787f3f0 con 0x557487975500 2023-10-18T12:38:29.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b11ffb700 1 -- 172.21.15.70:0/2013762394 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b1809a5e0 con 0x557487975500 2023-10-18T12:38:29.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b11ffb700 1 -- 172.21.15.70:0/2013762394 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5b20007940 con 0x557487975500 2023-10-18T12:38:29.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 10 client.27428 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:29.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:29.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 20 client.27428 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:29.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5574879ddfc0 0x5574879e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 4 client.27428 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:38:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.209+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574878bae50 con 0x5574879ddfc0 2023-10-18T12:38:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.210+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5574879ddfc0 0x5574879e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.210+0000 7f5b11ffb700 10 client.27428 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.210+0000 7f5b2e59b700 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5574879ddfc0 0x5574879e03e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb6775dabd3fa47 server_cookie=4bca8fae90152848 in_seq=0 out_seq=0 2023-10-18T12:38:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.210+0000 7f5b11ffb700 1 -- 172.21.15.70:0/2013762394 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574878bae50 con 0x5574879ddfc0 2023-10-18T12:38:29.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.210+0000 7f5b11ffb700 10 client.27428 handle_command_reply: tid=0 2023-10-18T12:38:29.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.210+0000 7f5b4413db80 1 client.27428 shutdown 2023-10-18T12:38:29.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 msgr2=0x5574879d5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x5574879d5df0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000f930 tx=0x7f5b20004ad0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 shutdown_connections 2023-10-18T12:38:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5574879ddfc0 0x5574879e03e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557487975500 0x5574879d5df0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55748796f5d0 0x5574879d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b11ffb700 0 client.27428 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 --2- 172.21.15.70:0/2013762394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557487974be0 0x5574879d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 >> 172.21.15.70:0/2013762394 conn(0x5574878e3720 msgr2=0x557487984600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 shutdown_connections 2023-10-18T12:38:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 1 -- 172.21.15.70:0/2013762394 wait complete. 2023-10-18T12:38:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.211+0000 7f5b4413db80 20 client.27428 trim_cache size 0 max 16384 2023-10-18T12:38:29.232 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:38:29.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5b140a15f0 msgr2=0x7f5b140a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5b140a15f0 0x7f5b140a3ab0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000fcb0 tx=0x7f5b20004b40 comp rx=0 tx=0).stop 2023-10-18T12:38:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 msgr2=0x7f5b281992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 0x7f5b281992e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800eb30 tx=0x7f5b1800ee40 comp rx=0 tx=0).stop 2023-10-18T12:38:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 shutdown_connections 2023-10-18T12:38:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28106730 0x7f5b2819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b28100720 0x7f5b28198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5b140a15f0 0x7f5b140a3ab0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 --2- 172.21.15.70:0/1601921638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b281010e0 0x7f5b281992e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 >> 172.21.15.70:0/1601921638 conn(0x7f5b280fc1b0 msgr2=0x7f5b2810be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 shutdown_connections 2023-10-18T12:38:29.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:29.212+0000 7f5b307ff700 1 -- 172.21.15.70:0/1601921638 wait complete. 2023-10-18T12:38:29.235 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:28 smithi070 ceph-mon[98639]: pgmap v700: 129 pgs: 129 active+clean; 582 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 222 op/s 2023-10-18T12:38:29.235 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:28 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:38:29.235 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:28 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:29.235 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:28 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:38:29.236 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:28 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:29.236 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:28 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:38:29.237 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:28 smithi186 ceph-mon[135253]: pgmap v700: 129 pgs: 129 active+clean; 582 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 222 op/s 2023-10-18T12:38:29.237 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:28 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:38:29.237 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:28 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:29.238 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:28 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:38:29.238 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:28 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:29.238 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:28 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:38:29.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1400291 11.03 MB/sec execute 538 sec latency 41.116 ms 2023-10-18T12:38:30.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:29 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2501666953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:38:30.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:29 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2501666953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:38:30.238 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:38:30.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:38:30.305 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:29 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2501666953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:38:30.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2956057393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 msgr2=0x7f78e8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2956057393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e8100ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f78d800b0a0 tx=0x7f78d800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2956057393 shutdown_connections 2023-10-18T12:38:30.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2956057393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2956057393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8106730 0x7f78e8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2956057393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e8100ba0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2956057393 >> 172.21.15.70:0/2956057393 conn(0x7f78e80fc1b0 msgr2=0x7f78e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.508+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2956057393 shutdown_connections 2023-10-18T12:38:30.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.509+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2956057393 wait complete. 2023-10-18T12:38:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.510+0000 7f78ee9ff700 1 Processor -- start 2023-10-18T12:38:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.510+0000 7f78ee9ff700 1 -- start start 2023-10-18T12:38:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78ee9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e810a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78ee9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78ee9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8106730 0x7f78e81a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78ee9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e81a2190 con 0x7f78e8106730 2023-10-18T12:38:30.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78ee9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e81a2300 con 0x7f78e81010e0 2023-10-18T12:38:30.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78ee9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e81a24c0 con 0x7f78e8100720 2023-10-18T12:38:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78e77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78e77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51680/0 (socket says 172.21.15.70:51680) 2023-10-18T12:38:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.511+0000 7f78e77fe700 1 -- 172.21.15.70:0/2810458533 learned_addr learned my addr 172.21.15.70:0/2810458533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e810a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e77fe700 1 -- 172.21.15.70:0/2810458533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 msgr2=0x7f78e810a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e77fe700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e810a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e77fe700 1 -- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8106730 msgr2=0x7f78e81a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e77fe700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8106730 0x7f78e81a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e77fe700 1 -- 172.21.15.70:0/2810458533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8009d00 con 0x7f78e81010e0 2023-10-18T12:38:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:38:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.512+0000 7f78e77fe700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810abe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f78d8006070 tx=0x7f78d80123f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.513+0000 7f78e57fa700 1 -- 172.21.15.70:0/2810458533 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d800e040 con 0x7f78e81010e0 2023-10-18T12:38:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.513+0000 7f78e57fa700 1 -- 172.21.15.70:0/2810458533 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d8003e10 con 0x7f78e81010e0 2023-10-18T12:38:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.513+0000 7f78e57fa700 1 -- 172.21.15.70:0/2810458533 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d800e040 con 0x7f78e81010e0 2023-10-18T12:38:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.513+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e81a2740 con 0x7f78e81010e0 2023-10-18T12:38:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.513+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e81a9e00 con 0x7f78e81010e0 2023-10-18T12:38:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.520+0000 7f78e57fa700 1 -- 172.21.15.70:0/2810458533 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f78d80055b0 con 0x7f78e81010e0 2023-10-18T12:38:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.522+0000 7f78e57fa700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f78d40a16d0 0x7f78d40a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.523+0000 7f78e57fa700 1 -- 172.21.15.70:0/2810458533 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f78d80ce6e0 con 0x7f78e81010e0 2023-10-18T12:38:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.523+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f78d40a16d0 0x7f78d40a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.524+0000 7f790226eb80 1 Processor -- start 2023-10-18T12:38:30.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.524+0000 7f790226eb80 1 -- start start 2023-10-18T12:38:30.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f790226eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d04f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f78d40a16d0 0x7f78d40a3b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78e0007900 tx=0x7f78e0008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f790226eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d049110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f790226eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d049ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f790226eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2cfbc760 con 0x563e2d049650 2023-10-18T12:38:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f790226eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2cfa2ce0 con 0x563e2d04f580 2023-10-18T12:38:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f790226eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2cfa0600 con 0x563e2d04ec60 2023-10-18T12:38:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d04f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d04f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57124/0 (socket says 172.21.15.70:57124) 2023-10-18T12:38:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f78e7fff700 1 -- 172.21.15.70:0/1067106883 learned_addr learned my addr 172.21.15.70:0/1067106883 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.525+0000 7f78e77fe700 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d049110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78ecf9c700 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d049ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78e7fff700 1 -- 172.21.15.70:0/1067106883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 msgr2=0x563e2d049110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78e7fff700 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d049110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78e7fff700 1 -- 172.21.15.70:0/1067106883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 msgr2=0x563e2d049ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78e7fff700 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d049ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78e7fff700 1 -- 172.21.15.70:0/1067106883 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e2cf59470 con 0x563e2d04ec60 2023-10-18T12:38:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.526+0000 7f78ecf9c700 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d049ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:38:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.527+0000 7f78e7fff700 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d04f040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c910 tx=0x7f78e000f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c6179ef55b83ac server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.527+0000 7f78de7fc700 1 -- 172.21.15.70:0/1067106883 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00138b0 con 0x563e2d04ec60 2023-10-18T12:38:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.527+0000 7f78de7fc700 1 -- 172.21.15.70:0/1067106883 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e0007e40 con 0x563e2d04ec60 2023-10-18T12:38:30.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.527+0000 7f78de7fc700 1 -- 172.21.15.70:0/1067106883 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00113d0 con 0x563e2d04ec60 2023-10-18T12:38:30.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.528+0000 7f790226eb80 1 -- 172.21.15.70:0/1067106883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 msgr2=0x563e2d04f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.528+0000 7f790226eb80 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d04f040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c910 tx=0x7f78e000f000 comp rx=0 tx=0).stop 2023-10-18T12:38:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 -- 172.21.15.70:0/1067106883 shutdown_connections 2023-10-18T12:38:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d049ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d04f040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 --2- 172.21.15.70:0/1067106883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d049110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 -- 172.21.15.70:0/1067106883 >> 172.21.15.70:0/1067106883 conn(0x563e2cfbd7a0 msgr2=0x563e2d043be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 -- 172.21.15.70:0/1067106883 shutdown_connections 2023-10-18T12:38:30.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.530+0000 7f790226eb80 1 -- 172.21.15.70:0/1067106883 wait complete. 2023-10-18T12:38:30.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.531+0000 7f790226eb80 1 -- start start 2023-10-18T12:38:30.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.531+0000 7f790226eb80 1 Processor -- start 2023-10-18T12:38:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d052950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d052e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04f580 0x563e2d0533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f78e77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d052e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d052950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2cf9bef0 con 0x563e2d04f580 2023-10-18T12:38:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2cf38790 con 0x563e2d049650 2023-10-18T12:38:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2cf38430 con 0x563e2d04ec60 2023-10-18T12:38:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f78e77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d052e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57140/0 (socket says 172.21.15.70:57140) 2023-10-18T12:38:30.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d052950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51700/0 (socket says 172.21.15.70:51700) 2023-10-18T12:38:30.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f78e77fe700 1 -- 172.21.15.70:0/741536257 learned_addr learned my addr 172.21.15.70:0/741536257 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.532+0000 7f790226eb80 10 client.0 _ll_register_callbacks cb 0x563e2ce82b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:30.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78ecf9c700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04f580 0x563e2d0533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78e77fe700 1 -- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 msgr2=0x563e2d052950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78e77fe700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d052950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78e77fe700 1 -- 172.21.15.70:0/741536257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04f580 msgr2=0x563e2d0533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78e77fe700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04f580 0x563e2d0533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78e77fe700 1 -- 172.21.15.70:0/741536257 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e000c680 con 0x563e2d04ec60 2023-10-18T12:38:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78ecf9c700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04f580 0x563e2d0533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:38:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78e77fe700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d052e90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78d8097ef0 tx=0x7f78d8012e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d0bea555e93720 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.533+0000 7f78c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:38:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f78c77fe700 1 -- 172.21.15.70:0/741536257 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80050b0 con 0x563e2d04ec60 2023-10-18T12:38:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e2cfa0f70 con 0x563e2d04ec60 2023-10-18T12:38:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f78c77fe700 1 -- 172.21.15.70:0/741536257 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f78d809b480 con 0x563e2d04ec60 2023-10-18T12:38:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f78c77fe700 1 -- 172.21.15.70:0/741536257 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78d809f5d0 con 0x563e2d04ec60 2023-10-18T12:38:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f790226eb80 10 client.27421 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563e2cdc6e60 con 0x563e2d04ec60 2023-10-18T12:38:30.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.534+0000 7f78c77fe700 1 -- 172.21.15.70:0/741536257 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e2cfa0f70 con 0x563e2d04ec60 2023-10-18T12:38:30.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f78c77fe700 1 -- 172.21.15.70:0/741536257 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f78d8019350 con 0x563e2d04ec60 2023-10-18T12:38:30.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f790226eb80 10 client.27421 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:30.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f790226eb80 10 client.27421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:30.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f790226eb80 20 client.27421 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:30.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f790226eb80 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d062c20 0x563e2d065010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f790226eb80 4 client.27421 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:30.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e2d018930 con 0x563e2d062c20 2023-10-18T12:38:30.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f78e7fff700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d062c20 0x563e2d065010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.535+0000 7f78c77fe700 10 client.27421 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.536+0000 7f78e7fff700 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d062c20 0x563e2d065010 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a318c484d1dbe1b server_cookie=f7ae904bec506a2a in_seq=0 out_seq=0 2023-10-18T12:38:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f78c77fe700 1 -- 172.21.15.70:0/741536257 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e2d018930 con 0x563e2d062c20 2023-10-18T12:38:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f78c77fe700 10 client.27421 handle_command_reply: tid=0 2023-10-18T12:38:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 client.27421 shutdown 2023-10-18T12:38:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 msgr2=0x563e2d052e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d052e90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78d8097ef0 tx=0x7f78d8012e00 comp rx=0 tx=0).stop 2023-10-18T12:38:30.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 shutdown_connections 2023-10-18T12:38:30.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d062c20 0x563e2d065010 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04f580 0x563e2d0533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d052950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 --2- 172.21.15.70:0/741536257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d052e90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f78c77fe700 0 client.27421 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 >> 172.21.15.70:0/741536257 conn(0x563e2cfbd7a0 msgr2=0x563e2d048100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 shutdown_connections 2023-10-18T12:38:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 1 -- 172.21.15.70:0/741536257 wait complete. 2023-10-18T12:38:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.537+0000 7f790226eb80 20 client.27421 trim_cache size 0 max 16384 2023-10-18T12:38:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 Processor -- start 2023-10-18T12:38:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 -- start start 2023-10-18T12:38:30.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d04e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 0x563e2d0518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04f580 0x563e2d04ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2ce1c9a0 con 0x563e2d04ec60 2023-10-18T12:38:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2d0446d0 con 0x563e2d049650 2023-10-18T12:38:30.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f790226eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2d055490 con 0x563e2d04f580 2023-10-18T12:38:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f78e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 0x563e2d0518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d04e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f78e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 0x563e2d0518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35002/0 (socket says 172.21.15.70:35002) 2023-10-18T12:38:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.555+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d04e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51716/0 (socket says 172.21.15.70:51716) 2023-10-18T12:38:30.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 -- 172.21.15.70:0/3986056147 learned_addr learned my addr 172.21.15.70:0/3986056147 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78ecf9c700 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04f580 0x563e2d04ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 -- 172.21.15.70:0/3986056147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04f580 msgr2=0x563e2d04ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04f580 0x563e2d04ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 -- 172.21.15.70:0/3986056147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 msgr2=0x563e2d04e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d04e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 -- 172.21.15.70:0/3986056147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e2cdafce0 con 0x563e2d04ec60 2023-10-18T12:38:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78e77fe700 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 0x563e2d0518c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f78d809b180 tx=0x7f78d8099a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120a3571f04d1d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78deffd700 1 -- 172.21.15.70:0/3986056147 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80050b0 con 0x563e2d04ec60 2023-10-18T12:38:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78deffd700 1 -- 172.21.15.70:0/3986056147 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d809b480 con 0x563e2d04ec60 2023-10-18T12:38:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.556+0000 7f78deffd700 1 -- 172.21.15.70:0/3986056147 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80194a0 con 0x563e2d04ec60 2023-10-18T12:38:30.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 -- 172.21.15.70:0/3986056147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 msgr2=0x563e2d0518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 0x563e2d0518c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f78d809b180 tx=0x7f78d8099a30 comp rx=0 tx=0).stop 2023-10-18T12:38:30.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 -- 172.21.15.70:0/3986056147 shutdown_connections 2023-10-18T12:38:30.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04f580 0x563e2d04ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d049650 0x563e2d04e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 --2- 172.21.15.70:0/3986056147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d04ec60 0x563e2d0518c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 -- 172.21.15.70:0/3986056147 >> 172.21.15.70:0/3986056147 conn(0x563e2cfbd7a0 msgr2=0x563e2d04c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 -- 172.21.15.70:0/3986056147 shutdown_connections 2023-10-18T12:38:30.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.557+0000 7f790226eb80 1 -- 172.21.15.70:0/3986056147 wait complete. 2023-10-18T12:38:30.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 -- start start 2023-10-18T12:38:30.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 Processor -- start 2023-10-18T12:38:30.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d059100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2d0597e0 con 0x563e2d049650 2023-10-18T12:38:30.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2d059950 con 0x563e2d04f580 2023-10-18T12:38:30.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2d059b10 con 0x563e2d04ec60 2023-10-18T12:38:30.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f790226eb80 10 client.0 _ll_register_callbacks cb 0x7f78d809b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:30.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d0b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35004/0 (socket says 172.21.15.70:35004) 2023-10-18T12:38:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f78ecf9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d059100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.558+0000 7f78e7fff700 1 -- 172.21.15.70:0/2325101381 learned_addr learned my addr 172.21.15.70:0/2325101381 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e77fe700 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e7fff700 1 -- 172.21.15.70:0/2325101381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 msgr2=0x563e2d0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d0b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e7fff700 1 -- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 msgr2=0x563e2d059100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d059100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e7fff700 1 -- 172.21.15.70:0/2325101381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e2cdc6e60 con 0x563e2d049650 2023-10-18T12:38:30.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78e7fff700 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d0b1020 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f78e0006b30 tx=0x7f78e0031380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a43eb9979335c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:38:30.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78c77fe700 1 -- 172.21.15.70:0/2325101381 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e002e040 con 0x563e2d049650 2023-10-18T12:38:30.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f78c77fe700 1 -- 172.21.15.70:0/2325101381 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f78e0031a80 con 0x563e2d049650 2023-10-18T12:38:30.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.559+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78e0007e40 con 0x563e2d049650 2023-10-18T12:38:30.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f78c77fe700 1 -- 172.21.15.70:0/2325101381 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78e0031c40 con 0x563e2d049650 2023-10-18T12:38:30.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 10 client.17388 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:30.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563e2cded2d0 con 0x563e2d049650 2023-10-18T12:38:30.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f78c77fe700 1 -- 172.21.15.70:0/2325101381 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e0007e40 con 0x563e2d049650 2023-10-18T12:38:30.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f78c77fe700 1 -- 172.21.15.70:0/2325101381 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f78e00136b0 con 0x563e2d049650 2023-10-18T12:38:30.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 10 client.17388 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:30.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:30.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 20 client.17388 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:30.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d0b8ee0 0x563e2d0bb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 4 client.17388 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:38:30.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563e2cf94ed0 con 0x563e2d0b8ee0 2023-10-18T12:38:30.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.560+0000 7f78e77fe700 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d0b8ee0 0x563e2d0bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.561+0000 7f78c77fe700 10 client.17388 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.561+0000 7f78e77fe700 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d0b8ee0 0x563e2d0bb300 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941f75706a32e780 server_cookie=7157ed5941e124e1 in_seq=0 out_seq=0 2023-10-18T12:38:30.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f78c77fe700 1 -- 172.21.15.70:0/2325101381 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563e2cf94ed0 con 0x563e2d0b8ee0 2023-10-18T12:38:30.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f78c77fe700 10 client.17388 handle_command_reply: tid=0 2023-10-18T12:38:30.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 client.17388 shutdown 2023-10-18T12:38:30.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 msgr2=0x563e2d0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d0b1020 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f78e0006b30 tx=0x7f78e0031380 comp rx=0 tx=0).stop 2023-10-18T12:38:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 shutdown_connections 2023-10-18T12:38:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x563e2d0b8ee0 0x563e2d0bb300 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e2d04f580 0x563e2d059100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563e2d049650 0x563e2d0b1020 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f78c77fe700 0 client.17388 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 --2- 172.21.15.70:0/2325101381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e2d04ec60 0x563e2d0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.562+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 >> 172.21.15.70:0/2325101381 conn(0x563e2cfbd7a0 msgr2=0x563e2d03d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 shutdown_connections 2023-10-18T12:38:30.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f790226eb80 1 -- 172.21.15.70:0/2325101381 wait complete. 2023-10-18T12:38:30.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f790226eb80 20 client.17388 trim_cache size 0 max 16384 2023-10-18T12:38:30.579 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:38:30.579 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:38:30.579 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "f5cc7d1c-d106-4148-9dca-64acd6c5fb47", 2023-10-18T12:38:30.579 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:38:30.579 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:38:30.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f78d40a16d0 msgr2=0x7f78d40a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f78d40a16d0 0x7f78d40a3b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78e0007900 tx=0x7f78e0008040 comp rx=0 tx=0).stop 2023-10-18T12:38:30.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 msgr2=0x7f78e810abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810abe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f78d8006070 tx=0x7f78d80123f0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 shutdown_connections 2023-10-18T12:38:30.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8106730 0x7f78e81a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e8100720 0x7f78e810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f78d40a16d0 0x7f78d40a3b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 --2- 172.21.15.70:0/2810458533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e81010e0 0x7f78e810abe0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.563+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 >> 172.21.15.70:0/2810458533 conn(0x7f78e80fc1b0 msgr2=0x7f78e810d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.564+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 shutdown_connections 2023-10-18T12:38:30.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.564+0000 7f78ee9ff700 1 -- 172.21.15.70:0/2810458533 wait complete. 2023-10-18T12:38:30.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:38:30.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/2739221168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c48106720 msgr2=0x7f3c48106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/2739221168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c48106720 0x7f3c48106b00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c380098f0 tx=0x7f3c38009c00 comp rx=0 tx=0).stop 2023-10-18T12:38:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/2739221168 shutdown_connections 2023-10-18T12:38:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/2739221168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c481010d0 0x7f3c4810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/2739221168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c48106720 0x7f3c48106b00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/2739221168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48100710 0x7f3c48100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/2739221168 >> 172.21.15.70:0/2739221168 conn(0x7f3c480fc200 msgr2=0x7f3c480fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/2739221168 shutdown_connections 2023-10-18T12:38:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.791+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/2739221168 wait complete. 2023-10-18T12:38:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 Processor -- start 2023-10-18T12:38:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 -- start start 2023-10-18T12:38:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c48100710 0x7f3c48198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 0x7f3c48199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 0x7f3c4819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4819dd10 con 0x7f3c481010d0 2023-10-18T12:38:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4819de80 con 0x7f3c48106720 2023-10-18T12:38:30.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c4d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4819e040 con 0x7f3c48100710 2023-10-18T12:38:30.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 0x7f3c48199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c477fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 0x7f3c4819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 0x7f3c48199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35030/0 (socket says 172.21.15.70:35030) 2023-10-18T12:38:30.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c477fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 0x7f3c4819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51730/0 (socket says 172.21.15.70:51730) 2023-10-18T12:38:30.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c467fc700 1 -- 172.21.15.70:0/1356395194 learned_addr learned my addr 172.21.15.70:0/1356395194 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.793+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c48100710 0x7f3c48198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c477fe700 1 -- 172.21.15.70:0/1356395194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c48100710 msgr2=0x7f3c48198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c477fe700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c48100710 0x7f3c48198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c477fe700 1 -- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 msgr2=0x7f3c48199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c477fe700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 0x7f3c48199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c477fe700 1 -- 172.21.15.70:0/1356395194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009520 con 0x7f3c48106720 2023-10-18T12:38:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 0x7f3c48199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c477fe700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 0x7f3c4819d6d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000efc0 tx=0x7f3c4000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c4c9fd700 1 -- 172.21.15.70:0/1356395194 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40018070 con 0x7f3c48106720 2023-10-18T12:38:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c4c9fd700 1 -- 172.21.15.70:0/1356395194 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4000f040 con 0x7f3c48106720 2023-10-18T12:38:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4819e2c0 con 0x7f3c48106720 2023-10-18T12:38:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c4c9fd700 1 -- 172.21.15.70:0/1356395194 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000cd40 con 0x7f3c48106720 2023-10-18T12:38:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.794+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c481a5b60 con 0x7f3c48106720 2023-10-18T12:38:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.798+0000 7f3c4c9fd700 1 -- 172.21.15.70:0/1356395194 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3c40003a00 con 0x7f3c48106720 2023-10-18T12:38:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.799+0000 7f3c4c9fd700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3c300a15f0 0x7f3c300a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.799+0000 7f3c612aeb80 1 Processor -- start 2023-10-18T12:38:30.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.799+0000 7f3c4c9fd700 1 -- 172.21.15.70:0/1356395194 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3c400ccfe0 con 0x7f3c48106720 2023-10-18T12:38:30.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.799+0000 7f3c612aeb80 1 -- start start 2023-10-18T12:38:30.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.799+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3c300a15f0 0x7f3c300a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe70be0 0x5604abe70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe6b5d0 0x5604abe6ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abdde6e0 con 0x5604abe71500 2023-10-18T12:38:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abdc4c60 con 0x5604abe6b5d0 2023-10-18T12:38:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abdc2580 con 0x5604abe70be0 2023-10-18T12:38:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe70be0 0x5604abe70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35036/0 (socket says 172.21.15.70:35036) 2023-10-18T12:38:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 -- 172.21.15.70:0/1152446053 learned_addr learned my addr 172.21.15.70:0/1152446053 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c477fe700 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe6b5d0 0x5604abe6ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3c300a15f0 0x7f3c300a3ab0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3c380098c0 tx=0x7f3c38019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 -- 172.21.15.70:0/1152446053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe70be0 msgr2=0x5604abe70fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe70be0 0x5604abe70fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 -- 172.21.15.70:0/1152446053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe6b5d0 msgr2=0x5604abe6ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe6b5d0 0x5604abe6ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c467fc700 1 -- 172.21.15.70:0/1152446053 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604abd7b3f0 con 0x5604abe71500 2023-10-18T12:38:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.800+0000 7f3c477fe700 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe6b5d0 0x5604abe6ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.801+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe6b090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c011680 tx=0x7f3c3c011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145f0f1488322ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.801+0000 7f3c2dffb700 1 -- 172.21.15.70:0/1152446053 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c004030 con 0x5604abe71500 2023-10-18T12:38:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.801+0000 7f3c2dffb700 1 -- 172.21.15.70:0/1152446053 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3c003680 con 0x5604abe71500 2023-10-18T12:38:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.801+0000 7f3c2dffb700 1 -- 172.21.15.70:0/1152446053 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0196d0 con 0x5604abe71500 2023-10-18T12:38:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1152446053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 msgr2=0x5604abe6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe6b090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c011680 tx=0x7f3c3c011990 comp rx=0 tx=0).stop 2023-10-18T12:38:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1152446053 shutdown_connections 2023-10-18T12:38:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe6b5d0 0x5604abe6ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe70be0 0x5604abe70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1152446053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe6b090 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1152446053 >> 172.21.15.70:0/1152446053 conn(0x5604abddf720 msgr2=0x5604abe65b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1152446053 shutdown_connections 2023-10-18T12:38:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1152446053 wait complete. 2023-10-18T12:38:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.802+0000 7f3c612aeb80 1 -- start start 2023-10-18T12:38:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 Processor -- start 2023-10-18T12:38:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe6b5d0 0x5604abe775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe74900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe71500 0x5604abe75040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abdbde70 con 0x5604abe6b5d0 2023-10-18T12:38:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abd5a710 con 0x5604abe70be0 2023-10-18T12:38:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abd5a3b0 con 0x5604abe71500 2023-10-18T12:38:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c612aeb80 10 client.0 _ll_register_callbacks cb 0x5604abca4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe74900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c477fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe71500 0x5604abe75040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe6b5d0 0x5604abe775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe74900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51754/0 (socket says 172.21.15.70:51754) 2023-10-18T12:38:30.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.803+0000 7f3c467fc700 1 -- 172.21.15.70:0/3984397152 learned_addr learned my addr 172.21.15.70:0/3984397152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c467fc700 1 -- 172.21.15.70:0/3984397152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe71500 msgr2=0x5604abe75040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe71500 0x5604abe75040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c467fc700 1 -- 172.21.15.70:0/3984397152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe6b5d0 msgr2=0x5604abe775a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe6b5d0 0x5604abe775a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c467fc700 1 -- 172.21.15.70:0/3984397152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c481a5b60 con 0x5604abe70be0 2023-10-18T12:38:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe74900 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c011650 tx=0x7f3c3c0042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480f8cf8c9a339d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:38:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3984397152 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c003db0 con 0x5604abe70be0 2023-10-18T12:38:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604abdc2ef0 con 0x5604abe70be0 2023-10-18T12:38:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3984397152 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3c3c005260 con 0x5604abe70be0 2023-10-18T12:38:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3984397152 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c3c005420 con 0x5604abe70be0 2023-10-18T12:38:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c612aeb80 10 client.27455 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.804+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f3c3c009e10 con 0x5604abe70be0 2023-10-18T12:38:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3984397152 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604abdc2ef0 con 0x5604abe70be0 2023-10-18T12:38:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3984397152 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3c3c022480 con 0x5604abe70be0 2023-10-18T12:38:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c612aeb80 10 client.27455 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c612aeb80 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c612aeb80 20 client.27455 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abe84f10 0x5604abe87330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c612aeb80 4 client.27455 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:38:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604abe3a8b0 con 0x5604abe84f10 2023-10-18T12:38:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.805+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abe84f10 0x5604abe87330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.806+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abe84f10 0x5604abe87330 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7f2bc3242a819b server_cookie=696a99c205806c74 in_seq=0 out_seq=0 2023-10-18T12:38:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.806+0000 7f3c26ffd700 10 client.27455 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.806+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3984397152 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5604abe3a8b0 con 0x5604abe84f10 2023-10-18T12:38:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.806+0000 7f3c26ffd700 10 client.27455 handle_command_reply: tid=0 2023-10-18T12:38:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.806+0000 7f3c612aeb80 1 client.27455 shutdown 2023-10-18T12:38:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 msgr2=0x5604abe74900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe74900 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c011650 tx=0x7f3c3c0042b0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 shutdown_connections 2023-10-18T12:38:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abe84f10 0x5604abe87330 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe71500 0x5604abe75040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c26ffd700 0 client.27455 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe6b5d0 0x5604abe775a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3984397152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe74900 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 >> 172.21.15.70:0/3984397152 conn(0x5604abddf720 msgr2=0x5604abe6a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 shutdown_connections 2023-10-18T12:38:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3984397152 wait complete. 2023-10-18T12:38:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.807+0000 7f3c612aeb80 20 client.27455 trim_cache size 0 max 16384 2023-10-18T12:38:30.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.824+0000 7f3c612aeb80 1 Processor -- start 2023-10-18T12:38:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.824+0000 7f3c612aeb80 1 -- start start 2023-10-18T12:38:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abc3e920 con 0x5604abe71500 2023-10-18T12:38:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abe81ae0 con 0x5604abe70be0 2023-10-18T12:38:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abe67560 con 0x5604abe6b5d0 2023-10-18T12:38:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c477fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:51766/0 (socket says 172.21.15.70:51766) 2023-10-18T12:38:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 -- 172.21.15.70:0/1648218377 learned_addr learned my addr 172.21.15.70:0/1648218377 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 -- 172.21.15.70:0/1648218377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 msgr2=0x5604abe6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 -- 172.21.15.70:0/1648218377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 msgr2=0x5604abe731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c467fc700 1 -- 172.21.15.70:0/1648218377 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604abbe8de0 con 0x5604abe70be0 2023-10-18T12:38:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c477fe700 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.825+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.826+0000 7f3c467fc700 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7bf10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0115a0 tx=0x7f3c3c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd9b76be9cda644 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.826+0000 7f3c2e7fc700 1 -- 172.21.15.70:0/1648218377 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c022680 con 0x5604abe70be0 2023-10-18T12:38:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.826+0000 7f3c2e7fc700 1 -- 172.21.15.70:0/1648218377 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3c005720 con 0x5604abe70be0 2023-10-18T12:38:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.826+0000 7f3c2e7fc700 1 -- 172.21.15.70:0/1648218377 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0198b0 con 0x5604abe70be0 2023-10-18T12:38:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.826+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1648218377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 msgr2=0x5604abe7bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.826+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7bf10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0115a0 tx=0x7f3c3c009c60 comp rx=0 tx=0).stop 2023-10-18T12:38:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1648218377 shutdown_connections 2023-10-18T12:38:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/1648218377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7bf10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1648218377 >> 172.21.15.70:0/1648218377 conn(0x5604abddf720 msgr2=0x5604abe65810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1648218377 shutdown_connections 2023-10-18T12:38:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 -- 172.21.15.70:0/1648218377 wait complete. 2023-10-18T12:38:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 -- start start 2023-10-18T12:38:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.827+0000 7f3c612aeb80 1 Processor -- start 2023-10-18T12:38:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe7a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe7b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abe7c4e0 con 0x5604abe71500 2023-10-18T12:38:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abed8d10 con 0x5604abe70be0 2023-10-18T12:38:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604abed8e80 con 0x5604abe6b5d0 2023-10-18T12:38:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c612aeb80 10 client.0 _ll_register_callbacks cb 0x5604abca4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:38:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe7a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c46ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe7a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:57194/0 (socket says 172.21.15.70:57194) 2023-10-18T12:38:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c46ffd700 1 -- 172.21.15.70:0/3294333927 learned_addr learned my addr 172.21.15.70:0/3294333927 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:38:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.828+0000 7f3c477fe700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe7b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c46ffd700 1 -- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 msgr2=0x5604abe7ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c46ffd700 1 -- 172.21.15.70:0/3294333927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 msgr2=0x5604abe7b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:38:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe7b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c46ffd700 1 -- 172.21.15.70:0/3294333927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4810dee0 con 0x5604abe6b5d0 2023-10-18T12:38:30.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:38:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c46ffd700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe7a5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005bf0 tx=0x7f3c38009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c917f2de62376e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:38:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:38:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3294333927 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380322d0 con 0x5604abe6b5d0 2023-10-18T12:38:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3294333927 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3c38032430 con 0x5604abe6b5d0 2023-10-18T12:38:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.829+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c3c003680 con 0x5604abe6b5d0 2023-10-18T12:38:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3294333927 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c38032e80 con 0x5604abe6b5d0 2023-10-18T12:38:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 10 client.27445 fetch_fsmap learned FSMap version 20 2023-10-18T12:38:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f3c4819e2c0 con 0x5604abe6b5d0 2023-10-18T12:38:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3294333927 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3c003680 con 0x5604abe6b5d0 2023-10-18T12:38:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3294333927 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3c3800b930 con 0x5604abe6b5d0 2023-10-18T12:38:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 10 client.27445 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:38:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:38:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 20 client.27445 populate_metadata read hostname 'smithi070' 2023-10-18T12:38:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abeda5f0 0x5604abedca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:38:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 4 client.27445 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:38:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604abdb6e50 con 0x5604abeda5f0 2023-10-18T12:38:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.830+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abeda5f0 0x5604abedca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:38:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.831+0000 7f3c26ffd700 10 client.27445 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.831+0000 7f3c467fc700 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abeda5f0 0x5604abedca10 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18085eed3b36b79 server_cookie=25cb15a558c85ac2 in_seq=0 out_seq=0 2023-10-18T12:38:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c26ffd700 1 -- 172.21.15.70:0/3294333927 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5604abdb6e50 con 0x5604abeda5f0 2023-10-18T12:38:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c26ffd700 10 client.27445 handle_command_reply: tid=0 2023-10-18T12:38:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 client.27445 shutdown 2023-10-18T12:38:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 msgr2=0x5604abe7a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe7a5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005bf0 tx=0x7f3c38009c00 comp rx=0 tx=0).stop 2023-10-18T12:38:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 shutdown_connections 2023-10-18T12:38:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x5604abeda5f0 0x5604abedca10 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5604abe71500 0x5604abe7b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5604abe6b5d0 0x5604abe7a5d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 --2- 172.21.15.70:0/3294333927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5604abe70be0 0x5604abe7ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c26ffd700 0 client.27445 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:38:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 >> 172.21.15.70:0/3294333927 conn(0x5604abddf720 msgr2=0x5604abe73110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.832+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 shutdown_connections 2023-10-18T12:38:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.833+0000 7f3c612aeb80 1 -- 172.21.15.70:0/3294333927 wait complete. 2023-10-18T12:38:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.833+0000 7f3c612aeb80 20 client.27445 trim_cache size 0 max 16384 2023-10-18T12:38:30.852 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:38:30.852 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:38:30.852 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:38:30.852 INFO:teuthology.orchestra.run.smithi070.stdout: "f5cc7d1c-d106-4148-9dca-64acd6c5fb47": { 2023-10-18T12:38:30.853 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:38:30.853 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "f5cc7d1c-d106-4148-9dca-64acd6c5fb47", 2023-10-18T12:38:30.853 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:38:30.853 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:38:30.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:38:30.854 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:38:30.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.833+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3c300a15f0 msgr2=0x7f3c300a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.833+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3c300a15f0 0x7f3c300a3ab0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3c380098c0 tx=0x7f3c38019040 comp rx=0 tx=0).stop 2023-10-18T12:38:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.833+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 msgr2=0x7f3c4819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:38:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.833+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 0x7f3c4819d6d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000efc0 tx=0x7f3c4000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 shutdown_connections 2023-10-18T12:38:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f3c300a15f0 0x7f3c300a3ab0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c48106720 0x7f3c4819d6d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c48100710 0x7f3c48198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 --2- 172.21.15.70:0/1356395194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c481010d0 0x7f3c48199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:38:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 >> 172.21.15.70:0/1356395194 conn(0x7f3c480fc200 msgr2=0x7f3c4810be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:38:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 shutdown_connections 2023-10-18T12:38:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:38:30.834+0000 7f3c4d9ff700 1 -- 172.21.15.70:0/1356395194 wait complete. 2023-10-18T12:38:30.857 INFO:tasks.cephfs.filesystem:scrub status for tag:f5cc7d1c-d106-4148-9dca-64acd6c5fb47 - {'path': '/', 'tag': 'f5cc7d1c-d106-4148-9dca-64acd6c5fb47', 'options': 'recursive,force'} 2023-10-18T12:38:30.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1402047 11.02 MB/sec execute 539 sec latency 19.558 ms 2023-10-18T12:38:31.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:30 smithi070 ceph-mon[98639]: pgmap v701: 129 pgs: 129 active+clean; 571 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 224 op/s 2023-10-18T12:38:31.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:30 smithi152 ceph-mon[134767]: pgmap v701: 129 pgs: 129 active+clean; 571 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 224 op/s 2023-10-18T12:38:31.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:30 smithi186 ceph-mon[135253]: pgmap v701: 129 pgs: 129 active+clean; 571 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 224 op/s 2023-10-18T12:38:31.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1404006 11.02 MB/sec execute 540 sec latency 63.548 ms 2023-10-18T12:38:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:31 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:38:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:31 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:31 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:38:32.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:31 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:32.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:31 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:38:32.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:31 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:32.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:31 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:38:32.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:31 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:32.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:31 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:38:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:31 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:31 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:38:32.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:31 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:38:32.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1405863 11.02 MB/sec execute 541 sec latency 44.232 ms 2023-10-18T12:38:33.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:32 smithi152 ceph-mon[134767]: pgmap v702: 129 pgs: 129 active+clean; 575 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 187 op/s 2023-10-18T12:38:33.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:32 smithi186 ceph-mon[135253]: pgmap v702: 129 pgs: 129 active+clean; 575 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 187 op/s 2023-10-18T12:38:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:32 smithi070 ceph-mon[98639]: pgmap v702: 129 pgs: 129 active+clean; 575 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 187 op/s 2023-10-18T12:38:33.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1407777 11.02 MB/sec execute 542 sec latency 20.653 ms 2023-10-18T12:38:34.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:33 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:38:34.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:33 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:38:34.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:33 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:38:34.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1410345 11.02 MB/sec execute 543 sec latency 15.911 ms 2023-10-18T12:38:35.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:34 smithi152 ceph-mon[134767]: pgmap v703: 129 pgs: 129 active+clean; 578 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-18T12:38:35.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:34 smithi186 ceph-mon[135253]: pgmap v703: 129 pgs: 129 active+clean; 578 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-18T12:38:35.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:34 smithi070 ceph-mon[98639]: pgmap v703: 129 pgs: 129 active+clean; 578 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 186 op/s 2023-10-18T12:38:35.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1411796 11.02 MB/sec execute 544 sec latency 16.485 ms 2023-10-18T12:38:36.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1414156 11.02 MB/sec execute 545 sec latency 41.424 ms 2023-10-18T12:38:37.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:36 smithi152 ceph-mon[134767]: pgmap v704: 129 pgs: 129 active+clean; 585 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2023-10-18T12:38:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:36 smithi186 ceph-mon[135253]: pgmap v704: 129 pgs: 129 active+clean; 585 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2023-10-18T12:38:37.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:36 smithi070 ceph-mon[98639]: pgmap v704: 129 pgs: 129 active+clean; 585 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2023-10-18T12:38:37.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1415077 11.01 MB/sec execute 546 sec latency 127.592 ms 2023-10-18T12:38:38.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1417444 11.01 MB/sec execute 547 sec latency 36.227 ms 2023-10-18T12:38:39.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:38 smithi152 ceph-mon[134767]: pgmap v705: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 189 op/s 2023-10-18T12:38:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:38 smithi186 ceph-mon[135253]: pgmap v705: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 189 op/s 2023-10-18T12:38:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:38 smithi070 ceph-mon[98639]: pgmap v705: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 189 op/s 2023-10-18T12:38:39.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1419064 11.01 MB/sec execute 548 sec latency 15.620 ms 2023-10-18T12:38:40.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:40 smithi152 ceph-mon[134767]: pgmap v706: 129 pgs: 129 active+clean; 603 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 203 op/s 2023-10-18T12:38:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:40 smithi186 ceph-mon[135253]: pgmap v706: 129 pgs: 129 active+clean; 603 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 203 op/s 2023-10-18T12:38:40.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:40 smithi070 ceph-mon[98639]: pgmap v706: 129 pgs: 129 active+clean; 603 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 203 op/s 2023-10-18T12:38:40.887 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1421456 11.02 MB/sec execute 549 sec latency 14.453 ms 2023-10-18T12:38:41.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1423005 11.01 MB/sec execute 550 sec latency 23.892 ms 2023-10-18T12:38:42.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1424948 11.01 MB/sec execute 551 sec latency 93.463 ms 2023-10-18T12:38:43.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:42 smithi152 ceph-mon[134767]: pgmap v707: 129 pgs: 129 active+clean; 606 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 195 op/s 2023-10-18T12:38:43.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:42 smithi186 ceph-mon[135253]: pgmap v707: 129 pgs: 129 active+clean; 606 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 195 op/s 2023-10-18T12:38:43.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:42 smithi070 ceph-mon[98639]: pgmap v707: 129 pgs: 129 active+clean; 606 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 195 op/s 2023-10-18T12:38:43.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1426331 11.00 MB/sec execute 552 sec latency 26.246 ms 2023-10-18T12:38:44.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1428495 11.01 MB/sec execute 553 sec latency 70.917 ms 2023-10-18T12:38:45.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:44 smithi152 ceph-mon[134767]: pgmap v708: 129 pgs: 129 active+clean; 609 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 199 op/s 2023-10-18T12:38:45.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:44 smithi186 ceph-mon[135253]: pgmap v708: 129 pgs: 129 active+clean; 609 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 199 op/s 2023-10-18T12:38:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:44 smithi070 ceph-mon[98639]: pgmap v708: 129 pgs: 129 active+clean; 609 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 199 op/s 2023-10-18T12:38:45.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1430247 11.00 MB/sec execute 554 sec latency 14.793 ms 2023-10-18T12:38:46.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1432261 11.00 MB/sec execute 555 sec latency 36.622 ms 2023-10-18T12:38:47.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:46 smithi152 ceph-mon[134767]: pgmap v709: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 194 op/s 2023-10-18T12:38:47.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:46 smithi186 ceph-mon[135253]: pgmap v709: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 194 op/s 2023-10-18T12:38:47.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:46 smithi070 ceph-mon[98639]: pgmap v709: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 194 op/s 2023-10-18T12:38:47.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1433903 10.99 MB/sec execute 556 sec latency 153.529 ms 2023-10-18T12:38:48.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1435726 11.00 MB/sec execute 557 sec latency 57.074 ms 2023-10-18T12:38:49.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:48 smithi152 ceph-mon[134767]: pgmap v710: 129 pgs: 129 active+clean; 617 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 185 op/s 2023-10-18T12:38:49.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:48 smithi186 ceph-mon[135253]: pgmap v710: 129 pgs: 129 active+clean; 617 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 185 op/s 2023-10-18T12:38:49.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:48 smithi070 ceph-mon[98639]: pgmap v710: 129 pgs: 129 active+clean; 617 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 185 op/s 2023-10-18T12:38:49.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1437562 10.99 MB/sec execute 558 sec latency 20.606 ms 2023-10-18T12:38:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:38:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:50 smithi152 ceph-mon[134767]: pgmap v711: 129 pgs: 129 active+clean; 636 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 224 op/s 2023-10-18T12:38:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:50.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:50 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:38:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:50 smithi186 ceph-mon[135253]: pgmap v711: 129 pgs: 129 active+clean; 636 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 224 op/s 2023-10-18T12:38:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:50 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:50.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:38:50.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:50 smithi070 ceph-mon[98639]: pgmap v711: 129 pgs: 129 active+clean; 636 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 224 op/s 2023-10-18T12:38:50.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:50.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:50 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:50.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1439290 10.99 MB/sec execute 559 sec latency 41.162 ms 2023-10-18T12:38:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:38:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:38:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:38:51.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:38:51.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:51.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:38:51.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:38:51.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:38:51.888 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1441240 10.98 MB/sec execute 560 sec latency 32.904 ms 2023-10-18T12:38:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:52 smithi152 ceph-mon[134767]: pgmap v712: 129 pgs: 129 active+clean; 646 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 190 op/s 2023-10-18T12:38:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:52 smithi186 ceph-mon[135253]: pgmap v712: 129 pgs: 129 active+clean; 646 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 190 op/s 2023-10-18T12:38:52.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:52 smithi070 ceph-mon[98639]: pgmap v712: 129 pgs: 129 active+clean; 646 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 190 op/s 2023-10-18T12:38:52.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1442852 10.99 MB/sec execute 561 sec latency 54.399 ms 2023-10-18T12:38:53.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1444408 10.98 MB/sec execute 562 sec latency 46.030 ms 2023-10-18T12:38:54.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1446484 10.98 MB/sec execute 563 sec latency 12.647 ms 2023-10-18T12:38:55.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:54 smithi152 ceph-mon[134767]: pgmap v713: 129 pgs: 129 active+clean; 640 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 191 op/s 2023-10-18T12:38:55.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:54 smithi186 ceph-mon[135253]: pgmap v713: 129 pgs: 129 active+clean; 640 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 191 op/s 2023-10-18T12:38:55.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:54 smithi070 ceph-mon[98639]: pgmap v713: 129 pgs: 129 active+clean; 640 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 191 op/s 2023-10-18T12:38:55.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1448619 10.98 MB/sec execute 564 sec latency 38.676 ms 2023-10-18T12:38:56.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1450385 10.98 MB/sec execute 565 sec latency 15.106 ms 2023-10-18T12:38:57.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:56 smithi152 ceph-mon[134767]: pgmap v714: 129 pgs: 129 active+clean; 621 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 182 op/s 2023-10-18T12:38:57.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:56 smithi186 ceph-mon[135253]: pgmap v714: 129 pgs: 129 active+clean; 621 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 182 op/s 2023-10-18T12:38:57.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:56 smithi070 ceph-mon[98639]: pgmap v714: 129 pgs: 129 active+clean; 621 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 182 op/s 2023-10-18T12:38:57.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1452870 10.98 MB/sec execute 566 sec latency 28.044 ms 2023-10-18T12:38:58.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1453692 10.97 MB/sec execute 567 sec latency 42.597 ms 2023-10-18T12:38:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:38:58 smithi152 ceph-mon[134767]: pgmap v715: 129 pgs: 129 active+clean; 605 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 196 op/s 2023-10-18T12:38:59.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:38:58 smithi186 ceph-mon[135253]: pgmap v715: 129 pgs: 129 active+clean; 605 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 196 op/s 2023-10-18T12:38:59.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:38:58 smithi070 ceph-mon[98639]: pgmap v715: 129 pgs: 129 active+clean; 605 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 196 op/s 2023-10-18T12:38:59.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1455726 10.97 MB/sec execute 568 sec latency 50.511 ms 2023-10-18T12:39:00.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:00 smithi152 ceph-mon[134767]: pgmap v716: 129 pgs: 129 active+clean; 591 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 214 op/s 2023-10-18T12:39:00.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:00 smithi186 ceph-mon[135253]: pgmap v716: 129 pgs: 129 active+clean; 591 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 214 op/s 2023-10-18T12:39:00.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:00 smithi070 ceph-mon[98639]: pgmap v716: 129 pgs: 129 active+clean; 591 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 214 op/s 2023-10-18T12:39:00.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:39:00.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1457409 10.97 MB/sec execute 569 sec latency 18.451 ms 2023-10-18T12:39:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 -- 172.21.15.70:0/1429108745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 msgr2=0x7f73fc1055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 --2- 172.21.15.70:0/1429108745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 0x7f73fc1055f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009990 tx=0x7f73ec009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 -- 172.21.15.70:0/1429108745 shutdown_connections 2023-10-18T12:39:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 --2- 172.21.15.70:0/1429108745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc105b30 0x7f73fc10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 --2- 172.21.15.70:0/1429108745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc067d40 0x7f73fc068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 --2- 172.21.15.70:0/1429108745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 0x7f73fc1055f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 -- 172.21.15.70:0/1429108745 >> 172.21.15.70:0/1429108745 conn(0x7f73fc0767f0 msgr2=0x7f73fc076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 -- 172.21.15.70:0/1429108745 shutdown_connections 2023-10-18T12:39:01.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.096+0000 7f74035ff700 1 -- 172.21.15.70:0/1429108745 wait complete. 2023-10-18T12:39:01.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.097+0000 7f74035ff700 1 Processor -- start 2023-10-18T12:39:01.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 -- start start 2023-10-18T12:39:01.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 0x7f73fc198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 0x7f73fc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 0x7f73fc19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc19dd20 con 0x7f73fc067d40 2023-10-18T12:39:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc19de90 con 0x7f73fc105170 2023-10-18T12:39:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.098+0000 7f74035ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc19e050 con 0x7f73fc105b30 2023-10-18T12:39:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f740139b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 0x7f73fc198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 0x7f73fc19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f740139b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 0x7f73fc198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53256/0 (socket says 172.21.15.70:53256) 2023-10-18T12:39:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 0x7f73fc19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37176/0 (socket says 172.21.15.70:37176) 2023-10-18T12:39:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f740139b700 1 -- 172.21.15.70:0/3819656784 learned_addr learned my addr 172.21.15.70:0/3819656784 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7400b9a700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 0x7f73fc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 -- 172.21.15.70:0/3819656784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 msgr2=0x7f73fc199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 0x7f73fc199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 -- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 msgr2=0x7f73fc198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 0x7f73fc198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 -- 172.21.15.70:0/3819656784 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec0095f0 con 0x7f73fc105b30 2023-10-18T12:39:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f740139b700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 0x7f73fc198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.099+0000 7f7401b9c700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 0x7f73fc19d6e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f73f400efc0 tx=0x7f73f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.100+0000 7f73f27fc700 1 -- 172.21.15.70:0/3819656784 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4018070 con 0x7f73fc105b30 2023-10-18T12:39:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.100+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc19e2d0 con 0x7f73fc105b30 2023-10-18T12:39:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.100+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc1a5b70 con 0x7f73fc105b30 2023-10-18T12:39:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.100+0000 7f73f27fc700 1 -- 172.21.15.70:0/3819656784 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f400f040 con 0x7f73fc105b30 2023-10-18T12:39:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.100+0000 7f73f27fc700 1 -- 172.21.15.70:0/3819656784 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400cc80 con 0x7f73fc105b30 2023-10-18T12:39:01.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.104+0000 7f73f27fc700 1 -- 172.21.15.70:0/3819656784 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f73f4004000 con 0x7f73fc105b30 2023-10-18T12:39:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.106+0000 7f7416ee0b80 1 Processor -- start 2023-10-18T12:39:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.106+0000 7f73f27fc700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f73e80a1670 0x7f73e80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.106+0000 7f73f27fc700 1 -- 172.21.15.70:0/3819656784 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73f40cce70 con 0x7f73fc105b30 2023-10-18T12:39:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.106+0000 7f7416ee0b80 1 -- start start 2023-10-18T12:39:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.106+0000 7f7400b9a700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f73e80a1670 0x7f73e80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc07fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc02a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bb756e0 con 0x55673bc08500 2023-10-18T12:39:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bb5bc60 con 0x55673bc025d0 2023-10-18T12:39:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bb59580 con 0x55673bc07be0 2023-10-18T12:39:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f740139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc07fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f7401b9c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc02a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f740139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc07fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37190/0 (socket says 172.21.15.70:37190) 2023-10-18T12:39:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.107+0000 7f740139b700 1 -- 172.21.15.70:0/1274911569 learned_addr learned my addr 172.21.15.70:0/1274911569 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f7400b9a700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f73e80a1670 0x7f73e80a3b30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f73fc19a350 tx=0x7f73ec005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f740139b700 1 -- 172.21.15.70:0/1274911569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 msgr2=0x55673bc02a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f740139b700 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc02a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f740139b700 1 -- 172.21.15.70:0/1274911569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 msgr2=0x55673bc02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f7400b9a700 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f740139b700 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc02090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f740139b700 1 -- 172.21.15.70:0/1274911569 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55673bb123f0 con 0x55673bc07be0 2023-10-18T12:39:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f7400b9a700 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:39:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f740139b700 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc07fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f73f8010770 tx=0x7f73f8010a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ac06f7ed4b407e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f73e77fe700 1 -- 172.21.15.70:0/1274911569 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f801c680 con 0x55673bc07be0 2023-10-18T12:39:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f73e77fe700 1 -- 172.21.15.70:0/1274911569 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f8015070 con 0x55673bc07be0 2023-10-18T12:39:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.108+0000 7f73e77fe700 1 -- 172.21.15.70:0/1274911569 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8011bc0 con 0x55673bc07be0 2023-10-18T12:39:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1274911569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 msgr2=0x55673bc07fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc07fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f73f8010770 tx=0x7f73f8010a80 comp rx=0 tx=0).stop 2023-10-18T12:39:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1274911569 shutdown_connections 2023-10-18T12:39:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc07fc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1274911569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.109+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1274911569 >> 172.21.15.70:0/1274911569 conn(0x55673bb76720 msgr2=0x55673bbfcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.110+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1274911569 shutdown_connections 2023-10-18T12:39:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.110+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1274911569 wait complete. 2023-10-18T12:39:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.111+0000 7f7416ee0b80 1 -- start start 2023-10-18T12:39:01.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.111+0000 7f7416ee0b80 1 Processor -- start 2023-10-18T12:39:01.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc0e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc0b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc0bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bb54e70 con 0x55673bc08500 2023-10-18T12:39:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673baf1710 con 0x55673bc07be0 2023-10-18T12:39:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673baf13b0 con 0x55673bc025d0 2023-10-18T12:39:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7416ee0b80 10 client.0 _ll_register_callbacks cb 0x55673ba3ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f740139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc0e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f740139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc0e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37196/0 (socket says 172.21.15.70:37196) 2023-10-18T12:39:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f740139b700 1 -- 172.21.15.70:0/1504322066 learned_addr learned my addr 172.21.15.70:0/1504322066 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.112+0000 7f7400b9a700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc0b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7401b9c700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc0bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7400b9a700 1 -- 172.21.15.70:0/1504322066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 msgr2=0x55673bc0e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7400b9a700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc0e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7400b9a700 1 -- 172.21.15.70:0/1504322066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 msgr2=0x55673bc0bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7400b9a700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc0bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7400b9a700 1 -- 172.21.15.70:0/1504322066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f80103d0 con 0x55673bc07be0 2023-10-18T12:39:01.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f740139b700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc0e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:01.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f7400b9a700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc0b6a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b2e0 tx=0x7f73ec01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b8f667998ee8c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.113+0000 7f73e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:01.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f73e4ff9700 1 -- 172.21.15.70:0/1504322066 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec004e40 con 0x55673bc07be0 2023-10-18T12:39:01.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f73e4ff9700 1 -- 172.21.15.70:0/1504322066 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73ec030470 con 0x55673bc07be0 2023-10-18T12:39:01.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55673bb59ef0 con 0x55673bc07be0 2023-10-18T12:39:01.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f73e4ff9700 1 -- 172.21.15.70:0/1504322066 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73ec030630 con 0x55673bc07be0 2023-10-18T12:39:01.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 10 client.27473 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:01.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55673b97fde0 con 0x55673bc07be0 2023-10-18T12:39:01.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f73e4ff9700 1 -- 172.21.15.70:0/1504322066 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55673bb59ef0 con 0x55673bc07be0 2023-10-18T12:39:01.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f73e4ff9700 1 -- 172.21.15.70:0/1504322066 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f73ec00f810 con 0x55673bc07be0 2023-10-18T12:39:01.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 10 client.27473 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:01.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 20 client.27473 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc1bc30 0x55673bc1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 4 client.27473 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.114+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55673bbd18b0 con 0x55673bc1bc30 2023-10-18T12:39:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.115+0000 7f740139b700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc1bc30 0x55673bc1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.115+0000 7f740139b700 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc1bc30 0x55673bc1e050 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703eb0a186836e90 server_cookie=51e96d52206bea52 in_seq=0 out_seq=0 2023-10-18T12:39:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.115+0000 7f73e4ff9700 10 client.27473 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.116+0000 7f73e4ff9700 1 -- 172.21.15.70:0/1504322066 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55673bbd18b0 con 0x55673bc1bc30 2023-10-18T12:39:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.116+0000 7f73e4ff9700 10 client.27473 handle_command_reply: tid=0 2023-10-18T12:39:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.116+0000 7f7416ee0b80 1 client.27473 shutdown 2023-10-18T12:39:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.116+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 msgr2=0x55673bc0b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.116+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc0b6a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b2e0 tx=0x7f73ec01a050 comp rx=0 tx=0).stop 2023-10-18T12:39:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 shutdown_connections 2023-10-18T12:39:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc1bc30 0x55673bc1e050 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc0bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc0e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f73e4ff9700 0 client.27473 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:01.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/1504322066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc0b6a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 >> 172.21.15.70:0/1504322066 conn(0x55673bb76720 msgr2=0x55673bc01080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 shutdown_connections 2023-10-18T12:39:01.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 1 -- 172.21.15.70:0/1504322066 wait complete. 2023-10-18T12:39:01.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.117+0000 7f7416ee0b80 20 client.27473 trim_cache size 0 max 16384 2023-10-18T12:39:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 Processor -- start 2023-10-18T12:39:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 -- start start 2023-10-18T12:39:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc12f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc17660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673b9d5920 con 0x55673bc08500 2023-10-18T12:39:01.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.139+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bc0ec90 con 0x55673bc025d0 2023-10-18T12:39:01.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bc023f0 con 0x55673bc07be0 2023-10-18T12:39:01.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f7400b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc12f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f740139b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f7400b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc12f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37200/0 (socket says 172.21.15.70:37200) 2023-10-18T12:39:01.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f7400b9a700 1 -- 172.21.15.70:0/2610164200 learned_addr learned my addr 172.21.15.70:0/2610164200 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:01.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f7401b9c700 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc17660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f740139b700 1 -- 172.21.15.70:0/2610164200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 msgr2=0x55673bc12f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f740139b700 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc12f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f740139b700 1 -- 172.21.15.70:0/2610164200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 msgr2=0x55673bc17660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f740139b700 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc17660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.140+0000 7f740139b700 1 -- 172.21.15.70:0/2610164200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55673bbfcd70 con 0x55673bc025d0 2023-10-18T12:39:01.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.141+0000 7f740139b700 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc05dc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55673bc1e960 tx=0x7f73f800b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd1618697721e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.141+0000 7f73e7fff700 1 -- 172.21.15.70:0/2610164200 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8020070 con 0x55673bc025d0 2023-10-18T12:39:01.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.141+0000 7f73e7fff700 1 -- 172.21.15.70:0/2610164200 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f8021040 con 0x55673bc025d0 2023-10-18T12:39:01.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.141+0000 7f73e7fff700 1 -- 172.21.15.70:0/2610164200 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80283f0 con 0x55673bc025d0 2023-10-18T12:39:01.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 -- 172.21.15.70:0/2610164200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 msgr2=0x55673bc05dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc05dc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55673bc1e960 tx=0x7f73f800b700 comp rx=0 tx=0).stop 2023-10-18T12:39:01.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 -- 172.21.15.70:0/2610164200 shutdown_connections 2023-10-18T12:39:01.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc17660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc025d0 0x55673bc05dc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/2610164200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc07be0 0x55673bc12f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 -- 172.21.15.70:0/2610164200 >> 172.21.15.70:0/2610164200 conn(0x55673bb76720 msgr2=0x55673bc056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 -- 172.21.15.70:0/2610164200 shutdown_connections 2023-10-18T12:39:01.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.142+0000 7f7416ee0b80 1 -- 172.21.15.70:0/2610164200 wait complete. 2023-10-18T12:39:01.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.143+0000 7f7416ee0b80 1 -- start start 2023-10-18T12:39:01.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.143+0000 7f7416ee0b80 1 Processor -- start 2023-10-18T12:39:01.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc11860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bc6fbd0 con 0x55673bc08500 2023-10-18T12:39:01.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bc6fd40 con 0x55673bc07be0 2023-10-18T12:39:01.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673bc6ff00 con 0x55673bc025d0 2023-10-18T12:39:01.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7416ee0b80 10 client.0 _ll_register_callbacks cb 0x55673ba3ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:01.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f740139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc11860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7400b9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7400b9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc11da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58206/0 (socket says 172.21.15.70:58206) 2023-10-18T12:39:01.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f740139b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc11860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37202/0 (socket says 172.21.15.70:37202) 2023-10-18T12:39:01.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7400b9a700 1 -- 172.21.15.70:0/120527917 learned_addr learned my addr 172.21.15.70:0/120527917 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:01.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7401b9c700 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7400b9a700 1 -- 172.21.15.70:0/120527917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 msgr2=0x55673bc11860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.144+0000 7f7400b9a700 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc11860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7400b9a700 1 -- 172.21.15.70:0/120527917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 msgr2=0x55673bc122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:01.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7400b9a700 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc122e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7400b9a700 1 -- 172.21.15.70:0/120527917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55673bb123f0 con 0x55673bc07be0 2023-10-18T12:39:01.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7400b9a700 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc11da0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b2e0 tx=0x7f73ec00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497149d133869b58 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f73e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:01.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f73e4ff9700 1 -- 172.21.15.70:0/120527917 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00f660 con 0x55673bc07be0 2023-10-18T12:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f73e4ff9700 1 -- 172.21.15.70:0/120527917 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73ec004e50 con 0x55673bc07be0 2023-10-18T12:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73f8015070 con 0x55673bc07be0 2023-10-18T12:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f73e4ff9700 1 -- 172.21.15.70:0/120527917 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73ec00f7c0 con 0x55673bc07be0 2023-10-18T12:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7416ee0b80 10 client.27479 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.145+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55673b968c60 con 0x55673bc07be0 2023-10-18T12:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f73e4ff9700 1 -- 172.21.15.70:0/120527917 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f8015070 con 0x55673bc07be0 2023-10-18T12:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f73e4ff9700 1 -- 172.21.15.70:0/120527917 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f73ec004500 con 0x55673bc07be0 2023-10-18T12:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f7416ee0b80 10 client.27479 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f7416ee0b80 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f7416ee0b80 20 client.27479 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc712d0 0x55673bc736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f7416ee0b80 4 client.27479 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55673bb4de50 con 0x55673bc712d0 2023-10-18T12:39:01.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.146+0000 7f740139b700 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc712d0 0x55673bc736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f73e4ff9700 10 client.27479 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:01.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f740139b700 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc712d0 0x55673bc736f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41efecced5b5435 server_cookie=e942b88311b82045 in_seq=0 out_seq=0 2023-10-18T12:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f73e4ff9700 1 -- 172.21.15.70:0/120527917 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55673bb4de50 con 0x55673bc712d0 2023-10-18T12:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f73e4ff9700 10 client.27479 handle_command_reply: tid=0 2023-10-18T12:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 client.27479 shutdown 2023-10-18T12:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 msgr2=0x55673bc11da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc11da0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b2e0 tx=0x7f73ec00bfd0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 shutdown_connections 2023-10-18T12:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55673bc712d0 0x55673bc736f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55673bc08500 0x55673bc122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55673bc025d0 0x55673bc11860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.147+0000 7f7416ee0b80 1 --2- 172.21.15.70:0/120527917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55673bc07be0 0x55673bc11da0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.148+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 >> 172.21.15.70:0/120527917 conn(0x55673bb76720 msgr2=0x55673bc0ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.148+0000 7f73e4ff9700 0 client.27479 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:01.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.148+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 shutdown_connections 2023-10-18T12:39:01.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.148+0000 7f7416ee0b80 1 -- 172.21.15.70:0/120527917 wait complete. 2023-10-18T12:39:01.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.148+0000 7f7416ee0b80 20 client.27479 trim_cache size 0 max 16384 2023-10-18T12:39:01.163 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:39:01.163 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:39:01.163 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:39:01.164 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:39:01.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f73e80a1670 msgr2=0x7f73e80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f73e80a1670 0x7f73e80a3b30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f73fc19a350 tx=0x7f73ec005fd0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 msgr2=0x7f73fc19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 0x7f73fc19d6e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f73f400efc0 tx=0x7f73f400c3a0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 shutdown_connections 2023-10-18T12:39:01.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73fc105b30 0x7f73fc19d6e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73fc067d40 0x7f73fc198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f73e80a1670 0x7f73e80a3b30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 --2- 172.21.15.70:0/3819656784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73fc105170 0x7f73fc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 >> 172.21.15.70:0/3819656784 conn(0x7f73fc0767f0 msgr2=0x7f73fc10bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 shutdown_connections 2023-10-18T12:39:01.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.149+0000 7f74035ff700 1 -- 172.21.15.70:0/3819656784 wait complete. 2023-10-18T12:39:01.173 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:39:01.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:39:01.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 -- 172.21.15.70:0/446601102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100ef0 msgr2=0x7f576810d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 --2- 172.21.15.70:0/446601102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100ef0 0x7f576810d3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5760009990 tx=0x7f5760009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 -- 172.21.15.70:0/446601102 shutdown_connections 2023-10-18T12:39:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 --2- 172.21.15.70:0/446601102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100ef0 0x7f576810d3b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 --2- 172.21.15.70:0/446601102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57681065f0 0x7f57681069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 --2- 172.21.15.70:0/446601102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100530 0x7f57681009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 -- 172.21.15.70:0/446601102 >> 172.21.15.70:0/446601102 conn(0x7f57680fc040 msgr2=0x7f57680fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 -- 172.21.15.70:0/446601102 shutdown_connections 2023-10-18T12:39:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.430+0000 7f576e99d700 1 -- 172.21.15.70:0/446601102 wait complete. 2023-10-18T12:39:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 Processor -- start 2023-10-18T12:39:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 -- start start 2023-10-18T12:39:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100530 0x7f576819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 0x7f576819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57681065f0 0x7f57681989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768199040 con 0x7f57681065f0 2023-10-18T12:39:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.431+0000 7f576e99d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681991b0 con 0x7f5768100ef0 2023-10-18T12:39:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576e99d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768199370 con 0x7f5768100530 2023-10-18T12:39:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 0x7f576819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576e19c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57681065f0 0x7f57681989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d99b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100530 0x7f576819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 0x7f576819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58238/0 (socket says 172.21.15.70:58238) 2023-10-18T12:39:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 -- 172.21.15.70:0/1651240989 learned_addr learned my addr 172.21.15.70:0/1651240989 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 -- 172.21.15.70:0/1651240989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100530 msgr2=0x7f576819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100530 0x7f576819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 -- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57681065f0 msgr2=0x7f57681989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57681065f0 0x7f57681989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.432+0000 7f576d19a700 1 -- 172.21.15.70:0/1651240989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5758009520 con 0x7f5768100ef0 2023-10-18T12:39:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.433+0000 7f576d19a700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 0x7f576819eda0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f575800eb30 tx=0x7f575800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.433+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758009880 con 0x7f5768100ef0 2023-10-18T12:39:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.433+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57580044e0 con 0x7f5768100ef0 2023-10-18T12:39:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.433+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5764009c40 con 0x7f5768100ef0 2023-10-18T12:39:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.433+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758003d80 con 0x7f5768100ef0 2023-10-18T12:39:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.433+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57600095f0 con 0x7f5768100ef0 2023-10-18T12:39:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.434+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768101c50 con 0x7f5768100ef0 2023-10-18T12:39:01.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.436+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5758003f20 con 0x7f5768100ef0 2023-10-18T12:39:01.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.437+0000 7f575effd700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f57540a15f0 0x7f57540a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.438+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5758014070 con 0x7f5768100ef0 2023-10-18T12:39:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.438+0000 7f576d99b700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f57540a15f0 0x7f57540a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.438+0000 7f576d99b700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f57540a15f0 0x7f57540a3ab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f576400f6c0 tx=0x7f576400b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.440+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5758096990 con 0x7f5768100ef0 2023-10-18T12:39:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.795+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5768067830 con 0x7f5768100ef0 2023-10-18T12:39:01.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.796+0000 7f575effd700 1 -- 172.21.15.70:0/1651240989 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f5758099fb0 con 0x7f5768100ef0 2023-10-18T12:39:01.799 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:39:01.800 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:39:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f57540a15f0 msgr2=0x7f57540a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f57540a15f0 0x7f57540a3ab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f576400f6c0 tx=0x7f576400b040 comp rx=0 tx=0).stop 2023-10-18T12:39:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 msgr2=0x7f576819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 0x7f576819eda0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f575800eb30 tx=0x7f575800ee40 comp rx=0 tx=0).stop 2023-10-18T12:39:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 shutdown_connections 2023-10-18T12:39:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57681065f0 0x7f57681989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5768100530 0x7f576819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f57540a15f0 0x7f57540a3ab0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 --2- 172.21.15.70:0/1651240989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5768100ef0 0x7f576819eda0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 >> 172.21.15.70:0/1651240989 conn(0x7f57680fc040 msgr2=0x7f57680fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.799+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 shutdown_connections 2023-10-18T12:39:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:01.800+0000 7f576e99d700 1 -- 172.21.15.70:0/1651240989 wait complete. 2023-10-18T12:39:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:39:01.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:39:01.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1457700 10.95 MB/sec execute 570 sec latency 117.014 ms 2023-10-18T12:39:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 -- 172.21.15.70:0/1375193780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8100720 msgr2=0x7f5ea8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 --2- 172.21.15.70:0/1375193780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8100720 0x7f5ea8100ba0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009990 tx=0x7f5e98009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 -- 172.21.15.70:0/1375193780 shutdown_connections 2023-10-18T12:39:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 --2- 172.21.15.70:0/1375193780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 --2- 172.21.15.70:0/1375193780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ea8106730 0x7f5ea8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 --2- 172.21.15.70:0/1375193780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8100720 0x7f5ea8100ba0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 -- 172.21.15.70:0/1375193780 >> 172.21.15.70:0/1375193780 conn(0x7f5ea80fc1b0 msgr2=0x7f5ea80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 -- 172.21.15.70:0/1375193780 shutdown_connections 2023-10-18T12:39:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.082+0000 7f5eafbff700 1 -- 172.21.15.70:0/1375193780 wait complete. 2023-10-18T12:39:02.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.084+0000 7f5eafbff700 1 Processor -- start 2023-10-18T12:39:02.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.084+0000 7f5eafbff700 1 -- start start 2023-10-18T12:39:02.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.084+0000 7f5eafbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ea8100720 0x7f5ea819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.084+0000 7f5eafbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5eafbff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8106730 0x7f5ea81969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5eafbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8197030 con 0x7f5ea81010e0 2023-10-18T12:39:02.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5eafbff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea81971a0 con 0x7f5ea8106730 2023-10-18T12:39:02.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5eafbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8197360 con 0x7f5ea8100720 2023-10-18T12:39:02.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5eae19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8106730 0x7f5ea81969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53322/0 (socket says 172.21.15.70:53322) 2023-10-18T12:39:02.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 -- 172.21.15.70:0/943774045 learned_addr learned my addr 172.21.15.70:0/943774045 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:02.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead99b700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ea8100720 0x7f5ea819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 -- 172.21.15.70:0/943774045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ea8100720 msgr2=0x7f5ea819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ea8100720 0x7f5ea819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 -- 172.21.15.70:0/943774045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8106730 msgr2=0x7f5ea81969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8106730 0x7f5ea81969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 -- 172.21.15.70:0/943774045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980095f0 con 0x7f5ea81010e0 2023-10-18T12:39:02.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.085+0000 7f5ead19a700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea819cde0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5e98000c00 tx=0x7f5e98005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:02.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.086+0000 7f5e9effd700 1 -- 172.21.15.70:0/943774045 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9801d070 con 0x7f5ea81010e0 2023-10-18T12:39:02.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.086+0000 7f5e9effd700 1 -- 172.21.15.70:0/943774045 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e98022470 con 0x7f5ea81010e0 2023-10-18T12:39:02.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.086+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea81975e0 con 0x7f5ea81010e0 2023-10-18T12:39:02.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.086+0000 7f5e9effd700 1 -- 172.21.15.70:0/943774045 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98017650 con 0x7f5ea81010e0 2023-10-18T12:39:02.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.086+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81a3890 con 0x7f5ea81010e0 2023-10-18T12:39:02.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.089+0000 7f5e9effd700 1 -- 172.21.15.70:0/943774045 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5e980225e0 con 0x7f5ea81010e0 2023-10-18T12:39:02.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.091+0000 7f5ec34fdb80 1 Processor -- start 2023-10-18T12:39:02.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.091+0000 7f5ec34fdb80 1 -- start start 2023-10-18T12:39:02.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.091+0000 7f5e9effd700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e940a15f0 0x7f5e940a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.091+0000 7f5e9effd700 1 -- 172.21.15.70:0/943774045 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5e980ce7a0 con 0x7f5ea81010e0 2023-10-18T12:39:02.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.091+0000 7f5eae19c700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e940a15f0 0x7f5e940a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a5386e0 con 0x55ac2a5cabe0 2023-10-18T12:39:02.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a51ec60 con 0x55ac2a5cb500 2023-10-18T12:39:02.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a51c580 con 0x55ac2a5c55d0 2023-10-18T12:39:02.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead99b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58260/0 (socket says 172.21.15.70:58260) 2023-10-18T12:39:02.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 -- 172.21.15.70:0/3482504730 learned_addr learned my addr 172.21.15.70:0/3482504730 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:02.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5eae19c700 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 -- 172.21.15.70:0/3482504730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 msgr2=0x55ac2a5c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 -- 172.21.15.70:0/3482504730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 msgr2=0x55ac2a5cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 -- 172.21.15.70:0/3482504730 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac2a4d53f0 con 0x55ac2a5cb500 2023-10-18T12:39:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5ead19a700 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c5090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e9809b3e0 tx=0x7f5e98097790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19b36bfa3e58b68 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.092+0000 7f5eae19c700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e940a15f0 0x7f5e940a3ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000f870 tx=0x7f5ea000fc30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5e93fff700 1 -- 172.21.15.70:0/3482504730 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9809adb0 con 0x55ac2a5cb500 2023-10-18T12:39:02.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5e93fff700 1 -- 172.21.15.70:0/3482504730 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9809bb50 con 0x55ac2a5cb500 2023-10-18T12:39:02.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5e93fff700 1 -- 172.21.15.70:0/3482504730 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800fb00 con 0x55ac2a5cb500 2023-10-18T12:39:02.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3482504730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 msgr2=0x55ac2a5c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c5090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e9809b3e0 tx=0x7f5e98097790 comp rx=0 tx=0).stop 2023-10-18T12:39:02.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3482504730 shutdown_connections 2023-10-18T12:39:02.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3482504730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c5090 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3482504730 >> 172.21.15.70:0/3482504730 conn(0x55ac2a539720 msgr2=0x55ac2a5bfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.093+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3482504730 shutdown_connections 2023-10-18T12:39:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.094+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3482504730 wait complete. 2023-10-18T12:39:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.094+0000 7f5ec34fdb80 1 -- start start 2023-10-18T12:39:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.094+0000 7f5ec34fdb80 1 Processor -- start 2023-10-18T12:39:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a517e70 con 0x55ac2a5cabe0 2023-10-18T12:39:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a4b4710 con 0x55ac2a5cb500 2023-10-18T12:39:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a4b43b0 con 0x55ac2a5c55d0 2023-10-18T12:39:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ec34fdb80 10 client.0 _ll_register_callbacks cb 0x55ac2a3fea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53354/0 (socket says 172.21.15.70:53354) 2023-10-18T12:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ead19a700 1 -- 172.21.15.70:0/1148809074 learned_addr learned my addr 172.21.15.70:0/1148809074 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58264/0 (socket says 172.21.15.70:58264) 2023-10-18T12:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5ead99b700 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 -- 172.21.15.70:0/1148809074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 msgr2=0x55ac2a5d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 -- 172.21.15.70:0/1148809074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 msgr2=0x55ac2a5ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.095+0000 7f5eae19c700 1 -- 172.21.15.70:0/1148809074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81a3890 con 0x55ac2a5cb500 2023-10-18T12:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5eae19c700 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5cece0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0018ad0 tx=0x7f5ea0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c08f3468f14ba78 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5e917fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5e917fa700 1 -- 172.21.15.70:0/1148809074 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0029070 con 0x55ac2a5cb500 2023-10-18T12:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac2a51cef0 con 0x55ac2a5cb500 2023-10-18T12:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5e917fa700 1 -- 172.21.15.70:0/1148809074 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5ea00093f0 con 0x55ac2a5cb500 2023-10-18T12:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5e917fa700 1 -- 172.21.15.70:0/1148809074 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea000e040 con 0x55ac2a5cb500 2023-10-18T12:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5ec34fdb80 10 client.27497 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.096+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ac2a342de0 con 0x55ac2a5cb500 2023-10-18T12:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5e917fa700 1 -- 172.21.15.70:0/1148809074 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac2a51cef0 con 0x55ac2a5cb500 2023-10-18T12:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5e917fa700 1 -- 172.21.15.70:0/1148809074 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5ea002c020 con 0x55ac2a5cb500 2023-10-18T12:39:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ec34fdb80 10 client.27497 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ec34fdb80 10 client.27497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ec34fdb80 20 client.27497 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a5e7460 0x55ac2a5e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ec34fdb80 4 client.27497 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac2a5948b0 con 0x55ac2a5e7460 2023-10-18T12:39:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.097+0000 7f5ead99b700 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a5e7460 0x55ac2a5e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.098+0000 7f5ead99b700 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a5e7460 0x55ac2a5e9880 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfab9366342d8cb server_cookie=975e64788eb51b24 in_seq=0 out_seq=0 2023-10-18T12:39:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.098+0000 7f5e917fa700 10 client.27497 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:02.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.099+0000 7f5e917fa700 1 -- 172.21.15.70:0/1148809074 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac2a5948b0 con 0x55ac2a5e7460 2023-10-18T12:39:02.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.099+0000 7f5e917fa700 10 client.27497 handle_command_reply: tid=0 2023-10-18T12:39:02.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.099+0000 7f5ec34fdb80 1 client.27497 shutdown 2023-10-18T12:39:02.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 msgr2=0x55ac2a5cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5cece0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0018ad0 tx=0x7f5ea0004b40 comp rx=0 tx=0).stop 2023-10-18T12:39:02.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 shutdown_connections 2023-10-18T12:39:02.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a5e7460 0x55ac2a5e9880 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5cece0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5e917fa700 0 client.27497 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:02.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/1148809074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 >> 172.21.15.70:0/1148809074 conn(0x55ac2a539720 msgr2=0x55ac2a5bfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:02.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 shutdown_connections 2023-10-18T12:39:02.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/1148809074 wait complete. 2023-10-18T12:39:02.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.100+0000 7f5ec34fdb80 20 client.27497 trim_cache size 0 max 16384 2023-10-18T12:39:02.119 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:01 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1651240989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:39:02.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.114+0000 7f5ec34fdb80 1 Processor -- start 2023-10-18T12:39:02.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.114+0000 7f5ec34fdb80 1 -- start start 2023-10-18T12:39:02.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a398920 con 0x55ac2a5c55d0 2023-10-18T12:39:02.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a5d5290 con 0x55ac2a5cb500 2023-10-18T12:39:02.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a5cfd70 con 0x55ac2a5cabe0 2023-10-18T12:39:02.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead99b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5eae19c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37266/0 (socket says 172.21.15.70:37266) 2023-10-18T12:39:02.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 -- 172.21.15.70:0/405207034 learned_addr learned my addr 172.21.15.70:0/405207034 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:02.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 -- 172.21.15.70:0/405207034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 msgr2=0x55ac2a5d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 -- 172.21.15.70:0/405207034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 msgr2=0x55ac2a5c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead19a700 1 -- 172.21.15.70:0/405207034 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac2a32bc60 con 0x55ac2a5cabe0 2023-10-18T12:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.115+0000 7f5ead99b700 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.116+0000 7f5ead19a700 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ba5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e98097900 tx=0x7f5e9801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eed6950cd863bf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.116+0000 7f5e9cbff700 1 -- 172.21.15.70:0/405207034 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9809adb0 con 0x55ac2a5cabe0 2023-10-18T12:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.116+0000 7f5e9cbff700 1 -- 172.21.15.70:0/405207034 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e980b5e90 con 0x55ac2a5cabe0 2023-10-18T12:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.116+0000 7f5e9cbff700 1 -- 172.21.15.70:0/405207034 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980ce380 con 0x55ac2a5cabe0 2023-10-18T12:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/405207034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 msgr2=0x55ac2a5ba5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ba5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e98097900 tx=0x7f5e9801a040 comp rx=0 tx=0).stop 2023-10-18T12:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/405207034 shutdown_connections 2023-10-18T12:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/405207034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5ba5d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/405207034 >> 172.21.15.70:0/405207034 conn(0x55ac2a539720 msgr2=0x55ac2a5c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/405207034 shutdown_connections 2023-10-18T12:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/405207034 wait complete. 2023-10-18T12:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 -- start start 2023-10-18T12:39:02.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.117+0000 7f5ec34fdb80 1 Processor -- start 2023-10-18T12:39:02.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a5c6e00 con 0x55ac2a5c55d0 2023-10-18T12:39:02.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a5c6f40 con 0x55ac2a5cb500 2023-10-18T12:39:02.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a5c7100 con 0x55ac2a5cabe0 2023-10-18T12:39:02.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ec34fdb80 10 client.0 _ll_register_callbacks cb 0x55ac2a5c1c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:02.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead99b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5e41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37268/0 (socket says 172.21.15.70:37268) 2023-10-18T12:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 -- 172.21.15.70:0/3609524628 learned_addr learned my addr 172.21.15.70:0/3609524628 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5eae19c700 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 -- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 msgr2=0x55ac2a5c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 -- 172.21.15.70:0/3609524628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 msgr2=0x55ac2a5e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.118+0000 7f5ead19a700 1 -- 172.21.15.70:0/3609524628 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4009b90 con 0x55ac2a5cabe0 2023-10-18T12:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ead19a700 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5e41d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800f740 tx=0x7f5e98022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2651fe350906d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5e917fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5e917fa700 1 -- 172.21.15.70:0/3609524628 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800fd10 con 0x55ac2a5cabe0 2023-10-18T12:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5e917fa700 1 -- 172.21.15.70:0/3609524628 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f5e9809b450 con 0x55ac2a5cabe0 2023-10-18T12:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e9809bb50 con 0x55ac2a5cabe0 2023-10-18T12:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5e917fa700 1 -- 172.21.15.70:0/3609524628 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e980a3750 con 0x55ac2a5cabe0 2023-10-18T12:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 10 client.27487 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ac2a4d53f0 con 0x55ac2a5cabe0 2023-10-18T12:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5e917fa700 1 -- 172.21.15.70:0/3609524628 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9809bb50 con 0x55ac2a5cabe0 2023-10-18T12:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5e917fa700 1 -- 172.21.15.70:0/3609524628 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5e980b2070 con 0x55ac2a5cabe0 2023-10-18T12:39:02.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 10 client.27487 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:02.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 10 client.27487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:02.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 20 client.27487 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:02.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a63c290 0x55ac2a63e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:02.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 4 client.27487 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:39:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.119+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac2a510e50 con 0x55ac2a63c290 2023-10-18T12:39:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.120+0000 7f5ead99b700 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a63c290 0x55ac2a63e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.120+0000 7f5ead99b700 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a63c290 0x55ac2a63e680 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a2ec2e41f7797e server_cookie=d4249b790e274612 in_seq=0 out_seq=0 2023-10-18T12:39:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.120+0000 7f5e917fa700 10 client.27487 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:02.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5e917fa700 1 -- 172.21.15.70:0/3609524628 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac2a510e50 con 0x55ac2a63c290 2023-10-18T12:39:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5e917fa700 10 client.27487 handle_command_reply: tid=0 2023-10-18T12:39:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 client.27487 shutdown 2023-10-18T12:39:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 msgr2=0x55ac2a5e41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5e41d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800f740 tx=0x7f5e98022af0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 shutdown_connections 2023-10-18T12:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ac2a63c290 0x55ac2a63e680 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ac2a5cb500 0x55ac2a5c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ac2a5c55d0 0x55ac2a5e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5e917fa700 0 client.27487 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:02.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 --2- 172.21.15.70:0/3609524628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac2a5cabe0 0x55ac2a5e41d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.122+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 >> 172.21.15.70:0/3609524628 conn(0x55ac2a539720 msgr2=0x55ac2a5ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:02.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 shutdown_connections 2023-10-18T12:39:02.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5ec34fdb80 1 -- 172.21.15.70:0/3609524628 wait complete. 2023-10-18T12:39:02.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5ec34fdb80 20 client.27487 trim_cache size 0 max 16384 2023-10-18T12:39:02.142 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:39:02.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e940a15f0 msgr2=0x7f5e940a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e940a15f0 0x7f5e940a3ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000f870 tx=0x7f5ea000fc30 comp rx=0 tx=0).stop 2023-10-18T12:39:02.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 msgr2=0x7f5ea819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:02.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea819cde0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5e98000c00 tx=0x7f5e98005740 comp rx=0 tx=0).stop 2023-10-18T12:39:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 shutdown_connections 2023-10-18T12:39:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5e940a15f0 0x7f5e940a3ab0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ea8106730 0x7f5ea81969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ea8100720 0x7f5ea819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 --2- 172.21.15.70:0/943774045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea81010e0 0x7f5ea819cde0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:02.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.123+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 >> 172.21.15.70:0/943774045 conn(0x7f5ea80fc1b0 msgr2=0x7f5ea80fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:02.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.124+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 shutdown_connections 2023-10-18T12:39:02.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:02.124+0000 7f5eafbff700 1 -- 172.21.15.70:0/943774045 wait complete. 2023-10-18T12:39:02.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:01 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1651240989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:39:02.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:01 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1651240989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:39:02.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1459581 10.95 MB/sec execute 571 sec latency 68.775 ms 2023-10-18T12:39:03.147 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:39:03.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:39:03.171 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:02 smithi152 ceph-mon[134767]: pgmap v717: 129 pgs: 129 active+clean; 599 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 208 op/s 2023-10-18T12:39:03.210 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:02 smithi070 ceph-mon[98639]: pgmap v717: 129 pgs: 129 active+clean; 599 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 208 op/s 2023-10-18T12:39:03.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:02 smithi186 ceph-mon[135253]: pgmap v717: 129 pgs: 129 active+clean; 599 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 208 op/s 2023-10-18T12:39:03.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 -- 172.21.15.70:0/3081379032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4106730 msgr2=0x7f04b4106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/3081379032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4106730 0x7f04b4106b10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009990 tx=0x7f04a4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 -- 172.21.15.70:0/3081379032 shutdown_connections 2023-10-18T12:39:03.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/3081379032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/3081379032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4106730 0x7f04b4106b10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/3081379032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04b4100720 0x7f04b4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 -- 172.21.15.70:0/3081379032 >> 172.21.15.70:0/3081379032 conn(0x7f04b40fc1b0 msgr2=0x7f04b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 -- 172.21.15.70:0/3081379032 shutdown_connections 2023-10-18T12:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.356+0000 7f04ba3ff700 1 -- 172.21.15.70:0/3081379032 wait complete. 2023-10-18T12:39:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.357+0000 7f04ba3ff700 1 Processor -- start 2023-10-18T12:39:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 -- start start 2023-10-18T12:39:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4100720 0x7f04b419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04b4106730 0x7f04b4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b4199200 con 0x7f04b4100720 2023-10-18T12:39:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b4199370 con 0x7f04b41010e0 2023-10-18T12:39:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04ba3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b4199530 con 0x7f04b4106730 2023-10-18T12:39:03.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.358+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4100720 0x7f04b419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58300/0 (socket says 172.21.15.70:58300) 2023-10-18T12:39:03.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 -- 172.21.15.70:0/2602998729 learned_addr learned my addr 172.21.15.70:0/2602998729 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b899c700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04b4106730 0x7f04b4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 -- 172.21.15.70:0/2602998729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04b4106730 msgr2=0x7f04b4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04b4106730 0x7f04b4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 -- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4100720 msgr2=0x7f04b419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4100720 0x7f04b419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 -- 172.21.15.70:0/2602998729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a40095f0 con 0x7f04b41010e0 2023-10-18T12:39:03.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b37fe700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b419efb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d950 tx=0x7f04a800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04b17fa700 1 -- 172.21.15.70:0/2602998729 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a800f890 con 0x7f04b41010e0 2023-10-18T12:39:03.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.359+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b4199810 con 0x7f04b41010e0 2023-10-18T12:39:03.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.360+0000 7f04b17fa700 1 -- 172.21.15.70:0/2602998729 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a8010460 con 0x7f04b41010e0 2023-10-18T12:39:03.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.360+0000 7f04b17fa700 1 -- 172.21.15.70:0/2602998729 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a800e610 con 0x7f04b41010e0 2023-10-18T12:39:03.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.360+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b41a5a60 con 0x7f04b41010e0 2023-10-18T12:39:03.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.363+0000 7f04b17fa700 1 -- 172.21.15.70:0/2602998729 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f04a80108d0 con 0x7f04b41010e0 2023-10-18T12:39:03.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 Processor -- start 2023-10-18T12:39:03.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 -- start start 2023-10-18T12:39:03.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b88040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b82110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 0x55ca91b82ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91af5760 con 0x55ca91b87c60 2023-10-18T12:39:03.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91adbce0 con 0x55ca91b82650 2023-10-18T12:39:03.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.364+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91ad9600 con 0x55ca91b88580 2023-10-18T12:39:03.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 0x55ca91b82ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b899c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b88040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 0x55ca91b82ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58306/0 (socket says 172.21.15.70:58306) 2023-10-18T12:39:03.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 -- 172.21.15.70:0/2529643967 learned_addr learned my addr 172.21.15.70:0/2529643967 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b3fff700 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b82110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 -- 172.21.15.70:0/2529643967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 msgr2=0x55ca91b82110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b82110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 -- 172.21.15.70:0/2529643967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 msgr2=0x55ca91b88040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b88040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 -- 172.21.15.70:0/2529643967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca91a92470 con 0x55ca91b82650 2023-10-18T12:39:03.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b17fa700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f049c0a15f0 0x7f049c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b17fa700 1 -- 172.21.15.70:0/2602998729 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f04a80cd260 con 0x7f04b41010e0 2023-10-18T12:39:03.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b899c700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f049c0a15f0 0x7f049c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.365+0000 7f04b37fe700 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 0x55ca91b82ad0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f04a80ca530 tx=0x7f04a80d5540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6916301d3b0a736b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f049a7fc700 1 -- 172.21.15.70:0/2529643967 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80961c0 con 0x55ca91b82650 2023-10-18T12:39:03.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f049a7fc700 1 -- 172.21.15.70:0/2529643967 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a80d0de0 con 0x55ca91b82650 2023-10-18T12:39:03.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f049a7fc700 1 -- 172.21.15.70:0/2529643967 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a809dfd0 con 0x55ca91b82650 2023-10-18T12:39:03.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2529643967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 msgr2=0x55ca91b82ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04b899c700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f049c0a15f0 0x7f049c0a3ab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0127f0 tx=0x7f04ac00b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 0x55ca91b82ad0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f04a80ca530 tx=0x7f04a80d5540 comp rx=0 tx=0).stop 2023-10-18T12:39:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2529643967 shutdown_connections 2023-10-18T12:39:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b82650 0x55ca91b82ad0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b88040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2529643967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b82110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2529643967 >> 172.21.15.70:0/2529643967 conn(0x55ca91af67a0 msgr2=0x55ca91b7cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.366+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2529643967 shutdown_connections 2023-10-18T12:39:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.367+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2529643967 wait complete. 2023-10-18T12:39:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.367+0000 7f04cdc23b80 1 -- start start 2023-10-18T12:39:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.367+0000 7f04cdc23b80 1 Processor -- start 2023-10-18T12:39:03.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b8e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b8b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91ad4ef0 con 0x55ca91b82650 2023-10-18T12:39:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91a71790 con 0x55ca91b87c60 2023-10-18T12:39:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91a71430 con 0x55ca91b88580 2023-10-18T12:39:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04cdc23b80 10 client.0 _ll_register_callbacks cb 0x55ca919bbb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b8e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b8e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53396/0 (socket says 172.21.15.70:53396) 2023-10-18T12:39:03.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 -- 172.21.15.70:0/981273411 learned_addr learned my addr 172.21.15.70:0/981273411 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b899c700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b37fe700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b8b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 -- 172.21.15.70:0/981273411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 msgr2=0x55ca91b8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 -- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 msgr2=0x55ca91b8b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b8b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b3fff700 1 -- 172.21.15.70:0/981273411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b41a5a60 con 0x55ca91b82650 2023-10-18T12:39:03.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.368+0000 7f04b37fe700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b8b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:03.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04b3fff700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b8e560 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009990 tx=0x7f04a4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67376c49b2709bdd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04937fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:39:03.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04937fe700 1 -- 172.21.15.70:0/981273411 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a401d070 con 0x55ca91b82650 2023-10-18T12:39:03.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca91b51930 con 0x55ca91b82650 2023-10-18T12:39:03.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04937fe700 1 -- 172.21.15.70:0/981273411 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f04a4004500 con 0x55ca91b82650 2023-10-18T12:39:03.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04937fe700 1 -- 172.21.15.70:0/981273411 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a402d7c0 con 0x55ca91b82650 2023-10-18T12:39:03.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04cdc23b80 10 client.17454 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:03.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.369+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ca918ffe60 con 0x55ca91b82650 2023-10-18T12:39:03.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04937fe700 1 -- 172.21.15.70:0/981273411 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca91b51930 con 0x55ca91b82650 2023-10-18T12:39:03.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04937fe700 1 -- 172.21.15.70:0/981273411 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04a40046c0 con 0x55ca91b82650 2023-10-18T12:39:03.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04cdc23b80 10 client.17454 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:03.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04cdc23b80 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:03.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04cdc23b80 20 client.17454 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:03.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91b9bfb0 0x55ca91b9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04cdc23b80 4 client.17454 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:03.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca91acded0 con 0x55ca91b9bfb0 2023-10-18T12:39:03.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.370+0000 7f04b37fe700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91b9bfb0 0x55ca91b9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.371+0000 7f04b37fe700 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91b9bfb0 0x55ca91b9e3d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4394cf56af5ac1d server_cookie=91e1e0ba414cc698 in_seq=0 out_seq=0 2023-10-18T12:39:03.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.371+0000 7f04937fe700 10 client.17454 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04937fe700 1 -- 172.21.15.70:0/981273411 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ca91acded0 con 0x55ca91b9bfb0 2023-10-18T12:39:03.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04937fe700 10 client.17454 handle_command_reply: tid=0 2023-10-18T12:39:03.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 client.17454 shutdown 2023-10-18T12:39:03.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 msgr2=0x55ca91b8e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b8e560 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009990 tx=0x7f04a4005800 comp rx=0 tx=0).stop 2023-10-18T12:39:03.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 shutdown_connections 2023-10-18T12:39:03.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91b9bfb0 0x55ca91b9e3d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b8e560 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04937fe700 0 client.17454 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/981273411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b8b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 >> 172.21.15.70:0/981273411 conn(0x55ca91af67a0 msgr2=0x55ca91b81100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 shutdown_connections 2023-10-18T12:39:03.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.372+0000 7f04cdc23b80 1 -- 172.21.15.70:0/981273411 wait complete. 2023-10-18T12:39:03.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.373+0000 7f04cdc23b80 20 client.17454 trim_cache size 0 max 16384 2023-10-18T12:39:03.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 Processor -- start 2023-10-18T12:39:03.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 -- start start 2023-10-18T12:39:03.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 0x55ca91b92f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b8b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b88580 0x55ca91b8b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca919559a0 con 0x55ca91b87c60 2023-10-18T12:39:03.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91b8cb30 con 0x55ca91b88580 2023-10-18T12:39:03.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.391+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91b83080 con 0x55ca91b82650 2023-10-18T12:39:03.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b8b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 0x55ca91b92f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b899c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b88580 0x55ca91b8b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 0x55ca91b92f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37312/0 (socket says 172.21.15.70:37312) 2023-10-18T12:39:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 -- 172.21.15.70:0/2270330622 learned_addr learned my addr 172.21.15.70:0/2270330622 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 -- 172.21.15.70:0/2270330622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b88580 msgr2=0x55ca91b8b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b88580 0x55ca91b8b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 -- 172.21.15.70:0/2270330622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 msgr2=0x55ca91b8b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b8b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 -- 172.21.15.70:0/2270330622 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca91ba3020 con 0x55ca91b82650 2023-10-18T12:39:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f04b3fff700 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 0x55ca91b92f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f04a4004a50 tx=0x7f04a4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e6fa0f66bd50d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.392+0000 7f049affd700 1 -- 172.21.15.70:0/2270330622 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a401d070 con 0x55ca91b82650 2023-10-18T12:39:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f049affd700 1 -- 172.21.15.70:0/2270330622 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a401e040 con 0x55ca91b82650 2023-10-18T12:39:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f049affd700 1 -- 172.21.15.70:0/2270330622 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4024770 con 0x55ca91b82650 2023-10-18T12:39:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2270330622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 msgr2=0x55ca91b92f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 0x55ca91b92f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f04a4004a50 tx=0x7f04a4009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2270330622 shutdown_connections 2023-10-18T12:39:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b88580 0x55ca91b8b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b82650 0x55ca91b92f60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/2270330622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b87c60 0x55ca91b8b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2270330622 >> 172.21.15.70:0/2270330622 conn(0x55ca91af67a0 msgr2=0x55ca91b7c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2270330622 shutdown_connections 2023-10-18T12:39:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.393+0000 7f04cdc23b80 1 -- 172.21.15.70:0/2270330622 wait complete. 2023-10-18T12:39:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 -- start start 2023-10-18T12:39:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 Processor -- start 2023-10-18T12:39:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b91780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b91cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b92200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91befe50 con 0x55ca91b82650 2023-10-18T12:39:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91beff90 con 0x55ca91b87c60 2023-10-18T12:39:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca91bf0150 con 0x55ca91b88580 2023-10-18T12:39:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.394+0000 7f04cdc23b80 10 client.0 _ll_register_callbacks cb 0x55ca919bbb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b91780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b91cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b91780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53428/0 (socket says 172.21.15.70:53428) 2023-10-18T12:39:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 -- 172.21.15.70:0/1699691135 learned_addr learned my addr 172.21.15.70:0/1699691135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b899c700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b92200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 -- 172.21.15.70:0/1699691135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 msgr2=0x55ca91b92200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b92200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 -- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 msgr2=0x55ca91b91cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b91cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 -- 172.21.15.70:0/1699691135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca918ffe60 con 0x55ca91b82650 2023-10-18T12:39:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b899c700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b92200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:39:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04b3fff700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b91780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009590 tx=0x7f04a4005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2bdb1795cc0215 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04937fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:39:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.395+0000 7f04937fe700 1 -- 172.21.15.70:0/1699691135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a401d040 con 0x55ca91b82650 2023-10-18T12:39:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04937fe700 1 -- 172.21.15.70:0/1699691135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f04a4000e40 con 0x55ca91b82650 2023-10-18T12:39:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca91ad9f70 con 0x55ca91b82650 2023-10-18T12:39:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04937fe700 1 -- 172.21.15.70:0/1699691135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a40059f0 con 0x55ca91b82650 2023-10-18T12:39:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 10 client.17460 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ca918e8ce0 con 0x55ca91b82650 2023-10-18T12:39:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04937fe700 1 -- 172.21.15.70:0/1699691135 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca91ad9f70 con 0x55ca91b82650 2023-10-18T12:39:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04937fe700 1 -- 172.21.15.70:0/1699691135 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04a400f460 con 0x55ca91b82650 2023-10-18T12:39:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 10 client.17460 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 20 client.17460 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91bf1480 0x55ca91bf3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 4 client.17460 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.396+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ca91acdbf0 con 0x55ca91bf1480 2023-10-18T12:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.397+0000 7f04b37fe700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91bf1480 0x55ca91bf3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.397+0000 7f04937fe700 10 client.17460 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.397+0000 7f04b37fe700 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91bf1480 0x55ca91bf3870 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa48b9588fe1aff2 server_cookie=4183e7896f9d5266 in_seq=0 out_seq=0 2023-10-18T12:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04937fe700 1 -- 172.21.15.70:0/1699691135 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ca91acdbf0 con 0x55ca91bf1480 2023-10-18T12:39:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04937fe700 10 client.17460 handle_command_reply: tid=0 2023-10-18T12:39:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 client.17460 shutdown 2023-10-18T12:39:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 msgr2=0x55ca91b91780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b91780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009590 tx=0x7f04a4005cc0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 shutdown_connections 2023-10-18T12:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55ca91bf1480 0x55ca91bf3870 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca91b88580 0x55ca91b92200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca91b82650 0x55ca91b91780 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 --2- 172.21.15.70:0/1699691135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca91b87c60 0x55ca91b91cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04937fe700 0 client.17460 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 >> 172.21.15.70:0/1699691135 conn(0x55ca91af67a0 msgr2=0x55ca91b85730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 shutdown_connections 2023-10-18T12:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 1 -- 172.21.15.70:0/1699691135 wait complete. 2023-10-18T12:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.399+0000 7f04cdc23b80 20 client.17460 trim_cache size 0 max 16384 2023-10-18T12:39:03.414 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:39:03.415 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:39:03.415 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "b421dbfe-4db0-4a6c-b666-123984b5736a", 2023-10-18T12:39:03.415 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:39:03.415 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f049c0a15f0 msgr2=0x7f049c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f049c0a15f0 0x7f049c0a3ab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0127f0 tx=0x7f04ac00b040 comp rx=0 tx=0).stop 2023-10-18T12:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 msgr2=0x7f04b419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b419efb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d950 tx=0x7f04a800dc60 comp rx=0 tx=0).stop 2023-10-18T12:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 shutdown_connections 2023-10-18T12:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04b4106730 0x7f04b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b4100720 0x7f04b419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f049c0a15f0 0x7f049c0a3ab0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 --2- 172.21.15.70:0/2602998729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04b41010e0 0x7f04b419efb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 >> 172.21.15.70:0/2602998729 conn(0x7f04b40fc1b0 msgr2=0x7f04b410be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 shutdown_connections 2023-10-18T12:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.400+0000 7f04ba3ff700 1 -- 172.21.15.70:0/2602998729 wait complete. 2023-10-18T12:39:03.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:39:03.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 -- 172.21.15.70:0/2358960027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105170 msgr2=0x7fbe841055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 --2- 172.21.15.70:0/2358960027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105170 0x7fbe841055f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00dda0 tx=0x7fbe6c0084b0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 -- 172.21.15.70:0/2358960027 shutdown_connections 2023-10-18T12:39:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 --2- 172.21.15.70:0/2358960027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84105b30 0x7fbe8410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 --2- 172.21.15.70:0/2358960027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe84067d40 0x7fbe84068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 --2- 172.21.15.70:0/2358960027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105170 0x7fbe841055f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 -- 172.21.15.70:0/2358960027 >> 172.21.15.70:0/2358960027 conn(0x7fbe840767f0 msgr2=0x7fbe84076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 -- 172.21.15.70:0/2358960027 shutdown_connections 2023-10-18T12:39:03.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.701+0000 7fbe88bff700 1 -- 172.21.15.70:0/2358960027 wait complete. 2023-10-18T12:39:03.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.702+0000 7fbe88bff700 1 Processor -- start 2023-10-18T12:39:03.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 -- start start 2023-10-18T12:39:03.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84067d40 0x7fbe8419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe84105170 0x7fbe8419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105b30 0x7fbe841948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84194f30 con 0x7fbe84067d40 2023-10-18T12:39:03.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841950a0 con 0x7fbe84105b30 2023-10-18T12:39:03.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe88bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84195260 con 0x7fbe84105170 2023-10-18T12:39:03.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.703+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84067d40 0x7fbe8419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105b30 0x7fbe841948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84067d40 0x7fbe8419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53444/0 (socket says 172.21.15.70:53444) 2023-10-18T12:39:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe8259c700 1 -- 172.21.15.70:0/908071504 learned_addr learned my addr 172.21.15.70:0/908071504 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe84105170 0x7fbe8419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 -- 172.21.15.70:0/908071504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe84105170 msgr2=0x7fbe8419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe84105170 0x7fbe8419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 -- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84067d40 msgr2=0x7fbe8419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84067d40 0x7fbe8419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 -- 172.21.15.70:0/908071504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c00da00 con 0x7fbe84105b30 2023-10-18T12:39:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.704+0000 7fbe82d9d700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105b30 0x7fbe841948d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c016f30 tx=0x7fbe7c0143a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.705+0000 7fbe7b7fe700 1 -- 172.21.15.70:0/908071504 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c020070 con 0x7fbe84105b30 2023-10-18T12:39:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.705+0000 7fbe7b7fe700 1 -- 172.21.15.70:0/908071504 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe7c017040 con 0x7fbe84105b30 2023-10-18T12:39:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.705+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe841954e0 con 0x7fbe84105b30 2023-10-18T12:39:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.705+0000 7fbe7b7fe700 1 -- 172.21.15.70:0/908071504 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c004830 con 0x7fbe84105b30 2023-10-18T12:39:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.705+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe841a1790 con 0x7fbe84105b30 2023-10-18T12:39:03.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.708+0000 7fbe7b7fe700 1 -- 172.21.15.70:0/908071504 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbe7c00fbe0 con 0x7fbe84105b30 2023-10-18T12:39:03.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 Processor -- start 2023-10-18T12:39:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 -- start start 2023-10-18T12:39:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33051fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33052500 0x562a3304c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a3304c5d0 0x562a3304ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32fbf6e0 con 0x562a33052500 2023-10-18T12:39:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32fa5c60 con 0x562a33051be0 2023-10-18T12:39:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.709+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32fa3580 con 0x562a3304c5d0 2023-10-18T12:39:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33051fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33051fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:58358/0 (socket says 172.21.15.70:58358) 2023-10-18T12:39:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 -- 172.21.15.70:0/2724783535 learned_addr learned my addr 172.21.15.70:0/2724783535 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe8259c700 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33052500 0x562a3304c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a3304c5d0 0x562a3304ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 -- 172.21.15.70:0/2724783535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a3304c5d0 msgr2=0x562a3304ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a3304c5d0 0x562a3304ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 -- 172.21.15.70:0/2724783535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33052500 msgr2=0x562a3304c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7b7fe700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe700a1630 0x7fbe700a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33052500 0x562a3304c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.710+0000 7fbe7bfff700 1 -- 172.21.15.70:0/2724783535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a32f5c3f0 con 0x562a33051be0 2023-10-18T12:39:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe7b7fe700 1 -- 172.21.15.70:0/908071504 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fbe7c0d4f90 con 0x7fbe84105b30 2023-10-18T12:39:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe8259c700 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33052500 0x562a3304c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33051fc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c000c00 tx=0x7fbe6c00df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ca09c75d6b613c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe82d9d700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe700a1630 0x7fbe700a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe78ff9700 1 -- 172.21.15.70:0/2724783535 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003780 con 0x562a33051be0 2023-10-18T12:39:03.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe78ff9700 1 -- 172.21.15.70:0/2724783535 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe6c003960 con 0x562a33051be0 2023-10-18T12:39:03.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.711+0000 7fbe78ff9700 1 -- 172.21.15.70:0/2724783535 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c005530 con 0x562a33051be0 2023-10-18T12:39:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2724783535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 msgr2=0x562a33051fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33051fc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c000c00 tx=0x7fbe6c00df60 comp rx=0 tx=0).stop 2023-10-18T12:39:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe82d9d700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe700a1630 0x7fbe700a3af0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c09e0a0 tx=0x7fbe7c004750 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2724783535 shutdown_connections 2023-10-18T12:39:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a3304c5d0 0x562a3304ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33051fc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2724783535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33052500 0x562a3304c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2724783535 >> 172.21.15.70:0/2724783535 conn(0x562a32fc0720 msgr2=0x562a33046b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2724783535 shutdown_connections 2023-10-18T12:39:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.712+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2724783535 wait complete. 2023-10-18T12:39:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 -- start start 2023-10-18T12:39:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 Processor -- start 2023-10-18T12:39:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 0x562a330558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33055e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a33056360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32f9ee70 con 0x562a3304c5d0 2023-10-18T12:39:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32f3b710 con 0x562a33051be0 2023-10-18T12:39:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32f3b3b0 con 0x562a33052500 2023-10-18T12:39:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe9c4b6b80 10 client.0 _ll_register_callbacks cb 0x562a32e85a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.713+0000 7fbe7bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 0x562a330558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 0x562a330558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53458/0 (socket says 172.21.15.70:53458) 2023-10-18T12:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 -- 172.21.15.70:0/3358101910 learned_addr learned my addr 172.21.15.70:0/3358101910 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe8259c700 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33055e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 -- 172.21.15.70:0/3358101910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 msgr2=0x562a33056360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a33056360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 -- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 msgr2=0x562a33055e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33055e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 -- 172.21.15.70:0/3358101910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7400e280 con 0x562a3304c5d0 2023-10-18T12:39:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 0x562a330558e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00df30 tx=0x7fbe6c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325662fdc1a14e5b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:39:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3358101910 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c004270 con 0x562a3304c5d0 2023-10-18T12:39:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a32fa3ef0 con 0x562a3304c5d0 2023-10-18T12:39:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.714+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3358101910 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fbe6c02c070 con 0x562a3304c5d0 2023-10-18T12:39:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.715+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3358101910 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe6c005150 con 0x562a3304c5d0 2023-10-18T12:39:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.715+0000 7fbe9c4b6b80 10 client.17472 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.715+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fbe6c01e040 con 0x562a3304c5d0 2023-10-18T12:39:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.715+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3358101910 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a32fa3ef0 con 0x562a3304c5d0 2023-10-18T12:39:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3358101910 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbe6c00b070 con 0x562a3304c5d0 2023-10-18T12:39:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe9c4b6b80 10 client.17472 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe9c4b6b80 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe9c4b6b80 20 client.17472 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a33065aa0 0x562a33067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe9c4b6b80 4 client.17472 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a3301b8b0 con 0x562a33065aa0 2023-10-18T12:39:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.716+0000 7fbe8259c700 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a33065aa0 0x562a33067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.717+0000 7fbe69ffb700 10 client.17472 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.717+0000 7fbe8259c700 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a33065aa0 0x562a33067ec0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751b0098ab1df5f2 server_cookie=7e3e998586a97ad4 in_seq=0 out_seq=0 2023-10-18T12:39:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3358101910 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562a3301b8b0 con 0x562a33065aa0 2023-10-18T12:39:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe69ffb700 10 client.17472 handle_command_reply: tid=0 2023-10-18T12:39:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 client.17472 shutdown 2023-10-18T12:39:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 msgr2=0x562a330558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 0x562a330558e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00df30 tx=0x7fbe6c003c30 comp rx=0 tx=0).stop 2023-10-18T12:39:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 shutdown_connections 2023-10-18T12:39:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a33065aa0 0x562a33067ec0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a33056360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a3304c5d0 0x562a330558e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3358101910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a33055e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 >> 172.21.15.70:0/3358101910 conn(0x562a32fc0720 msgr2=0x562a3304b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe69ffb700 0 client.17472 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 shutdown_connections 2023-10-18T12:39:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.718+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3358101910 wait complete. 2023-10-18T12:39:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.719+0000 7fbe9c4b6b80 20 client.17472 trim_cache size 0 max 16384 2023-10-18T12:39:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.736+0000 7fbe9c4b6b80 1 Processor -- start 2023-10-18T12:39:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 -- start start 2023-10-18T12:39:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33051be0 0x562a3304c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a330561b0 0x562a330541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32e1f920 con 0x562a33051be0 2023-10-18T12:39:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a33056ee0 con 0x562a330561b0 2023-10-18T12:39:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a330538f0 con 0x562a33052500 2023-10-18T12:39:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37364/0 (socket says 172.21.15.70:37364) 2023-10-18T12:39:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 -- 172.21.15.70:0/2867377385 learned_addr learned my addr 172.21.15.70:0/2867377385 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33051be0 0x562a3304c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 -- 172.21.15.70:0/2867377385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a330561b0 msgr2=0x562a330541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a330561b0 0x562a330541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 -- 172.21.15.70:0/2867377385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33051be0 msgr2=0x562a3304c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33051be0 0x562a3304c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe8259c700 1 -- 172.21.15.70:0/2867377385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a32dc9de0 con 0x562a33052500 2023-10-18T12:39:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.737+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33051be0 0x562a3304c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:39:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.738+0000 7fbe8259c700 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305ce80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x562a33047a70 tx=0x7fbe7400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce2a4a4b3c51a83 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.738+0000 7fbe797fa700 1 -- 172.21.15.70:0/2867377385 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400e040 con 0x562a33052500 2023-10-18T12:39:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.738+0000 7fbe797fa700 1 -- 172.21.15.70:0/2867377385 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe7400f040 con 0x562a33052500 2023-10-18T12:39:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.738+0000 7fbe797fa700 1 -- 172.21.15.70:0/2867377385 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74016c60 con 0x562a33052500 2023-10-18T12:39:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2867377385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 msgr2=0x562a3305ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305ce80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x562a33047a70 tx=0x7fbe7400b040 comp rx=0 tx=0).stop 2023-10-18T12:39:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2867377385 shutdown_connections 2023-10-18T12:39:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a330561b0 0x562a330541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a33051be0 0x562a3304c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/2867377385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305ce80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2867377385 >> 172.21.15.70:0/2867377385 conn(0x562a32fc0720 msgr2=0x562a33046810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2867377385 shutdown_connections 2023-10-18T12:39:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/2867377385 wait complete. 2023-10-18T12:39:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 -- start start 2023-10-18T12:39:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.739+0000 7fbe9c4b6b80 1 Processor -- start 2023-10-18T12:39:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a3305bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a330561b0 0x562a330b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a330b3e20 con 0x562a330561b0 2023-10-18T12:39:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a330b3f90 con 0x562a33051be0 2023-10-18T12:39:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a330b4150 con 0x562a33052500 2023-10-18T12:39:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe9c4b6b80 10 client.0 _ll_register_callbacks cb 0x562a32e85a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:37376/0 (socket says 172.21.15.70:37376) 2023-10-18T12:39:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 -- 172.21.15.70:0/3298574164 learned_addr learned my addr 172.21.15.70:0/3298574164 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a3305bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a330561b0 0x562a330b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 -- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 msgr2=0x562a3305bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a3305bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 -- 172.21.15.70:0/3298574164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a330561b0 msgr2=0x562a330b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a330561b0 0x562a330b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe8259c700 1 -- 172.21.15.70:0/3298574164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7400e690 con 0x562a33052500 2023-10-18T12:39:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.740+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a330561b0 0x562a330b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:39:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe8259c700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305c4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe74000f80 tx=0x7fbe7401f550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c55a196a2a808f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:39:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3298574164 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74003a00 con 0x562a33052500 2023-10-18T12:39:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a3304d040 con 0x562a33052500 2023-10-18T12:39:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3298574164 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fbe7401c050 con 0x562a33052500 2023-10-18T12:39:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3298574164 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe74003b60 con 0x562a33052500 2023-10-18T12:39:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe9c4b6b80 10 client.27520 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.741+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562a32db2c60 con 0x562a33052500 2023-10-18T12:39:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3298574164 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a3304d040 con 0x562a33052500 2023-10-18T12:39:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3298574164 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbe74025020 con 0x562a33052500 2023-10-18T12:39:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe9c4b6b80 10 client.27520 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe9c4b6b80 10 client.27520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe9c4b6b80 20 client.27520 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a330bb9f0 0x562a330bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe9c4b6b80 4 client.27520 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:39:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fbe6c003960 con 0x562a330bb9f0 2023-10-18T12:39:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.742+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a330bb9f0 0x562a330bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.743+0000 7fbe69ffb700 10 client.27520 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.743+0000 7fbe7bfff700 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a330bb9f0 0x562a330bde10 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ca14f10b2416e3 server_cookie=f1d39f6131be7f3a in_seq=0 out_seq=0 2023-10-18T12:39:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe69ffb700 1 -- 172.21.15.70:0/3298574164 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fbe6c003960 con 0x562a330bb9f0 2023-10-18T12:39:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe69ffb700 10 client.27520 handle_command_reply: tid=0 2023-10-18T12:39:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 client.27520 shutdown 2023-10-18T12:39:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 msgr2=0x562a3305c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305c4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe74000f80 tx=0x7fbe7401f550 comp rx=0 tx=0).stop 2023-10-18T12:39:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 shutdown_connections 2023-10-18T12:39:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x562a330bb9f0 0x562a330bde10 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562a330561b0 0x562a330b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562a33051be0 0x562a3305bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 --2- 172.21.15.70:0/3298574164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562a33052500 0x562a3305c4c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 >> 172.21.15.70:0/3298574164 conn(0x562a32fc0720 msgr2=0x562a33054100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe69ffb700 0 client.27520 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 shutdown_connections 2023-10-18T12:39:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 1 -- 172.21.15.70:0/3298574164 wait complete. 2023-10-18T12:39:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.744+0000 7fbe9c4b6b80 20 client.27520 trim_cache size 0 max 16384 2023-10-18T12:39:03.761 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:39:03.762 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T12:39:03.762 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:39:03.762 INFO:teuthology.orchestra.run.smithi070.stdout: "b421dbfe-4db0-4a6c-b666-123984b5736a": { 2023-10-18T12:39:03.762 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:39:03.762 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "b421dbfe-4db0-4a6c-b666-123984b5736a", 2023-10-18T12:39:03.763 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:39:03.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:39:03.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:39:03.763 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:39:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe700a1630 msgr2=0x7fbe700a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe700a1630 0x7fbe700a3af0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c09e0a0 tx=0x7fbe7c004750 comp rx=0 tx=0).stop 2023-10-18T12:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105b30 msgr2=0x7fbe841948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105b30 0x7fbe841948d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c016f30 tx=0x7fbe7c0143a0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 shutdown_connections 2023-10-18T12:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbe700a1630 0x7fbe700a3af0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe84105b30 0x7fbe841948d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84067d40 0x7fbe8419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 --2- 172.21.15.70:0/908071504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe84105170 0x7fbe8419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 >> 172.21.15.70:0/908071504 conn(0x7fbe840767f0 msgr2=0x7fbe840fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 shutdown_connections 2023-10-18T12:39:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:03.745+0000 7fbe88bff700 1 -- 172.21.15.70:0/908071504 wait complete. 2023-10-18T12:39:03.767 INFO:tasks.cephfs.filesystem:scrub status for tag:b421dbfe-4db0-4a6c-b666-123984b5736a - {'path': '/', 'tag': 'b421dbfe-4db0-4a6c-b666-123984b5736a', 'options': 'recursive,force'} 2023-10-18T12:39:03.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1460513 10.94 MB/sec execute 572 sec latency 39.374 ms 2023-10-18T12:39:04.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1461900 10.93 MB/sec execute 573 sec latency 37.748 ms 2023-10-18T12:39:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:04 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:39:05.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:04 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:05.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:04 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:39:05.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:04 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:05.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:04 smithi152 ceph-mon[134767]: pgmap v718: 129 pgs: 129 active+clean; 601 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 213 op/s 2023-10-18T12:39:05.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:04 smithi152 ceph-mon[134767]: osdmap e106: 12 total, 12 up, 12 in 2023-10-18T12:39:05.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:04 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:39:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:04 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:04 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:39:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:04 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:04 smithi186 ceph-mon[135253]: pgmap v718: 129 pgs: 129 active+clean; 601 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 213 op/s 2023-10-18T12:39:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:04 smithi186 ceph-mon[135253]: osdmap e106: 12 total, 12 up, 12 in 2023-10-18T12:39:05.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:04 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:39:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:04 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:04 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:39:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:04 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:05.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:04 smithi070 ceph-mon[98639]: pgmap v718: 129 pgs: 129 active+clean; 601 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 213 op/s 2023-10-18T12:39:05.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:04 smithi070 ceph-mon[98639]: osdmap e106: 12 total, 12 up, 12 in 2023-10-18T12:39:05.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1464149 10.94 MB/sec execute 574 sec latency 14.375 ms 2023-10-18T12:39:06.889 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1466054 10.93 MB/sec execute 575 sec latency 39.355 ms 2023-10-18T12:39:07.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:06 smithi152 ceph-mon[134767]: pgmap v720: 129 pgs: 129 active+clean; 601 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 225 op/s 2023-10-18T12:39:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:06 smithi186 ceph-mon[135253]: pgmap v720: 129 pgs: 129 active+clean; 601 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 225 op/s 2023-10-18T12:39:07.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:06 smithi070 ceph-mon[98639]: pgmap v720: 129 pgs: 129 active+clean; 601 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 225 op/s 2023-10-18T12:39:07.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1467880 10.93 MB/sec execute 576 sec latency 14.291 ms 2023-10-18T12:39:08.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1469393 10.92 MB/sec execute 577 sec latency 72.131 ms 2023-10-18T12:39:09.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:08 smithi152 ceph-mon[134767]: pgmap v721: 129 pgs: 129 active+clean; 607 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2023-10-18T12:39:09.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:08 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:39:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:08 smithi186 ceph-mon[135253]: pgmap v721: 129 pgs: 129 active+clean; 607 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2023-10-18T12:39:09.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:08 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:39:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:08 smithi070 ceph-mon[98639]: pgmap v721: 129 pgs: 129 active+clean; 607 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2023-10-18T12:39:09.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:08 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:39:09.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1471426 10.93 MB/sec execute 578 sec latency 16.334 ms 2023-10-18T12:39:10.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:10 smithi152 ceph-mon[134767]: pgmap v722: 129 pgs: 129 active+clean; 612 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 222 op/s 2023-10-18T12:39:10.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:10 smithi186 ceph-mon[135253]: pgmap v722: 129 pgs: 129 active+clean; 612 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 222 op/s 2023-10-18T12:39:10.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:10 smithi070 ceph-mon[98639]: pgmap v722: 129 pgs: 129 active+clean; 612 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 222 op/s 2023-10-18T12:39:10.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1473439 10.92 MB/sec execute 579 sec latency 42.565 ms 2023-10-18T12:39:11.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1475174 10.92 MB/sec execute 580 sec latency 16.104 ms 2023-10-18T12:39:12.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1477401 10.93 MB/sec execute 581 sec latency 15.571 ms 2023-10-18T12:39:13.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:12 smithi152 ceph-mon[134767]: pgmap v723: 129 pgs: 129 active+clean; 617 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 209 op/s 2023-10-18T12:39:13.224 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:12 smithi186 ceph-mon[135253]: pgmap v723: 129 pgs: 129 active+clean; 617 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 209 op/s 2023-10-18T12:39:13.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:12 smithi070 ceph-mon[98639]: pgmap v723: 129 pgs: 129 active+clean; 617 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 209 op/s 2023-10-18T12:39:13.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1478393 10.92 MB/sec execute 582 sec latency 173.396 ms 2023-10-18T12:39:14.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1480360 10.91 MB/sec execute 583 sec latency 43.779 ms 2023-10-18T12:39:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:14 smithi152 ceph-mon[134767]: pgmap v724: 129 pgs: 129 active+clean; 626 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-18T12:39:15.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:14 smithi152 ceph-mon[134767]: osdmap e107: 12 total, 12 up, 12 in 2023-10-18T12:39:15.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:14 smithi186 ceph-mon[135253]: pgmap v724: 129 pgs: 129 active+clean; 626 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-18T12:39:15.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:14 smithi186 ceph-mon[135253]: osdmap e107: 12 total, 12 up, 12 in 2023-10-18T12:39:15.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:14 smithi070 ceph-mon[98639]: pgmap v724: 129 pgs: 129 active+clean; 626 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 226 op/s 2023-10-18T12:39:15.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:14 smithi070 ceph-mon[98639]: osdmap e107: 12 total, 12 up, 12 in 2023-10-18T12:39:15.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1482168 10.91 MB/sec execute 584 sec latency 14.564 ms 2023-10-18T12:39:16.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1484494 10.92 MB/sec execute 585 sec latency 38.653 ms 2023-10-18T12:39:17.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:16 smithi152 ceph-mon[134767]: pgmap v726: 129 pgs: 129 active+clean; 630 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2023-10-18T12:39:17.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:16 smithi186 ceph-mon[135253]: pgmap v726: 129 pgs: 129 active+clean; 630 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2023-10-18T12:39:17.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:16 smithi070 ceph-mon[98639]: pgmap v726: 129 pgs: 129 active+clean; 630 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2023-10-18T12:39:17.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1486011 10.91 MB/sec execute 586 sec latency 22.523 ms 2023-10-18T12:39:18.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1487517 10.90 MB/sec execute 587 sec latency 123.726 ms 2023-10-18T12:39:19.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:18 smithi152 ceph-mon[134767]: pgmap v727: 129 pgs: 129 active+clean; 634 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 197 op/s 2023-10-18T12:39:19.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:18 smithi186 ceph-mon[135253]: pgmap v727: 129 pgs: 129 active+clean; 634 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 197 op/s 2023-10-18T12:39:19.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:18 smithi070 ceph-mon[98639]: pgmap v727: 129 pgs: 129 active+clean; 634 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 197 op/s 2023-10-18T12:39:19.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1489173 10.90 MB/sec execute 588 sec latency 58.510 ms 2023-10-18T12:39:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:20 smithi152 ceph-mon[134767]: pgmap v728: 129 pgs: 129 active+clean; 641 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 202 op/s 2023-10-18T12:39:20.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:20 smithi186 ceph-mon[135253]: pgmap v728: 129 pgs: 129 active+clean; 641 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 202 op/s 2023-10-18T12:39:20.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:20 smithi070 ceph-mon[98639]: pgmap v728: 129 pgs: 129 active+clean; 641 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 202 op/s 2023-10-18T12:39:20.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1491448 10.90 MB/sec execute 589 sec latency 23.183 ms 2023-10-18T12:39:21.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1492701 10.90 MB/sec execute 590 sec latency 22.518 ms 2023-10-18T12:39:22.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1495019 10.90 MB/sec execute 591 sec latency 12.614 ms 2023-10-18T12:39:23.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:22 smithi152 ceph-mon[134767]: pgmap v729: 129 pgs: 129 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 191 op/s 2023-10-18T12:39:23.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:22 smithi186 ceph-mon[135253]: pgmap v729: 129 pgs: 129 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 191 op/s 2023-10-18T12:39:23.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:22 smithi070 ceph-mon[98639]: pgmap v729: 129 pgs: 129 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 191 op/s 2023-10-18T12:39:23.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1496131 10.89 MB/sec execute 592 sec latency 87.454 ms 2023-10-18T12:39:24.890 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1497806 10.89 MB/sec execute 593 sec latency 61.281 ms 2023-10-18T12:39:25.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:24 smithi152 ceph-mon[134767]: pgmap v730: 129 pgs: 129 active+clean; 643 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 155 op/s 2023-10-18T12:39:25.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:24 smithi186 ceph-mon[135253]: pgmap v730: 129 pgs: 129 active+clean; 643 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 155 op/s 2023-10-18T12:39:25.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:24 smithi070 ceph-mon[98639]: pgmap v730: 129 pgs: 129 active+clean; 643 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 155 op/s 2023-10-18T12:39:25.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1499584 10.89 MB/sec execute 594 sec latency 64.863 ms 2023-10-18T12:39:26.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1501519 10.88 MB/sec execute 595 sec latency 29.044 ms 2023-10-18T12:39:27.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:26 smithi152 ceph-mon[134767]: pgmap v731: 129 pgs: 129 active+clean; 659 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 194 op/s 2023-10-18T12:39:27.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:26 smithi186 ceph-mon[135253]: pgmap v731: 129 pgs: 129 active+clean; 659 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 194 op/s 2023-10-18T12:39:27.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:26 smithi070 ceph-mon[98639]: pgmap v731: 129 pgs: 129 active+clean; 659 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 194 op/s 2023-10-18T12:39:27.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1503361 10.89 MB/sec execute 596 sec latency 15.417 ms 2023-10-18T12:39:28.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1505200 10.88 MB/sec execute 597 sec latency 77.846 ms 2023-10-18T12:39:29.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:28 smithi152 ceph-mon[134767]: pgmap v732: 129 pgs: 129 active+clean; 663 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 200 op/s 2023-10-18T12:39:29.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:28 smithi186 ceph-mon[135253]: pgmap v732: 129 pgs: 129 active+clean; 663 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 200 op/s 2023-10-18T12:39:29.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:28 smithi070 ceph-mon[98639]: pgmap v732: 129 pgs: 129 active+clean; 663 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 200 op/s 2023-10-18T12:39:29.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1506866 10.88 MB/sec execute 598 sec latency 44.627 ms 2023-10-18T12:39:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:30 smithi152 ceph-mon[134767]: pgmap v733: 129 pgs: 129 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 216 op/s 2023-10-18T12:39:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:30 smithi186 ceph-mon[135253]: pgmap v733: 129 pgs: 129 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 216 op/s 2023-10-18T12:39:30.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:30 smithi070 ceph-mon[98639]: pgmap v733: 129 pgs: 129 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 216 op/s 2023-10-18T12:39:30.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 1508916 10.88 MB/sec execute 599 sec latency 28.187 ms 2023-10-18T12:39:31.891 INFO:tasks.workunit.client.0.smithi070.stdout: 1 cleanup 600 sec 2023-10-18T12:39:32.067 INFO:tasks.workunit.client.0.smithi070.stdout: 0 cleanup 600 sec 2023-10-18T12:39:32.067 INFO:tasks.workunit.client.0.smithi070.stdout: 2023-10-18T12:39:32.068 INFO:tasks.workunit.client.0.smithi070.stdout: Operation Count AvgLat MaxLat 2023-10-18T12:39:32.068 INFO:tasks.workunit.client.0.smithi070.stdout: ---------------------------------------- 2023-10-18T12:39:32.068 INFO:tasks.workunit.client.0.smithi070.stdout: NTCreateX 207633 0.999 8464.717 2023-10-18T12:39:32.068 INFO:tasks.workunit.client.0.smithi070.stdout: Close 152530 0.003 0.535 2023-10-18T12:39:32.068 INFO:tasks.workunit.client.0.smithi070.stdout: Rename 8791 4.380 209.966 2023-10-18T12:39:32.069 INFO:tasks.workunit.client.0.smithi070.stdout: Unlink 41924 1.760 93.457 2023-10-18T12:39:32.069 INFO:tasks.workunit.client.0.smithi070.stdout: Deltree 6 81.794 108.076 2023-10-18T12:39:32.069 INFO:tasks.workunit.client.0.smithi070.stdout: Mkdir 3 0.015 0.016 2023-10-18T12:39:32.069 INFO:tasks.workunit.client.0.smithi070.stdout: Qpathinfo 188176 0.843 190.734 2023-10-18T12:39:32.069 INFO:tasks.workunit.client.0.smithi070.stdout: Qfileinfo 33001 0.002 0.064 2023-10-18T12:39:32.070 INFO:tasks.workunit.client.0.smithi070.stdout: Qfsinfo 34506 0.200 8.630 2023-10-18T12:39:32.070 INFO:tasks.workunit.client.0.smithi070.stdout: Sfileinfo 16912 0.028 6.789 2023-10-18T12:39:32.070 INFO:tasks.workunit.client.0.smithi070.stdout: Find 72760 0.288 5517.108 2023-10-18T12:39:32.070 INFO:tasks.workunit.client.0.smithi070.stdout: WriteX 103654 0.120 214.818 2023-10-18T12:39:32.070 INFO:tasks.workunit.client.0.smithi070.stdout: ReadX 325437 0.011 32.676 2023-10-18T12:39:32.071 INFO:tasks.workunit.client.0.smithi070.stdout: LockX 676 0.652 8.680 2023-10-18T12:39:32.071 INFO:tasks.workunit.client.0.smithi070.stdout: UnlockX 676 0.554 4.896 2023-10-18T12:39:32.071 INFO:tasks.workunit.client.0.smithi070.stdout: Flush 14551 4.894 336.131 2023-10-18T12:39:32.071 INFO:tasks.workunit.client.0.smithi070.stdout: 2023-10-18T12:39:32.071 INFO:tasks.workunit.client.0.smithi070.stdout:Throughput 10.8769 MB/sec 1 clients 1 procs max_latency=8464.723 ms 2023-10-18T12:39:32.072 INFO:tasks.workunit.client.0.smithi070.stderr:+ dbench 10 2023-10-18T12:39:32.072 INFO:tasks.workunit.client.0.smithi070.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-10-18T12:39:32.073 INFO:tasks.workunit.client.0.smithi070.stdout: 2023-10-18T12:39:32.073 INFO:tasks.workunit.client.0.smithi070.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-10-18T12:39:32.073 INFO:tasks.workunit.client.0.smithi070.stdout:9 of 10 processes prepared for launch 0 sec 2023-10-18T12:39:32.172 INFO:tasks.workunit.client.0.smithi070.stdout:10 of 10 processes prepared for launch 0 sec 2023-10-18T12:39:32.172 INFO:tasks.workunit.client.0.smithi070.stdout:releasing clients 2023-10-18T12:39:33.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:32 smithi152 ceph-mon[134767]: pgmap v734: 129 pgs: 129 active+clean; 627 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-10-18T12:39:33.172 INFO:tasks.workunit.client.0.smithi070.stdout: 10 377 134.58 MB/sec warmup 1 sec latency 163.382 ms 2023-10-18T12:39:33.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:32 smithi186 ceph-mon[135253]: pgmap v734: 129 pgs: 129 active+clean; 627 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-10-18T12:39:33.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:32 smithi070 ceph-mon[98639]: pgmap v734: 129 pgs: 129 active+clean; 627 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-10-18T12:39:33.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:39:34.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2745484896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc0ff640 msgr2=0x7f9cdc0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2745484896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc0ff640 0x7f9cdc0ffac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009990 tx=0x7f9ccc009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2745484896 shutdown_connections 2023-10-18T12:39:34.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2745484896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc100000 0x7f9cdc10c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2745484896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 0x7f9cdc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2745484896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc0ff640 0x7f9cdc0ffac0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2745484896 >> 172.21.15.70:0/2745484896 conn(0x7f9cdc0767f0 msgr2=0x7f9cdc076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2745484896 shutdown_connections 2023-10-18T12:39:34.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.103+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2745484896 wait complete. 2023-10-18T12:39:34.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.105+0000 7f9ce39ff700 1 Processor -- start 2023-10-18T12:39:34.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.105+0000 7f9ce39ff700 1 -- start start 2023-10-18T12:39:34.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce39ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc0ff640 0x7f9cdc198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce39ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 0x7f9cdc199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce39ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 0x7f9cdc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce39ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc19de00 con 0x7f9cdc0ff640 2023-10-18T12:39:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce39ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc19df70 con 0x7f9cdc100000 2023-10-18T12:39:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce39ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc19e130 con 0x7f9cdc1056e0 2023-10-18T12:39:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce0f9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 0x7f9cdc199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce0f9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 0x7f9cdc199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32768/0 (socket says 172.21.15.70:32768) 2023-10-18T12:39:34.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2864726309 learned_addr learned my addr 172.21.15.70:0/2864726309 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:34.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.106+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 0x7f9cdc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2864726309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 msgr2=0x7f9cdc19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 0x7f9cdc19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc0ff640 msgr2=0x7f9cdc198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc0ff640 0x7f9cdc198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2864726309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0095f0 con 0x7f9cdc100000 2023-10-18T12:39:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 0x7f9cdc19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.107+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 0x7f9cdc199420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.108+0000 7f9cd27fc700 1 -- 172.21.15.70:0/2864726309 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc01d070 con 0x7f9cdc100000 2023-10-18T12:39:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.108+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc19e3b0 con 0x7f9cdc100000 2023-10-18T12:39:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.108+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc1a5c50 con 0x7f9cdc100000 2023-10-18T12:39:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.108+0000 7f9cd27fc700 1 -- 172.21.15.70:0/2864726309 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc005030 con 0x7f9cdc100000 2023-10-18T12:39:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.108+0000 7f9cd27fc700 1 -- 172.21.15.70:0/2864726309 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc004b30 con 0x7f9cdc100000 2023-10-18T12:39:34.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.113+0000 7f9cd27fc700 1 -- 172.21.15.70:0/2864726309 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9ccc004d50 con 0x7f9cdc100000 2023-10-18T12:39:34.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.114+0000 7f9cf7366b80 1 Processor -- start 2023-10-18T12:39:34.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.114+0000 7f9cf7366b80 1 -- start start 2023-10-18T12:39:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1639090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1639a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e15ac6e0 con 0x55c8e163ebe0 2023-10-18T12:39:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e1592c60 con 0x55c8e16395d0 2023-10-18T12:39:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e1590580 con 0x55c8e163f500 2023-10-18T12:39:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9ce1f9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9ce1f9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36854/0 (socket says 172.21.15.70:36854) 2023-10-18T12:39:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9ce1f9c700 1 -- 172.21.15.70:0/3403646561 learned_addr learned my addr 172.21.15.70:0/3403646561 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.115+0000 7f9ce179b700 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1639090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9ce1f9c700 1 -- 172.21.15.70:0/3403646561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 msgr2=0x55c8e1639090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1639090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9ce1f9c700 1 -- 172.21.15.70:0/3403646561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 msgr2=0x55c8e1639a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T12:39:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1639a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9ce1f9c700 1 -- 172.21.15.70:0/3403646561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8e15493f0 con 0x55c8e163ebe0 2023-10-18T12:39:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9cd27fc700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9cc80a16b0 0x7f9cc80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163efc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40078a0 tx=0x7f9cd40090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc592690b09c52b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.116+0000 7f9cd27fc700 1 -- 172.21.15.70:0/2864726309 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9ccc0cdc10 con 0x7f9cdc100000 2023-10-18T12:39:34.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.117+0000 7f9cc77fe700 1 -- 172.21.15.70:0/3403646561 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4015850 con 0x55c8e163ebe0 2023-10-18T12:39:34.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.117+0000 7f9cc77fe700 1 -- 172.21.15.70:0/3403646561 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cd400f040 con 0x55c8e163ebe0 2023-10-18T12:39:34.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.117+0000 7f9cc77fe700 1 -- 172.21.15.70:0/3403646561 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4014dd0 con 0x55c8e163ebe0 2023-10-18T12:39:34.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.117+0000 7f9ce179b700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9cc80a16b0 0x7f9cc80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 -- 172.21.15.70:0/3403646561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 msgr2=0x55c8e163efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9ce179b700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9cc80a16b0 0x7f9cc80a3b70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55c8e1634a70 tx=0x7f9cd800b610 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163efc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40078a0 tx=0x7f9cd40090e0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 -- 172.21.15.70:0/3403646561 shutdown_connections 2023-10-18T12:39:34.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1639a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163efc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/3403646561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1639090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 -- 172.21.15.70:0/3403646561 >> 172.21.15.70:0/3403646561 conn(0x55c8e15ad720 msgr2=0x55c8e1633b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 -- 172.21.15.70:0/3403646561 shutdown_connections 2023-10-18T12:39:34.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.118+0000 7f9cf7366b80 1 -- 172.21.15.70:0/3403646561 wait complete. 2023-10-18T12:39:34.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.119+0000 7f9cf7366b80 1 -- start start 2023-10-18T12:39:34.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.119+0000 7f9cf7366b80 1 Processor -- start 2023-10-18T12:39:34.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1645550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 0x55c8e1642890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1642fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e158be70 con 0x55c8e163f500 2023-10-18T12:39:34.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e1528710 con 0x55c8e16395d0 2023-10-18T12:39:34.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e15283b0 con 0x55c8e163ebe0 2023-10-18T12:39:34.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9cf7366b80 10 client.0 _ll_register_callbacks cb 0x55c8e1472a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:34.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9ce0f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 0x55c8e1642890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9ce0f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 0x55c8e1642890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:34956/0 (socket says 172.21.15.70:34956) 2023-10-18T12:39:34.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2304646299 learned_addr learned my addr 172.21.15.70:0/2304646299 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:34.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9ce179b700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1645550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.120+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1642fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 msgr2=0x55c8e1645550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1645550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2304646299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 msgr2=0x55c8e1642fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1642fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2304646299 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd4007500 con 0x55c8e163ebe0 2023-10-18T12:39:34.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1642fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:39:34.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 0x55c8e1642890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0a26f0 tx=0x7f9ccc09a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c929231e338482 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9cc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:39:34.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/2304646299 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc09ad80 con 0x55c8e163ebe0 2023-10-18T12:39:34.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8e1590ef0 con 0x55c8e163ebe0 2023-10-18T12:39:34.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/2304646299 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9ccc09ea70 con 0x55c8e163ebe0 2023-10-18T12:39:34.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/2304646299 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ccc0a2c20 con 0x55c8e163ebe0 2023-10-18T12:39:34.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.121+0000 7f9cf7366b80 10 client.27532 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:34.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c8e13b6de0 con 0x55c8e163ebe0 2023-10-18T12:39:34.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/2304646299 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8e1590ef0 con 0x55c8e163ebe0 2023-10-18T12:39:34.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/2304646299 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ccc0a2790 con 0x55c8e163ebe0 2023-10-18T12:39:34.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 10 client.27532 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:34.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:34.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 20 client.27532 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:34.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e1652b40 0x55c8e1654f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 4 client.27532 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:34.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.122+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8e16088b0 con 0x55c8e1652b40 2023-10-18T12:39:34.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.123+0000 7f9ce179b700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e1652b40 0x55c8e1654f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.123+0000 7f9ce179b700 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e1652b40 0x55c8e1654f30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb41605b3563e7d server_cookie=847a3688d53df063 in_seq=0 out_seq=0 2023-10-18T12:39:34.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.123+0000 7f9cc4ff9700 10 client.27532 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:34.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.126+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/2304646299 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8e16088b0 con 0x55c8e1652b40 2023-10-18T12:39:34.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.126+0000 7f9cc4ff9700 10 client.27532 handle_command_reply: tid=0 2023-10-18T12:39:34.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 client.27532 shutdown 2023-10-18T12:39:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 msgr2=0x55c8e1642890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 0x55c8e1642890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0a26f0 tx=0x7f9ccc09a2f0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 shutdown_connections 2023-10-18T12:39:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e1652b40 0x55c8e1654f30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1642fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1645550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cc4ff9700 0 client.27532 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:34.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2304646299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163ebe0 0x55c8e1642890 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 >> 172.21.15.70:0/2304646299 conn(0x55c8e15ad720 msgr2=0x55c8e1638080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 shutdown_connections 2023-10-18T12:39:34.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2304646299 wait complete. 2023-10-18T12:39:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.127+0000 7f9cf7366b80 20 client.27532 trim_cache size 0 max 16384 2023-10-18T12:39:34.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 Processor -- start 2023-10-18T12:39:34.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 -- start start 2023-10-18T12:39:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1649f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1642170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e140c920 con 0x55c8e163ebe0 2023-10-18T12:39:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e16088b0 con 0x55c8e16395d0 2023-10-18T12:39:34.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.147+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e16301a0 con 0x55c8e163f500 2023-10-18T12:39:34.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce179b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1649f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce179b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1649f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32802/0 (socket says 172.21.15.70:32802) 2023-10-18T12:39:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce179b700 1 -- 172.21.15.70:0/2173787381 learned_addr learned my addr 172.21.15.70:0/2173787381 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1642170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2173787381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 msgr2=0x55c8e1642170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1642170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2173787381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 msgr2=0x55c8e1649f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1649f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/2173787381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8e139fc60 con 0x55c8e163ebe0 2023-10-18T12:39:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce179b700 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1649f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.148+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163cdc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc097390 tx=0x7f9ccc09aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2785567caa8c48 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cc7fff700 1 -- 172.21.15.70:0/2173787381 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc09b020 con 0x55c8e163ebe0 2023-10-18T12:39:34.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cc7fff700 1 -- 172.21.15.70:0/2173787381 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc0b3140 con 0x55c8e163ebe0 2023-10-18T12:39:34.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cc7fff700 1 -- 172.21.15.70:0/2173787381 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc09ab80 con 0x55c8e163ebe0 2023-10-18T12:39:34.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2173787381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 msgr2=0x55c8e163cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163cdc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc097390 tx=0x7f9ccc09aaa0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2173787381 shutdown_connections 2023-10-18T12:39:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e163f500 0x55c8e1642170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e16395d0 0x55c8e1649f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/2173787381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163ebe0 0x55c8e163cdc0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2173787381 >> 172.21.15.70:0/2173787381 conn(0x55c8e15ad720 msgr2=0x55c8e163c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.149+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2173787381 shutdown_connections 2023-10-18T12:39:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.150+0000 7f9cf7366b80 1 -- 172.21.15.70:0/2173787381 wait complete. 2023-10-18T12:39:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.150+0000 7f9cf7366b80 1 -- start start 2023-10-18T12:39:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.150+0000 7f9cf7366b80 1 Processor -- start 2023-10-18T12:39:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 0x55c8e1649c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e163ebe0 0x55c8e16480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1648620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e164a520 con 0x55c8e163f500 2023-10-18T12:39:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e1648b60 con 0x55c8e163ebe0 2023-10-18T12:39:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e1648d20 con 0x55c8e16395d0 2023-10-18T12:39:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cf7366b80 10 client.0 _ll_register_callbacks cb 0x55c8e1472a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce179b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 0x55c8e1649c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce179b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 0x55c8e1649c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:34978/0 (socket says 172.21.15.70:34978) 2023-10-18T12:39:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce179b700 1 -- 172.21.15.70:0/4016590114 learned_addr learned my addr 172.21.15.70:0/4016590114 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e163ebe0 0x55c8e16480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce1f9c700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1648620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/4016590114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 msgr2=0x55c8e1649c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 0x55c8e1649c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/4016590114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 msgr2=0x55c8e1648620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1648620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 -- 172.21.15.70:0/4016590114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd800fcc0 con 0x55c8e163ebe0 2023-10-18T12:39:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce179b700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 0x55c8e1649c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9ce0f9a700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e163ebe0 0x55c8e16480e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55c8e163a160 tx=0x7f9ccc09a0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8b998029e72e8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.151+0000 7f9cc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/4016590114 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc09ad80 con 0x55c8e163ebe0 2023-10-18T12:39:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/4016590114 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f9ccc09ea70 con 0x55c8e163ebe0 2023-10-18T12:39:34.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cd400f040 con 0x55c8e163ebe0 2023-10-18T12:39:34.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/4016590114 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ccc0a2e70 con 0x55c8e163ebe0 2023-10-18T12:39:34.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 10 client.27551 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:34.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c8e13b6de0 con 0x55c8e163ebe0 2023-10-18T12:39:34.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/4016590114 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc0a6070 con 0x55c8e163ebe0 2023-10-18T12:39:34.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/4016590114 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ccc099880 con 0x55c8e163ebe0 2023-10-18T12:39:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 10 client.27551 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 10 client.27551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 20 client.27551 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e16a7840 0x55c8e16a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 4 client.27551 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:39:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.152+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c8e1584e50 con 0x55c8e16a7840 2023-10-18T12:39:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.153+0000 7f9ce179b700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e16a7840 0x55c8e16a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.153+0000 7f9cc4ff9700 10 client.27551 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.153+0000 7f9ce179b700 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e16a7840 0x55c8e16a9c60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=badb1cf6a32b9142 server_cookie=4a6c0f324f044d63 in_seq=0 out_seq=0 2023-10-18T12:39:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.155+0000 7f9cc4ff9700 1 -- 172.21.15.70:0/4016590114 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c8e1584e50 con 0x55c8e16a7840 2023-10-18T12:39:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.155+0000 7f9cc4ff9700 10 client.27551 handle_command_reply: tid=0 2023-10-18T12:39:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.155+0000 7f9cf7366b80 1 client.27551 shutdown 2023-10-18T12:39:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e163ebe0 msgr2=0x55c8e16480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e163ebe0 0x55c8e16480e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55c8e163a160 tx=0x7f9ccc09a0c0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 shutdown_connections 2023-10-18T12:39:34.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55c8e16a7840 0x55c8e16a9c60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c8e163f500 0x55c8e1648620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8e16395d0 0x55c8e1649c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 --2- 172.21.15.70:0/4016590114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c8e163ebe0 0x55c8e16480e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cc4ff9700 0 client.27551 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 >> 172.21.15.70:0/4016590114 conn(0x55c8e15ad720 msgr2=0x55c8e162d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 shutdown_connections 2023-10-18T12:39:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 1 -- 172.21.15.70:0/4016590114 wait complete. 2023-10-18T12:39:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9cf7366b80 20 client.27551 trim_cache size 0 max 16384 2023-10-18T12:39:34.172 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:39:34.172 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:39:34.172 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:39:34.172 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:39:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9cc80a16b0 msgr2=0x7f9cc80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9cc80a16b0 0x7f9cc80a3b70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55c8e1634a70 tx=0x7f9cd800b610 comp rx=0 tx=0).stop 2023-10-18T12:39:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 msgr2=0x7f9cdc199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.156+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 0x7f9cdc199420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc00f690 comp rx=0 tx=0).stop 2023-10-18T12:39:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 shutdown_connections 2023-10-18T12:39:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cdc1056e0 0x7f9cdc19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc0ff640 0x7f9cdc198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f9cc80a16b0 0x7f9cc80a3b70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 --2- 172.21.15.70:0/2864726309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9cdc100000 0x7f9cdc199420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 >> 172.21.15.70:0/2864726309 conn(0x7f9cdc0767f0 msgr2=0x7f9cdc0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 shutdown_connections 2023-10-18T12:39:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.157+0000 7f9ce39ff700 1 -- 172.21.15.70:0/2864726309 wait complete. 2023-10-18T12:39:34.176 INFO:tasks.workunit.client.0.smithi070.stdout: 10 729 101.74 MB/sec warmup 2 sec latency 197.571 ms 2023-10-18T12:39:34.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:39:34.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:39:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.452+0000 7f84cd1ff700 1 -- 172.21.15.70:0/2007150569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c8108900 msgr2=0x7f84c8108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.452+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/2007150569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c8108900 0x7f84c8108ce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009990 tx=0x7f84b8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 -- 172.21.15.70:0/2007150569 shutdown_connections 2023-10-18T12:39:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/2007150569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c81032b0 0x7f84c810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/2007150569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c8108900 0x7f84c8108ce0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/2007150569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84c81028f0 0x7f84c8102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 -- 172.21.15.70:0/2007150569 >> 172.21.15.70:0/2007150569 conn(0x7f84c80fe3a0 msgr2=0x7f84c81007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 -- 172.21.15.70:0/2007150569 shutdown_connections 2023-10-18T12:39:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.453+0000 7f84cd1ff700 1 -- 172.21.15.70:0/2007150569 wait complete. 2023-10-18T12:39:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.454+0000 7f84cd1ff700 1 Processor -- start 2023-10-18T12:39:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.454+0000 7f84cd1ff700 1 -- start start 2023-10-18T12:39:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84cd1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c81028f0 0x7f84c8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84cd1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84c81032b0 0x7f84c8075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84cd1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 0x7f84c8075780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84cd1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81a9310 con 0x7f84c81032b0 2023-10-18T12:39:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84cd1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81a9450 con 0x7f84c81028f0 2023-10-18T12:39:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84cd1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81a95c0 con 0x7f84c8108900 2023-10-18T12:39:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84c759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 0x7f84c8075780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84c6d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c81028f0 0x7f84c8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84c759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 0x7f84c8075780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:34994/0 (socket says 172.21.15.70:34994) 2023-10-18T12:39:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84c759e700 1 -- 172.21.15.70:0/1872559855 learned_addr learned my addr 172.21.15.70:0/1872559855 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.455+0000 7f84c659c700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84c81032b0 0x7f84c8075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84c759e700 1 -- 172.21.15.70:0/1872559855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c81028f0 msgr2=0x7f84c8074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84c759e700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c81028f0 0x7f84c8074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84c759e700 1 -- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84c81032b0 msgr2=0x7f84c8075240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84c759e700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84c81032b0 0x7f84c8075240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84c759e700 1 -- 172.21.15.70:0/1872559855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc009520 con 0x7f84c8108900 2023-10-18T12:39:34.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84c759e700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 0x7f84c8075780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84c000ef30 tx=0x7f84c000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0018070 con 0x7f84c8108900 2023-10-18T12:39:34.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.456+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b80095f0 con 0x7f84c8108900 2023-10-18T12:39:34.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.457+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c000f040 con 0x7f84c8108900 2023-10-18T12:39:34.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.457+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000cdb0 con 0x7f84c8108900 2023-10-18T12:39:34.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.457+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c81a9b00 con 0x7f84c8108900 2023-10-18T12:39:34.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.459+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c810ae80 con 0x7f84c8108900 2023-10-18T12:39:34.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.461+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f84c0004000 con 0x7f84c8108900 2023-10-18T12:39:34.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.463+0000 7f84affff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84b00a1670 0x7f84b00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:34.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.463+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f84c00cca50 con 0x7f84c8108900 2023-10-18T12:39:34.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.463+0000 7f84c6d9d700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84b00a1670 0x7f84b00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.464+0000 7f84c6d9d700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84b00a1670 0x7f84b00a3b30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b2e0 tx=0x7f84b80058e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:34.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.467+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f84c0095a10 con 0x7f84c8108900 2023-10-18T12:39:34.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.820+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c80761e0 con 0x7f84c8108900 2023-10-18T12:39:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.821+0000 7f84affff700 1 -- 172.21.15.70:0/1872559855 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f84c0099030 con 0x7f84c8108900 2023-10-18T12:39:34.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:39:34.824 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:39:34.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84b00a1670 msgr2=0x7f84b00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84b00a1670 0x7f84b00a3b30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b2e0 tx=0x7f84b80058e0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 msgr2=0x7f84c8075780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:34.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 0x7f84c8075780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84c000ef30 tx=0x7f84c000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 shutdown_connections 2023-10-18T12:39:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c8108900 0x7f84c8075780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f84b00a1670 0x7f84b00a3b30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84c81028f0 0x7f84c8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 --2- 172.21.15.70:0/1872559855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84c81032b0 0x7f84c8075240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.823+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 >> 172.21.15.70:0/1872559855 conn(0x7f84c80fe3a0 msgr2=0x7f84c80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.824+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 shutdown_connections 2023-10-18T12:39:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:34.824+0000 7f84cd1ff700 1 -- 172.21.15.70:0/1872559855 wait complete. 2023-10-18T12:39:34.829 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:39:34.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:39:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 -- 172.21.15.70:0/621333353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6641010d0 msgr2=0x7fa66410d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/621333353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6641010d0 0x7fa66410d5e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b0a0 tx=0x7fa65c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 -- 172.21.15.70:0/621333353 shutdown_connections 2023-10-18T12:39:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/621333353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6641010d0 0x7fa66410d5e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/621333353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa664106720 0x7fa664106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/621333353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 0x7fa664100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 -- 172.21.15.70:0/621333353 >> 172.21.15.70:0/621333353 conn(0x7fa6640fc200 msgr2=0x7fa6640fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 -- 172.21.15.70:0/621333353 shutdown_connections 2023-10-18T12:39:35.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.084+0000 7fa66c3ff700 1 -- 172.21.15.70:0/621333353 wait complete. 2023-10-18T12:39:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.085+0000 7fa66c3ff700 1 Processor -- start 2023-10-18T12:39:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.085+0000 7fa66c3ff700 1 -- start start 2023-10-18T12:39:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66c3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 0x7fa664198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6641010d0 0x7fa664199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66c3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa664106720 0x7fa66419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66c3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66419cb30 con 0x7fa664100710 2023-10-18T12:39:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66c3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66419cca0 con 0x7fa664106720 2023-10-18T12:39:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66419ce60 con 0x7fa6641010d0 2023-10-18T12:39:35.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66a19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 0x7fa664198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66999a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6641010d0 0x7fa664199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66a19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 0x7fa664198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36896/0 (socket says 172.21.15.70:36896) 2023-10-18T12:39:35.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66a19b700 1 -- 172.21.15.70:0/1730036030 learned_addr learned my addr 172.21.15.70:0/1730036030 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66a99c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa664106720 0x7fa66419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.086+0000 7fa66a99c700 1 -- 172.21.15.70:0/1730036030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6641010d0 msgr2=0x7fa664199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66a99c700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6641010d0 0x7fa664199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66a99c700 1 -- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 msgr2=0x7fa664198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66a99c700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 0x7fa664198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66a99c700 1 -- 172.21.15.70:0/1730036030 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009520 con 0x7fa664106720 2023-10-18T12:39:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66a99c700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa664106720 0x7fa66419c530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b070 tx=0x7fa65c0077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa65b7fe700 1 -- 172.21.15.70:0/1730036030 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00e040 con 0x7fa664106720 2023-10-18T12:39:35.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa660009520 con 0x7fa664106720 2023-10-18T12:39:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa65b7fe700 1 -- 172.21.15.70:0/1730036030 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa65c004e60 con 0x7fa664106720 2023-10-18T12:39:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa65b7fe700 1 -- 172.21.15.70:0/1730036030 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c005040 con 0x7fa664106720 2023-10-18T12:39:35.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.087+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c009d00 con 0x7fa664106720 2023-10-18T12:39:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.090+0000 7fa65b7fe700 1 -- 172.21.15.70:0/1730036030 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa65c01d470 con 0x7fa664106720 2023-10-18T12:39:35.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 Processor -- start 2023-10-18T12:39:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 -- start start 2023-10-18T12:39:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa65b7fe700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa6500a15f0 0x7fa6500a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa65b7fe700 1 -- 172.21.15.70:0/1730036030 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa65c0ce920 con 0x7fa664106720 2023-10-18T12:39:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7fbe0 0x55837ad7ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa66999a700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa6500a15f0 0x7fa6500a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad7aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837aced6e0 con 0x55837ad7a5d0 2023-10-18T12:39:35.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837acd3c60 con 0x55837ad80500 2023-10-18T12:39:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.092+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837acd1580 con 0x55837ad7fbe0 2023-10-18T12:39:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad7aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7fbe0 0x55837ad7ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad7a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32850/0 (socket says 172.21.15.70:32850) 2023-10-18T12:39:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66999a700 1 -- 172.21.15.70:0/2165056379 learned_addr learned my addr 172.21.15.70:0/2165056379 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 -- 172.21.15.70:0/2165056379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 msgr2=0x55837ad7a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad7a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 -- 172.21.15.70:0/2165056379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 msgr2=0x55837ad7aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad7aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 -- 172.21.15.70:0/2165056379 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837ac8a3f0 con 0x55837ad7fbe0 2023-10-18T12:39:35.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66999a700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa6500a15f0 0x7fa6500a3ab0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa654009fd0 tx=0x7fa654009d60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:35.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.093+0000 7fa66a19b700 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7fbe0 0x55837ad7ffc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa660009e80 tx=0x7fa660011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633816ace1d28161 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:35.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa658bfe700 1 -- 172.21.15.70:0/2165056379 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660004a30 con 0x55837ad7fbe0 2023-10-18T12:39:35.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa658bfe700 1 -- 172.21.15.70:0/2165056379 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6600054e0 con 0x55837ad7fbe0 2023-10-18T12:39:35.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa658bfe700 1 -- 172.21.15.70:0/2165056379 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66000f8c0 con 0x55837ad7fbe0 2023-10-18T12:39:35.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 -- 172.21.15.70:0/2165056379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7fbe0 msgr2=0x55837ad7ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7fbe0 0x55837ad7ffc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa660009e80 tx=0x7fa660011720 comp rx=0 tx=0).stop 2023-10-18T12:39:35.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 -- 172.21.15.70:0/2165056379 shutdown_connections 2023-10-18T12:39:35.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad7aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7fbe0 0x55837ad7ffc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/2165056379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad7a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 -- 172.21.15.70:0/2165056379 >> 172.21.15.70:0/2165056379 conn(0x55837acee720 msgr2=0x55837ad74b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.094+0000 7fa67fd1db80 1 -- 172.21.15.70:0/2165056379 shutdown_connections 2023-10-18T12:39:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.095+0000 7fa67fd1db80 1 -- 172.21.15.70:0/2165056379 wait complete. 2023-10-18T12:39:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.095+0000 7fa67fd1db80 1 -- start start 2023-10-18T12:39:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.095+0000 7fa67fd1db80 1 Processor -- start 2023-10-18T12:39:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7a5d0 0x55837ad864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837accce70 con 0x55837ad7fbe0 2023-10-18T12:39:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ac69710 con 0x55837ad80500 2023-10-18T12:39:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ac693b0 con 0x55837ad7a5d0 2023-10-18T12:39:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa67fd1db80 10 client.0 _ll_register_callbacks cb 0x55837abb3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa66999a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa66a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7a5d0 0x55837ad864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa66999a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad83720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36904/0 (socket says 172.21.15.70:36904) 2023-10-18T12:39:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.096+0000 7fa66999a700 1 -- 172.21.15.70:0/576929533 learned_addr learned my addr 172.21.15.70:0/576929533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a99c700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a19b700 1 -- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 msgr2=0x55837ad83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a19b700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a19b700 1 -- 172.21.15.70:0/576929533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 msgr2=0x55837ad83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a19b700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad83720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a19b700 1 -- 172.21.15.70:0/576929533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa660009ab0 con 0x55837ad7a5d0 2023-10-18T12:39:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66999a700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa66a19b700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7a5d0 0x55837ad864a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa660009230 tx=0x7fa6600053c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f653e5a422fbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:39:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa64dffb700 1 -- 172.21.15.70:0/576929533 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660005d60 con 0x55837ad7a5d0 2023-10-18T12:39:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.097+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55837acd1ef0 con 0x55837ad7a5d0 2023-10-18T12:39:35.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.098+0000 7fa64dffb700 1 -- 172.21.15.70:0/576929533 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa660004960 con 0x55837ad7a5d0 2023-10-18T12:39:35.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.098+0000 7fa64dffb700 1 -- 172.21.15.70:0/576929533 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa660004b20 con 0x55837ad7a5d0 2023-10-18T12:39:35.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.098+0000 7fa67fd1db80 10 client.27556 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:35.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.098+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55837aaf7de0 con 0x55837ad7a5d0 2023-10-18T12:39:35.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.098+0000 7fa64dffb700 1 -- 172.21.15.70:0/576929533 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55837acd1ef0 con 0x55837ad7a5d0 2023-10-18T12:39:35.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.098+0000 7fa64dffb700 1 -- 172.21.15.70:0/576929533 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa660021040 con 0x55837ad7a5d0 2023-10-18T12:39:35.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa67fd1db80 10 client.27556 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:35.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa67fd1db80 10 client.27556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:35.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa67fd1db80 20 client.27556 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:35.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837ad93d20 0x55837ad96140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa67fd1db80 4 client.27556 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:35.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55837ad498b0 con 0x55837ad93d20 2023-10-18T12:39:35.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa66999a700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837ad93d20 0x55837ad96140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa66999a700 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837ad93d20 0x55837ad96140 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76083833d3249c59 server_cookie=637a97847cfacb7 in_seq=0 out_seq=0 2023-10-18T12:39:35.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.099+0000 7fa64dffb700 10 client.27556 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:35.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa64dffb700 1 -- 172.21.15.70:0/576929533 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55837ad498b0 con 0x55837ad93d20 2023-10-18T12:39:35.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa64dffb700 10 client.27556 handle_command_reply: tid=0 2023-10-18T12:39:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 client.27556 shutdown 2023-10-18T12:39:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7a5d0 msgr2=0x55837ad864a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7a5d0 0x55837ad864a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa660009230 tx=0x7fa6600053c0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 shutdown_connections 2023-10-18T12:39:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837ad93d20 0x55837ad96140 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad80500 0x55837ad83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad7a5d0 0x55837ad864a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/576929533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 >> 172.21.15.70:0/576929533 conn(0x55837acee720 msgr2=0x55837ad79080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:35.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa64dffb700 0 client.27556 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 shutdown_connections 2023-10-18T12:39:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 1 -- 172.21.15.70:0/576929533 wait complete. 2023-10-18T12:39:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.101+0000 7fa67fd1db80 20 client.27556 trim_cache size 0 max 16384 2023-10-18T12:39:35.120 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:34 smithi070 ceph-mon[98639]: pgmap v735: 129 pgs: 129 active+clean; 595 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 206 op/s 2023-10-18T12:39:35.121 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:34 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1872559855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:39:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 Processor -- start 2023-10-18T12:39:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 -- start start 2023-10-18T12:39:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7a5d0 0x55837ad83630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837ad8aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ad6faf0 con 0x55837ad7fbe0 2023-10-18T12:39:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ab4d920 con 0x55837ad7a5d0 2023-10-18T12:39:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.116+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ad750c0 con 0x55837ad80500 2023-10-18T12:39:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837ad8aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837ad8aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35028/0 (socket says 172.21.15.70:35028) 2023-10-18T12:39:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 -- 172.21.15.70:0/4103174959 learned_addr learned my addr 172.21.15.70:0/4103174959 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:35.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a19b700 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7a5d0 0x55837ad83630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66999a700 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 -- 172.21.15.70:0/4103174959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7a5d0 msgr2=0x55837ad83630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7a5d0 0x55837ad83630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 -- 172.21.15.70:0/4103174959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 msgr2=0x55837ad8fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad8fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 -- 172.21.15.70:0/4103174959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55837aae0c60 con 0x55837ad80500 2023-10-18T12:39:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.117+0000 7fa66a99c700 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837ad8aeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa65c09b560 tx=0x7fa65c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c80c597025598810 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa6593ff700 1 -- 172.21.15.70:0/4103174959 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c09b110 con 0x55837ad80500 2023-10-18T12:39:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa6593ff700 1 -- 172.21.15.70:0/4103174959 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa65c09af30 con 0x55837ad80500 2023-10-18T12:39:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa6593ff700 1 -- 172.21.15.70:0/4103174959 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0a38d0 con 0x55837ad80500 2023-10-18T12:39:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4103174959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 msgr2=0x55837ad8aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837ad8aeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa65c09b560 tx=0x7fa65c0099e0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4103174959 shutdown_connections 2023-10-18T12:39:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837ad8aeb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7a5d0 0x55837ad83630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4103174959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7fbe0 0x55837ad8fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4103174959 >> 172.21.15.70:0/4103174959 conn(0x55837acee720 msgr2=0x55837ad8f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.118+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4103174959 shutdown_connections 2023-10-18T12:39:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.119+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4103174959 wait complete. 2023-10-18T12:39:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.119+0000 7fa67fd1db80 1 -- start start 2023-10-18T12:39:35.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.119+0000 7fa67fd1db80 1 Processor -- start 2023-10-18T12:39:35.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7fbe0 0x55837ad895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837adef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837ad89d00 con 0x55837ad7a5d0 2023-10-18T12:39:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837adef890 con 0x55837ad7fbe0 2023-10-18T12:39:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55837adefa20 con 0x55837ad80500 2023-10-18T12:39:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa67fd1db80 10 client.0 _ll_register_callbacks cb 0x55837abb3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66a19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66999a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7fbe0 0x55837ad895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66a19b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad89080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36922/0 (socket says 172.21.15.70:36922) 2023-10-18T12:39:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66a19b700 1 -- 172.21.15.70:0/4070267414 learned_addr learned my addr 172.21.15.70:0/4070267414 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66a99c700 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837adef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66999a700 1 -- 172.21.15.70:0/4070267414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 msgr2=0x55837adef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66999a700 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837adef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66999a700 1 -- 172.21.15.70:0/4070267414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 msgr2=0x55837ad89080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66999a700 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad89080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.120+0000 7fa66999a700 1 -- 172.21.15.70:0/4070267414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c009d00 con 0x55837ad7fbe0 2023-10-18T12:39:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa66999a700 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7fbe0 0x55837ad895c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6540099e0 tx=0x7fa6540047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1769720039d18f50 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa64dffb700 1 -- 172.21.15.70:0/4070267414 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65402a8f0 con 0x55837ad7fbe0 2023-10-18T12:39:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6600054e0 con 0x55837ad7fbe0 2023-10-18T12:39:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa64dffb700 1 -- 172.21.15.70:0/4070267414 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa654026040 con 0x55837ad7fbe0 2023-10-18T12:39:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa64dffb700 1 -- 172.21.15.70:0/4070267414 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa65402aa50 con 0x55837ad7fbe0 2023-10-18T12:39:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa67fd1db80 10 client.27575 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55837ab1e250 con 0x55837ad7fbe0 2023-10-18T12:39:35.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.121+0000 7fa64dffb700 1 -- 172.21.15.70:0/4070267414 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6600054e0 con 0x55837ad7fbe0 2023-10-18T12:39:35.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa64dffb700 1 -- 172.21.15.70:0/4070267414 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa654003680 con 0x55837ad7fbe0 2023-10-18T12:39:35.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa67fd1db80 10 client.27575 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:35.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa67fd1db80 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:35.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa67fd1db80 20 client.27575 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837adf1210 0x55837adf3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa67fd1db80 4 client.27575 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:39:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55837acc5e50 con 0x55837adf1210 2023-10-18T12:39:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa66a19b700 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837adf1210 0x55837adf3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa64dffb700 10 client.27575 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.122+0000 7fa66a19b700 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837adf1210 0x55837adf3630 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653dc4e4df9d87a4 server_cookie=c4e1c831f2b8cee5 in_seq=0 out_seq=0 2023-10-18T12:39:35.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa64dffb700 1 -- 172.21.15.70:0/4070267414 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55837acc5e50 con 0x55837adf1210 2023-10-18T12:39:35.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa64dffb700 10 client.27575 handle_command_reply: tid=0 2023-10-18T12:39:35.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 client.27575 shutdown 2023-10-18T12:39:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7fbe0 msgr2=0x55837ad895c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7fbe0 0x55837ad895c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6540099e0 tx=0x7fa6540047c0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 shutdown_connections 2023-10-18T12:39:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55837adf1210 0x55837adf3630 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55837ad80500 0x55837adef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55837ad7a5d0 0x55837ad89080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa64dffb700 0 client.27575 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 --2- 172.21.15.70:0/4070267414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55837ad7fbe0 0x55837ad895c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 >> 172.21.15.70:0/4070267414 conn(0x55837acee720 msgr2=0x55837ad81ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 shutdown_connections 2023-10-18T12:39:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 1 -- 172.21.15.70:0/4070267414 wait complete. 2023-10-18T12:39:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.127+0000 7fa67fd1db80 20 client.27575 trim_cache size 0 max 16384 2023-10-18T12:39:35.143 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:39:35.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa6500a15f0 msgr2=0x7fa6500a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa6500a15f0 0x7fa6500a3ab0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa654009fd0 tx=0x7fa654009d60 comp rx=0 tx=0).stop 2023-10-18T12:39:35.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa664106720 msgr2=0x7fa66419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:35.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa664106720 0x7fa66419c530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b070 tx=0x7fa65c0077f0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 shutdown_connections 2023-10-18T12:39:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa6500a15f0 0x7fa6500a3ab0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa664106720 0x7fa66419c530 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa664100710 0x7fa664198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 --2- 172.21.15.70:0/1730036030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6641010d0 0x7fa664199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 >> 172.21.15.70:0/1730036030 conn(0x7fa6640fc200 msgr2=0x7fa6640fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 shutdown_connections 2023-10-18T12:39:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:35.128+0000 7fa66c3ff700 1 -- 172.21.15.70:0/1730036030 wait complete. 2023-10-18T12:39:35.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:34 smithi152 ceph-mon[134767]: pgmap v735: 129 pgs: 129 active+clean; 595 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 206 op/s 2023-10-18T12:39:35.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:34 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1872559855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:39:35.172 INFO:tasks.workunit.client.0.smithi070.stdout: 10 1195 75.98 MB/sec warmup 3 sec latency 180.205 ms 2023-10-18T12:39:35.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:34 smithi186 ceph-mon[135253]: pgmap v735: 129 pgs: 129 active+clean; 595 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 206 op/s 2023-10-18T12:39:35.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:34 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1872559855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:39:36.152 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T12:39:36.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T12:39:36.172 INFO:tasks.workunit.client.0.smithi070.stdout: 10 2147 66.38 MB/sec warmup 4 sec latency 224.886 ms 2023-10-18T12:39:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 -- 172.21.15.70:0/394412348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04381025b0 msgr2=0x7f0438102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 --2- 172.21.15.70:0/394412348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04381025b0 0x7f0438102990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0428009990 tx=0x7f0428009ca0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 -- 172.21.15.70:0/394412348 shutdown_connections 2023-10-18T12:39:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 --2- 172.21.15.70:0/394412348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0438069390 0x7f043810d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 --2- 172.21.15.70:0/394412348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04381025b0 0x7f0438102990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 --2- 172.21.15.70:0/394412348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04380689d0 0x7f0438068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 -- 172.21.15.70:0/394412348 >> 172.21.15.70:0/394412348 conn(0x7f0438077570 msgr2=0x7f0438077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 -- 172.21.15.70:0/394412348 shutdown_connections 2023-10-18T12:39:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.386+0000 7f043d3ff700 1 -- 172.21.15.70:0/394412348 wait complete. 2023-10-18T12:39:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.387+0000 7f043d3ff700 1 Processor -- start 2023-10-18T12:39:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 -- start start 2023-10-18T12:39:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 0x7f0438198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0438069390 0x7f0438199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04381025b0 0x7f043819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043819de00 con 0x7f0438069390 2023-10-18T12:39:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043819df70 con 0x7f04381025b0 2023-10-18T12:39:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.388+0000 7f043d3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043819e130 con 0x7f04380689d0 2023-10-18T12:39:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 0x7f0438198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 0x7f0438198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35054/0 (socket says 172.21.15.70:35054) 2023-10-18T12:39:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 -- 172.21.15.70:0/1143823346 learned_addr learned my addr 172.21.15.70:0/1143823346 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f04377fe700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04381025b0 0x7f043819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f04367fc700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0438069390 0x7f0438199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 -- 172.21.15.70:0/1143823346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04381025b0 msgr2=0x7f043819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04381025b0 0x7f043819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 -- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0438069390 msgr2=0x7f0438199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0438069390 0x7f0438199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 -- 172.21.15.70:0/1143823346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280095f0 con 0x7f04380689d0 2023-10-18T12:39:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f0436ffd700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 0x7f0438198ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f0428005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.389+0000 7f041ffff700 1 -- 172.21.15.70:0/1143823346 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042801d070 con 0x7f04380689d0 2023-10-18T12:39:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.390+0000 7f041ffff700 1 -- 172.21.15.70:0/1143823346 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0428022470 con 0x7f04380689d0 2023-10-18T12:39:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.390+0000 7f041ffff700 1 -- 172.21.15.70:0/1143823346 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428017650 con 0x7f04380689d0 2023-10-18T12:39:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.390+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043819e3b0 con 0x7f04380689d0 2023-10-18T12:39:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.390+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043819e630 con 0x7f04380689d0 2023-10-18T12:39:36.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.394+0000 7f041ffff700 1 -- 172.21.15.70:0/1143823346 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f04280225e0 con 0x7f04380689d0 2023-10-18T12:39:36.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.395+0000 7f0450e0eb80 1 Processor -- start 2023-10-18T12:39:36.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.395+0000 7f0450e0eb80 1 -- start start 2023-10-18T12:39:36.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.395+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b746c60 0x55aa3b747040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 0x55aa3b741110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b741ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b6b4760 con 0x55aa3b747580 2023-10-18T12:39:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b69ace0 con 0x55aa3b746c60 2023-10-18T12:39:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b698600 con 0x55aa3b741650 2023-10-18T12:39:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f041ffff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04200a15f0 0x7f04200a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f04377fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b746c60 0x55aa3b747040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 0x55aa3b741110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 0x55aa3b741110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36934/0 (socket says 172.21.15.70:36934) 2023-10-18T12:39:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 -- 172.21.15.70:0/421537815 learned_addr learned my addr 172.21.15.70:0/421537815 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f04367fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b741ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f041ffff700 1 -- 172.21.15.70:0/1143823346 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f04280ce640 con 0x7f04380689d0 2023-10-18T12:39:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f04377fe700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04200a15f0 0x7f04200a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 -- 172.21.15.70:0/421537815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 msgr2=0x55aa3b741ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b741ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 -- 172.21.15.70:0/421537815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b746c60 msgr2=0x55aa3b747040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.396+0000 7f0436ffd700 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b746c60 0x55aa3b747040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f0436ffd700 1 -- 172.21.15.70:0/421537815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3b651470 con 0x55aa3b747580 2023-10-18T12:39:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f04377fe700 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b746c60 0x55aa3b747040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f0436ffd700 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 0x55aa3b741110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f04280a3240 tx=0x7f042806e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbb70fe4301a7b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f041d7fa700 1 -- 172.21.15.70:0/421537815 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042809b240 con 0x55aa3b747580 2023-10-18T12:39:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f041d7fa700 1 -- 172.21.15.70:0/421537815 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042809afb0 con 0x55aa3b747580 2023-10-18T12:39:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f04377fe700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04200a15f0 0x7f04200a3ab0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0430009ec0 tx=0x7f0430004b40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.397+0000 7f041d7fa700 1 -- 172.21.15.70:0/421537815 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042809f5c0 con 0x55aa3b747580 2023-10-18T12:39:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 -- 172.21.15.70:0/421537815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 msgr2=0x55aa3b741110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 0x55aa3b741110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f04280a3240 tx=0x7f042806e500 comp rx=0 tx=0).stop 2023-10-18T12:39:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 -- 172.21.15.70:0/421537815 shutdown_connections 2023-10-18T12:39:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b741ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b746c60 0x55aa3b747040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/421537815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b747580 0x55aa3b741110 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 -- 172.21.15.70:0/421537815 >> 172.21.15.70:0/421537815 conn(0x55aa3b6b57a0 msgr2=0x55aa3b73bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 -- 172.21.15.70:0/421537815 shutdown_connections 2023-10-18T12:39:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.398+0000 7f0450e0eb80 1 -- 172.21.15.70:0/421537815 wait complete. 2023-10-18T12:39:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 -- start start 2023-10-18T12:39:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 Processor -- start 2023-10-18T12:39:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b74d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b693ef0 con 0x55aa3b746c60 2023-10-18T12:39:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b630790 con 0x55aa3b747580 2023-10-18T12:39:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b630430 con 0x55aa3b741650 2023-10-18T12:39:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.399+0000 7f0450e0eb80 10 client.0 _ll_register_callbacks cb 0x55aa3b57ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04367fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32924/0 (socket says 172.21.15.70:32924) 2023-10-18T12:39:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 -- 172.21.15.70:0/2022909849 learned_addr learned my addr 172.21.15.70:0/2022909849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f0436ffd700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b74d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 -- 172.21.15.70:0/2022909849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 msgr2=0x55aa3b74d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b74d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 -- 172.21.15.70:0/2022909849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 msgr2=0x55aa3b74a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 -- 172.21.15.70:0/2022909849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04300137f0 con 0x55aa3b747580 2023-10-18T12:39:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04367fc700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f04377fe700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74ad80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0430018fb0 tx=0x7f0430004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd667048ab3d65b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.400+0000 7f041aff5700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f041aff5700 1 -- 172.21.15.70:0/2022909849 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000e040 con 0x55aa3b747580 2023-10-18T12:39:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa3b698f70 con 0x55aa3b747580 2023-10-18T12:39:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f041aff5700 1 -- 172.21.15.70:0/2022909849 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f043001f050 con 0x55aa3b747580 2023-10-18T12:39:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f041aff5700 1 -- 172.21.15.70:0/2022909849 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0430007c70 con 0x55aa3b747580 2023-10-18T12:39:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f0450e0eb80 10 client.27587 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f043819e3b0 con 0x55aa3b747580 2023-10-18T12:39:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.401+0000 7f041aff5700 1 -- 172.21.15.70:0/2022909849 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa3b698f70 con 0x55aa3b747580 2023-10-18T12:39:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f041aff5700 1 -- 172.21.15.70:0/2022909849 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f043001abe0 con 0x55aa3b747580 2023-10-18T12:39:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0450e0eb80 10 client.27587 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0450e0eb80 10 client.27587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0450e0eb80 20 client.27587 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:36.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b75ac40 0x55aa3b75d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0450e0eb80 4 client.27587 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa3b710930 con 0x55aa3b75ac40 2023-10-18T12:39:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0436ffd700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b75ac40 0x55aa3b75d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f0436ffd700 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b75ac40 0x55aa3b75d060 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a68ff157ce26b84 server_cookie=e9fc856b5b4855b9 in_seq=0 out_seq=0 2023-10-18T12:39:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.402+0000 7f041aff5700 10 client.27587 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.405+0000 7f041aff5700 1 -- 172.21.15.70:0/2022909849 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa3b710930 con 0x55aa3b75ac40 2023-10-18T12:39:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.405+0000 7f041aff5700 10 client.27587 handle_command_reply: tid=0 2023-10-18T12:39:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.405+0000 7f0450e0eb80 1 client.27587 shutdown 2023-10-18T12:39:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 msgr2=0x55aa3b74ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74ad80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0430018fb0 tx=0x7f0430004ad0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 shutdown_connections 2023-10-18T12:39:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b75ac40 0x55aa3b75d060 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74ad80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b74d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f041aff5700 0 client.27587 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/2022909849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 >> 172.21.15.70:0/2022909849 conn(0x55aa3b6b57a0 msgr2=0x55aa3b73bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.406+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 shutdown_connections 2023-10-18T12:39:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.408+0000 7f0450e0eb80 1 -- 172.21.15.70:0/2022909849 wait complete. 2023-10-18T12:39:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.409+0000 7f0450e0eb80 20 client.27587 trim_cache size 0 max 16384 2023-10-18T12:39:36.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.428+0000 7f0450e0eb80 1 Processor -- start 2023-10-18T12:39:36.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.428+0000 7f0450e0eb80 1 -- start start 2023-10-18T12:39:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 0x55aa3b756b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b746c60 0x55aa3b751cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b5149a0 con 0x55aa3b741650 2023-10-18T12:39:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b74bd90 con 0x55aa3b747580 2023-10-18T12:39:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b73d430 con 0x55aa3b746c60 2023-10-18T12:39:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 0x55aa3b756b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f04377fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 0x55aa3b756b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36942/0 (socket says 172.21.15.70:36942) 2023-10-18T12:39:36.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f04377fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32940/0 (socket says 172.21.15.70:32940) 2023-10-18T12:39:36.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 -- 172.21.15.70:0/4109193803 learned_addr learned my addr 172.21.15.70:0/4109193803 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f04367fc700 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b746c60 0x55aa3b751cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 -- 172.21.15.70:0/4109193803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b746c60 msgr2=0x55aa3b751cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b746c60 0x55aa3b751cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 -- 172.21.15.70:0/4109193803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 msgr2=0x55aa3b74cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.429+0000 7f0436ffd700 1 -- 172.21.15.70:0/4109193803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3b73bdf0 con 0x55aa3b741650 2023-10-18T12:39:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.430+0000 7f0436ffd700 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 0x55aa3b756b10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f04280cbae0 tx=0x7f042801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200fd15d5cb5c011 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.430+0000 7f041dffb700 1 -- 172.21.15.70:0/4109193803 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280b3ea0 con 0x55aa3b741650 2023-10-18T12:39:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.430+0000 7f041dffb700 1 -- 172.21.15.70:0/4109193803 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042809add0 con 0x55aa3b741650 2023-10-18T12:39:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.430+0000 7f041dffb700 1 -- 172.21.15.70:0/4109193803 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042809b470 con 0x55aa3b741650 2023-10-18T12:39:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 -- 172.21.15.70:0/4109193803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 msgr2=0x55aa3b756b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 0x55aa3b756b10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f04280cbae0 tx=0x7f042801a040 comp rx=0 tx=0).stop 2023-10-18T12:39:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 -- 172.21.15.70:0/4109193803 shutdown_connections 2023-10-18T12:39:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b74cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b741650 0x55aa3b756b10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/4109193803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b746c60 0x55aa3b751cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 -- 172.21.15.70:0/4109193803 >> 172.21.15.70:0/4109193803 conn(0x55aa3b6b57a0 msgr2=0x55aa3b756400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 -- 172.21.15.70:0/4109193803 shutdown_connections 2023-10-18T12:39:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 -- 172.21.15.70:0/4109193803 wait complete. 2023-10-18T12:39:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.431+0000 7f0450e0eb80 1 -- start start 2023-10-18T12:39:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 Processor -- start 2023-10-18T12:39:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b751c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b750140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b750850 con 0x55aa3b746c60 2023-10-18T12:39:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b7509c0 con 0x55aa3b747580 2023-10-18T12:39:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3b7afa80 con 0x55aa3b741650 2023-10-18T12:39:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0450e0eb80 10 client.0 _ll_register_callbacks cb 0x55aa3b57ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f04367fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b751c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0436ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b751c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35098/0 (socket says 172.21.15.70:35098) 2023-10-18T12:39:36.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f04367fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36952/0 (socket says 172.21.15.70:36952) 2023-10-18T12:39:36.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.432+0000 7f0436ffd700 1 -- 172.21.15.70:0/308312225 learned_addr learned my addr 172.21.15.70:0/308312225 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f04377fe700 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b750140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0436ffd700 1 -- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 msgr2=0x55aa3b750140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0436ffd700 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b750140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0436ffd700 1 -- 172.21.15.70:0/308312225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 msgr2=0x55aa3b74fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0436ffd700 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0436ffd700 1 -- 172.21.15.70:0/308312225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa3b651470 con 0x55aa3b741650 2023-10-18T12:39:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0436ffd700 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b751c20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f04280997c0 tx=0x7f04280978e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbdecabfd02aaffe server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f041aff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:39:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f041aff5700 1 -- 172.21.15.70:0/308312225 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428069020 con 0x55aa3b741650 2023-10-18T12:39:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f042809afb0 con 0x55aa3b741650 2023-10-18T12:39:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f041aff5700 1 -- 172.21.15.70:0/308312225 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f042809b470 con 0x55aa3b741650 2023-10-18T12:39:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f041aff5700 1 -- 172.21.15.70:0/308312225 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04280b3ea0 con 0x55aa3b741650 2023-10-18T12:39:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0450e0eb80 10 client.27580 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.433+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55aa3b4bee60 con 0x55aa3b741650 2023-10-18T12:39:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f041aff5700 1 -- 172.21.15.70:0/308312225 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042809afb0 con 0x55aa3b741650 2023-10-18T12:39:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f041aff5700 1 -- 172.21.15.70:0/308312225 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04280ce260 con 0x55aa3b741650 2023-10-18T12:39:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f0450e0eb80 10 client.27580 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f0450e0eb80 10 client.27580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f0450e0eb80 20 client.27580 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b7b0c10 0x55aa3b7b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f0450e0eb80 4 client.27580 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T12:39:36.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55aa3b68ced0 con 0x55aa3b7b0c10 2023-10-18T12:39:36.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.434+0000 7f04367fc700 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b7b0c10 0x55aa3b7b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.435+0000 7f04367fc700 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b7b0c10 0x55aa3b7b3000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16d35618be60c0d server_cookie=60e9936389f230ea in_seq=0 out_seq=0 2023-10-18T12:39:36.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.435+0000 7f041aff5700 10 client.27580 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.438+0000 7f041aff5700 1 -- 172.21.15.70:0/308312225 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aa3b68ced0 con 0x55aa3b7b0c10 2023-10-18T12:39:36.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.438+0000 7f041aff5700 10 client.27580 handle_command_reply: tid=0 2023-10-18T12:39:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.438+0000 7f0450e0eb80 1 client.27580 shutdown 2023-10-18T12:39:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 msgr2=0x55aa3b751c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b751c20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f04280997c0 tx=0x7f04280978e0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 shutdown_connections 2023-10-18T12:39:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x55aa3b7b0c10 0x55aa3b7b3000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa3b747580 0x55aa3b750140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3b741650 0x55aa3b751c20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 --2- 172.21.15.70:0/308312225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa3b746c60 0x55aa3b74fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f041aff5700 0 client.27580 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 >> 172.21.15.70:0/308312225 conn(0x55aa3b6b57a0 msgr2=0x55aa3b735530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 shutdown_connections 2023-10-18T12:39:36.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 1 -- 172.21.15.70:0/308312225 wait complete. 2023-10-18T12:39:36.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.439+0000 7f0450e0eb80 20 client.27580 trim_cache size 0 max 16384 2023-10-18T12:39:36.453 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:39:36.454 INFO:teuthology.orchestra.run.smithi070.stdout: "return_code": 0, 2023-10-18T12:39:36.454 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_tag": "7767e7d1-a466-439e-ae45-63ceb0299142", 2023-10-18T12:39:36.454 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": "asynchronous" 2023-10-18T12:39:36.454 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:39:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04200a15f0 msgr2=0x7f04200a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04200a15f0 0x7f04200a3ab0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0430009ec0 tx=0x7f0430004b40 comp rx=0 tx=0).stop 2023-10-18T12:39:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 msgr2=0x7f0438198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 0x7f0438198ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f0428005740 comp rx=0 tx=0).stop 2023-10-18T12:39:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 shutdown_connections 2023-10-18T12:39:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f04200a15f0 0x7f04200a3ab0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04381025b0 0x7f043819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04380689d0 0x7f0438198ee0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 --2- 172.21.15.70:0/1143823346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0438069390 0x7f0438199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 >> 172.21.15.70:0/1143823346 conn(0x7f0438077570 msgr2=0x7f0438105490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 shutdown_connections 2023-10-18T12:39:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.440+0000 7f043d3ff700 1 -- 172.21.15.70:0/1143823346 wait complete. 2023-10-18T12:39:36.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:39:36.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.765+0000 7fe6287ff700 1 -- 172.21.15.70:0/1417915185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 msgr2=0x7fe62010d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.765+0000 7fe6287ff700 1 --2- 172.21.15.70:0/1417915185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 0x7fe62010d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe61800b0d0 tx=0x7fe61800b3e0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 -- 172.21.15.70:0/1417915185 shutdown_connections 2023-10-18T12:39:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 --2- 172.21.15.70:0/1417915185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 0x7fe62010d540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 --2- 172.21.15.70:0/1417915185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 0x7fe620068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 --2- 172.21.15.70:0/1417915185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe6201055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 -- 172.21.15.70:0/1417915185 >> 172.21.15.70:0/1417915185 conn(0x7fe6200767f0 msgr2=0x7fe620076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 -- 172.21.15.70:0/1417915185 shutdown_connections 2023-10-18T12:39:36.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.766+0000 7fe6287ff700 1 -- 172.21.15.70:0/1417915185 wait complete. 2023-10-18T12:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 Processor -- start 2023-10-18T12:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 -- start start 2023-10-18T12:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 0x7fe620107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe620107a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 0x7fe620107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201a94f0 con 0x7fe620067d40 2023-10-18T12:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201a9630 con 0x7fe620105170 2023-10-18T12:39:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.767+0000 7fe6287ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201a97a0 con 0x7fe620105b30 2023-10-18T12:39:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe625d9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe620107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe625d9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe620107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32958/0 (socket says 172.21.15.70:32958) 2023-10-18T12:39:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe625d9a700 1 -- 172.21.15.70:0/3651580597 learned_addr learned my addr 172.21.15.70:0/3651580597 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 0x7fe620107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe626d9c700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 0x7fe620107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 -- 172.21.15.70:0/3651580597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 msgr2=0x7fe620107fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 0x7fe620107fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 -- 172.21.15.70:0/3651580597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 msgr2=0x7fe620107a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe620107a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 -- 172.21.15.70:0/3651580597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618009d00 con 0x7fe620067d40 2023-10-18T12:39:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe620107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:39:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe62659b700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 0x7fe620107550 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00e9f0 tx=0x7fe61c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.768+0000 7fe6177fe700 1 -- 172.21.15.70:0/3651580597 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00c9c0 con 0x7fe620067d40 2023-10-18T12:39:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.769+0000 7fe6177fe700 1 -- 172.21.15.70:0/3651580597 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61c010430 con 0x7fe620067d40 2023-10-18T12:39:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.769+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6201a9a80 con 0x7fe620067d40 2023-10-18T12:39:36.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.769+0000 7fe6177fe700 1 -- 172.21.15.70:0/3651580597 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c018680 con 0x7fe620067d40 2023-10-18T12:39:36.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.769+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6201a9f90 con 0x7fe620067d40 2023-10-18T12:39:36.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.772+0000 7fe6177fe700 1 -- 172.21.15.70:0/3651580597 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe61c010860 con 0x7fe620067d40 2023-10-18T12:39:36.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.774+0000 7fe6177fe700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe60c0a15f0 0x7fe60c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.774+0000 7fe63c13ab80 1 Processor -- start 2023-10-18T12:39:36.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.774+0000 7fe6177fe700 1 -- 172.21.15.70:0/3651580597 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe61c014070 con 0x7fe620067d40 2023-10-18T12:39:36.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe60c0a15f0 0x7fe60c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 -- start start 2023-10-18T12:39:36.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 0x557a0250cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a0250d500 0x557a02507090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025075d0 0x557a02507a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0247a6e0 con 0x557a0250cbe0 2023-10-18T12:39:36.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a02460c60 con 0x557a0250d500 2023-10-18T12:39:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0245e580 con 0x557a025075d0 2023-10-18T12:39:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe62659b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 0x557a0250cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe62659b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 0x557a0250cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36972/0 (socket says 172.21.15.70:36972) 2023-10-18T12:39:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe62659b700 1 -- 172.21.15.70:0/3527926419 learned_addr learned my addr 172.21.15.70:0/3527926419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.775+0000 7fe626d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025075d0 0x557a02507a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe60c0a15f0 0x7fe60c0a3ab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe620108c80 tx=0x7fe610007480 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a0250d500 0x557a02507090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe62659b700 1 -- 172.21.15.70:0/3527926419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025075d0 msgr2=0x557a02507a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe62659b700 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025075d0 0x557a02507a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe62659b700 1 -- 172.21.15.70:0/3527926419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a0250d500 msgr2=0x557a02507090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe62659b700 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a0250d500 0x557a02507090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe62659b700 1 -- 172.21.15.70:0/3527926419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a024173f0 con 0x557a0250cbe0 2023-10-18T12:39:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a0250d500 0x557a02507090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:39:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe62659b700 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 0x557a0250cfc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe61c096c60 tx=0x7fe61c099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dca4ae320cc7079 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe614ff9700 1 -- 172.21.15.70:0/3527926419 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c09e120 con 0x557a0250cbe0 2023-10-18T12:39:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe614ff9700 1 -- 172.21.15.70:0/3527926419 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61c09e340 con 0x557a0250cbe0 2023-10-18T12:39:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.776+0000 7fe614ff9700 1 -- 172.21.15.70:0/3527926419 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0cd3c0 con 0x557a0250cbe0 2023-10-18T12:39:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3527926419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 msgr2=0x557a0250cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 0x557a0250cfc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe61c096c60 tx=0x7fe61c099c50 comp rx=0 tx=0).stop 2023-10-18T12:39:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3527926419 shutdown_connections 2023-10-18T12:39:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025075d0 0x557a02507a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a0250cbe0 0x557a0250cfc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3527926419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a0250d500 0x557a02507090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3527926419 >> 172.21.15.70:0/3527926419 conn(0x557a0247b720 msgr2=0x557a02501b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3527926419 shutdown_connections 2023-10-18T12:39:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.777+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3527926419 wait complete. 2023-10-18T12:39:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.778+0000 7fe63c13ab80 1 -- start start 2023-10-18T12:39:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.778+0000 7fe63c13ab80 1 Processor -- start 2023-10-18T12:39:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.778+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02510a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 0x557a02510f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a0251cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a02459e70 con 0x557a025114b0 2023-10-18T12:39:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a023f6710 con 0x557a025075d0 2023-10-18T12:39:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a023f63b0 con 0x557a0250d500 2023-10-18T12:39:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe63c13ab80 10 client.0 _ll_register_callbacks cb 0x557a02340a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 0x557a02510f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 0x557a02510f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35126/0 (socket says 172.21.15.70:35126) 2023-10-18T12:39:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 -- 172.21.15.70:0/471646556 learned_addr learned my addr 172.21.15.70:0/471646556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe62659b700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02510a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 -- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 msgr2=0x557a02510a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02510a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 -- 172.21.15.70:0/471646556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 msgr2=0x557a0251cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe626d9c700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a0251cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a0251cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe625d9a700 1 -- 172.21.15.70:0/471646556 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6201a9f90 con 0x557a0250d500 2023-10-18T12:39:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.779+0000 7fe626d9c700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a0251cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:39:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe625d9a700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 0x557a02510f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe610011790 tx=0x7fe6100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba2099a5fc0f2818 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-18T12:39:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe609ffb700 1 -- 172.21.15.70:0/471646556 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61001d8c0 con 0x557a0250d500 2023-10-18T12:39:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe609ffb700 1 -- 172.21.15.70:0/471646556 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe610017070 con 0x557a0250d500 2023-10-18T12:39:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a0245eef0 con 0x557a0250d500 2023-10-18T12:39:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe609ffb700 1 -- 172.21.15.70:0/471646556 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe61001da20 con 0x557a0250d500 2023-10-18T12:39:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe63c13ab80 10 client.27592 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:36.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557a02284de0 con 0x557a0250d500 2023-10-18T12:39:36.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.780+0000 7fe609ffb700 1 -- 172.21.15.70:0/471646556 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a0245eef0 con 0x557a0250d500 2023-10-18T12:39:36.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe609ffb700 1 -- 172.21.15.70:0/471646556 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe6100263f0 con 0x557a0250d500 2023-10-18T12:39:36.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe63c13ab80 10 client.27592 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:36.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe63c13ab80 10 client.27592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe63c13ab80 20 client.27592 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02521510 0x557a02523930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe63c13ab80 4 client.27592 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:39:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a024d68b0 con 0x557a02521510 2023-10-18T12:39:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.781+0000 7fe62659b700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02521510 0x557a02523930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.782+0000 7fe62659b700 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02521510 0x557a02523930 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31c60db5bc44cef7 server_cookie=3c2ab8d040ecdec9 in_seq=0 out_seq=0 2023-10-18T12:39:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.782+0000 7fe609ffb700 10 client.27592 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.786+0000 7fe609ffb700 1 -- 172.21.15.70:0/471646556 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a024d68b0 con 0x557a02521510 2023-10-18T12:39:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.786+0000 7fe609ffb700 10 client.27592 handle_command_reply: tid=0 2023-10-18T12:39:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.786+0000 7fe63c13ab80 1 client.27592 shutdown 2023-10-18T12:39:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.786+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 msgr2=0x557a02510f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.786+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 0x557a02510f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe610011790 tx=0x7fe6100047c0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 shutdown_connections 2023-10-18T12:39:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02521510 0x557a02523930 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a0251cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02510a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/471646556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a0250d500 0x557a02510f70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe609ffb700 0 client.27592 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 >> 172.21.15.70:0/471646556 conn(0x557a0247b720 msgr2=0x557a02506080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 shutdown_connections 2023-10-18T12:39:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 1 -- 172.21.15.70:0/471646556 wait complete. 2023-10-18T12:39:36.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.787+0000 7fe63c13ab80 20 client.27592 trim_cache size 0 max 16384 2023-10-18T12:39:36.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 Processor -- start 2023-10-18T12:39:36.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 -- start start 2023-10-18T12:39:36.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02517e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 0x557a0250f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a02528250 0x557a0250f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a022da920 con 0x557a02528250 2023-10-18T12:39:36.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a024d68e0 con 0x557a025075d0 2023-10-18T12:39:36.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.807+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a025033b0 con 0x557a025114b0 2023-10-18T12:39:36.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 0x557a0250f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe62659b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02517e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 0x557a0250f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:35138/0 (socket says 172.21.15.70:35138) 2023-10-18T12:39:36.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 -- 172.21.15.70:0/1281831357 learned_addr learned my addr 172.21.15.70:0/1281831357 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe626d9c700 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a02528250 0x557a0250f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 -- 172.21.15.70:0/1281831357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 msgr2=0x557a02517e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02517e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 -- 172.21.15.70:0/1281831357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a02528250 msgr2=0x557a0250f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a02528250 0x557a0250f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 -- 172.21.15.70:0/1281831357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0226dc60 con 0x557a025114b0 2023-10-18T12:39:36.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe625d9a700 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 0x557a0250f320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe610026750 tx=0x7fe610011ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70fea16af6376e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.808+0000 7fe6157fa700 1 -- 172.21.15.70:0/1281831357 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610039070 con 0x557a025114b0 2023-10-18T12:39:36.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe6157fa700 1 -- 172.21.15.70:0/1281831357 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe610015070 con 0x557a025114b0 2023-10-18T12:39:36.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe6157fa700 1 -- 172.21.15.70:0/1281831357 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61001db80 con 0x557a025114b0 2023-10-18T12:39:36.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 -- 172.21.15.70:0/1281831357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 msgr2=0x557a0250f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 0x557a0250f320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe610026750 tx=0x7fe610011ee0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 -- 172.21.15.70:0/1281831357 shutdown_connections 2023-10-18T12:39:36.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a02528250 0x557a0250f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02517e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/1281831357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a025114b0 0x557a0250f320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 -- 172.21.15.70:0/1281831357 >> 172.21.15.70:0/1281831357 conn(0x557a0247b720 msgr2=0x557a02506080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 -- 172.21.15.70:0/1281831357 shutdown_connections 2023-10-18T12:39:36.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.809+0000 7fe63c13ab80 1 -- 172.21.15.70:0/1281831357 wait complete. 2023-10-18T12:39:36.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.810+0000 7fe63c13ab80 1 -- start start 2023-10-18T12:39:36.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.810+0000 7fe63c13ab80 1 Processor -- start 2023-10-18T12:39:36.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02516810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a02516d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a02528250 0x557a02517290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a02575490 con 0x557a025114b0 2023-10-18T12:39:36.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a02575600 con 0x557a025075d0 2023-10-18T12:39:36.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a025757c0 con 0x557a02528250 2023-10-18T12:39:36.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe63c13ab80 10 client.0 _ll_register_callbacks cb 0x557a02340a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:39:36.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02516810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02516810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:32994/0 (socket says 172.21.15.70:32994) 2023-10-18T12:39:36.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 -- 172.21.15.70:0/3710110973 learned_addr learned my addr 172.21.15.70:0/3710110973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:39:36.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe626d9c700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a02528250 0x557a02517290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a02516d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 -- 172.21.15.70:0/3710110973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a02528250 msgr2=0x557a02517290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a02528250 0x557a02517290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 -- 172.21.15.70:0/3710110973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 msgr2=0x557a02516d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:39:36.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.811+0000 7fe62659b700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a02516d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe62659b700 1 -- 172.21.15.70:0/3710110973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a02284de0 con 0x557a025075d0 2023-10-18T12:39:36.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a02516d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:39:36.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe62659b700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02516810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009800 tx=0x7fe61c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8d674586ba2335 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:39:36.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:39:36.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe609ffb700 1 -- 172.21.15.70:0/3710110973 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0cd1c0 con 0x557a025075d0 2023-10-18T12:39:36.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe609ffb700 1 -- 172.21.15.70:0/3710110973 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fe61c0cd320 con 0x557a025075d0 2023-10-18T12:39:36.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe61c09e340 con 0x557a025075d0 2023-10-18T12:39:36.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe609ffb700 1 -- 172.21.15.70:0/3710110973 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe61c09a450 con 0x557a025075d0 2023-10-18T12:39:36.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe63c13ab80 10 client.27608 fetch_fsmap learned FSMap version 20 2023-10-18T12:39:36.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.812+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557a022ab250 con 0x557a025075d0 2023-10-18T12:39:36.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe609ffb700 1 -- 172.21.15.70:0/3710110973 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61c09e340 con 0x557a025075d0 2023-10-18T12:39:36.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe609ffb700 1 -- 172.21.15.70:0/3710110973 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe61c0a20f0 con 0x557a025075d0 2023-10-18T12:39:36.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe63c13ab80 10 client.27608 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:39:36.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe63c13ab80 10 client.27608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:39:36.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe63c13ab80 20 client.27608 populate_metadata read hostname 'smithi070' 2023-10-18T12:39:36.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02576c80 0x557a02579070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:39:36.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe63c13ab80 4 client.27608 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:39:36.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a02452e50 con 0x557a02576c80 2023-10-18T12:39:36.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.813+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02576c80 0x557a02579070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:39:36.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.814+0000 7fe625d9a700 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02576c80 0x557a02579070 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98bf3551ffb90ac server_cookie=cf9c4c7ffe570272 in_seq=0 out_seq=0 2023-10-18T12:39:36.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.814+0000 7fe609ffb700 10 client.27608 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe609ffb700 1 -- 172.21.15.70:0/3710110973 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x557a02452e50 con 0x557a02576c80 2023-10-18T12:39:36.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe609ffb700 10 client.27608 handle_command_reply: tid=0 2023-10-18T12:39:36.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 client.27608 shutdown 2023-10-18T12:39:36.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 msgr2=0x557a02516810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02516810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009800 tx=0x7fe61c009790 comp rx=0 tx=0).stop 2023-10-18T12:39:36.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 shutdown_connections 2023-10-18T12:39:36.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x557a02576c80 0x557a02579070 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557a02528250 0x557a02517290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a025075d0 0x557a02516810 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 --2- 172.21.15.70:0/3710110973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557a025114b0 0x557a02516d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 >> 172.21.15.70:0/3710110973 conn(0x557a0247b720 msgr2=0x557a0251c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.817+0000 7fe609ffb700 0 client.27608 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:39:36.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 shutdown_connections 2023-10-18T12:39:36.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe63c13ab80 1 -- 172.21.15.70:0/3710110973 wait complete. 2023-10-18T12:39:36.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe63c13ab80 20 client.27608 trim_cache size 0 max 16384 2023-10-18T12:39:36.837 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:39:36.837 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "scrub active (423 inodes in the stack)", 2023-10-18T12:39:36.837 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": { 2023-10-18T12:39:36.837 INFO:teuthology.orchestra.run.smithi070.stdout: "7767e7d1-a466-439e-ae45-63ceb0299142": { 2023-10-18T12:39:36.837 INFO:teuthology.orchestra.run.smithi070.stdout: "path": "/", 2023-10-18T12:39:36.838 INFO:teuthology.orchestra.run.smithi070.stdout: "tag": "7767e7d1-a466-439e-ae45-63ceb0299142", 2023-10-18T12:39:36.838 INFO:teuthology.orchestra.run.smithi070.stdout: "options": "recursive,force" 2023-10-18T12:39:36.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:39:36.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:39:36.839 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:39:36.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe60c0a15f0 msgr2=0x7fe60c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe6287ff700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe60c0a15f0 0x7fe60c0a3ab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe620108c80 tx=0x7fe610007480 comp rx=0 tx=0).stop 2023-10-18T12:39:36.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 msgr2=0x7fe620107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:39:36.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.818+0000 7fe6287ff700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 0x7fe620107550 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00e9f0 tx=0x7fe61c00ed00 comp rx=0 tx=0).stop 2023-10-18T12:39:36.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 shutdown_connections 2023-10-18T12:39:36.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe620105b30 0x7fe620107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe620067d40 0x7fe620107550 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe60c0a15f0 0x7fe60c0a3ab0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 --2- 172.21.15.70:0/3651580597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe620105170 0x7fe620107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:39:36.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 >> 172.21.15.70:0/3651580597 conn(0x7fe6200767f0 msgr2=0x7fe6200fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:39:36.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 shutdown_connections 2023-10-18T12:39:36.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:39:36.819+0000 7fe6287ff700 1 -- 172.21.15.70:0/3651580597 wait complete. 2023-10-18T12:39:36.843 INFO:tasks.cephfs.filesystem:scrub status for tag:7767e7d1-a466-439e-ae45-63ceb0299142 - {'path': '/', 'tag': '7767e7d1-a466-439e-ae45-63ceb0299142', 'options': 'recursive,force'} 2023-10-18T12:39:37.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:36 smithi070 ceph-mon[98639]: pgmap v736: 129 pgs: 129 active+clean; 599 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 217 op/s 2023-10-18T12:39:37.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:36 smithi152 ceph-mon[134767]: pgmap v736: 129 pgs: 129 active+clean; 599 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 217 op/s 2023-10-18T12:39:37.172 INFO:tasks.workunit.client.0.smithi070.stdout: 10 2559 61.07 MB/sec warmup 5 sec latency 174.404 ms 2023-10-18T12:39:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:36 smithi186 ceph-mon[135253]: pgmap v736: 129 pgs: 129 active+clean; 599 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 217 op/s 2023-10-18T12:39:38.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:37 smithi152 ceph-mon[134767]: scrub queued for path: / 2023-10-18T12:39:38.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:37 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:38.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:37 smithi152 ceph-mon[134767]: scrub summary: active paths [/] 2023-10-18T12:39:38.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 3168 57.95 MB/sec warmup 6 sec latency 46.373 ms 2023-10-18T12:39:38.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:37 smithi186 ceph-mon[135253]: scrub queued for path: / 2023-10-18T12:39:38.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:37 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:38.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:37 smithi186 ceph-mon[135253]: scrub summary: active paths [/] 2023-10-18T12:39:38.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:37 smithi070 ceph-mon[98639]: scrub queued for path: / 2023-10-18T12:39:38.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:37 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:38.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:37 smithi070 ceph-mon[98639]: scrub summary: active paths [/] 2023-10-18T12:39:39.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 3504 54.25 MB/sec warmup 7 sec latency 212.401 ms 2023-10-18T12:39:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:38 smithi186 ceph-mon[135253]: pgmap v737: 129 pgs: 129 active+clean; 600 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-18T12:39:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:38 smithi070 ceph-mon[98639]: pgmap v737: 129 pgs: 129 active+clean; 600 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-18T12:39:39.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:38 smithi152 ceph-mon[134767]: pgmap v737: 129 pgs: 129 active+clean; 600 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 186 op/s 2023-10-18T12:39:40.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 3674 48.23 MB/sec warmup 8 sec latency 180.034 ms 2023-10-18T12:39:40.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:39 smithi186 ceph-mon[135253]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:40.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:39 smithi070 ceph-mon[98639]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:40.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:39 smithi152 ceph-mon[134767]: scrub summary: idle+waiting paths [/] 2023-10-18T12:39:41.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 3850 43.04 MB/sec warmup 9 sec latency 263.576 ms 2023-10-18T12:39:41.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:40 smithi186 ceph-mon[135253]: pgmap v738: 129 pgs: 129 active+clean; 611 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 243 op/s 2023-10-18T12:39:41.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:40 smithi070 ceph-mon[98639]: pgmap v738: 129 pgs: 129 active+clean; 611 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 243 op/s 2023-10-18T12:39:41.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:40 smithi152 ceph-mon[134767]: pgmap v738: 129 pgs: 129 active+clean; 611 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 243 op/s 2023-10-18T12:39:42.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 4072 39.04 MB/sec warmup 10 sec latency 249.095 ms 2023-10-18T12:39:43.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 4557 37.55 MB/sec warmup 11 sec latency 186.339 ms 2023-10-18T12:39:43.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:42 smithi186 ceph-mon[135253]: pgmap v739: 129 pgs: 129 active+clean; 618 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 248 op/s 2023-10-18T12:39:43.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:42 smithi070 ceph-mon[98639]: pgmap v739: 129 pgs: 129 active+clean; 618 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 248 op/s 2023-10-18T12:39:43.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:42 smithi152 ceph-mon[134767]: pgmap v739: 129 pgs: 129 active+clean; 618 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 248 op/s 2023-10-18T12:39:44.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 4991 36.52 MB/sec warmup 12 sec latency 214.290 ms 2023-10-18T12:39:44.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:43 smithi070 ceph-mon[98639]: scrub summary: idle 2023-10-18T12:39:44.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:43 smithi152 ceph-mon[134767]: scrub summary: idle 2023-10-18T12:39:44.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:43 smithi186 ceph-mon[135253]: scrub summary: idle 2023-10-18T12:39:45.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 5694 35.53 MB/sec warmup 13 sec latency 72.730 ms 2023-10-18T12:39:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:44 smithi070 ceph-mon[98639]: pgmap v740: 129 pgs: 129 active+clean; 628 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2023-10-18T12:39:45.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:44 smithi152 ceph-mon[134767]: pgmap v740: 129 pgs: 129 active+clean; 628 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2023-10-18T12:39:45.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:44 smithi186 ceph-mon[135253]: pgmap v740: 129 pgs: 129 active+clean; 628 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2023-10-18T12:39:46.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 6299 37.00 MB/sec warmup 14 sec latency 148.294 ms 2023-10-18T12:39:47.173 INFO:tasks.workunit.client.0.smithi070.stdout: 10 6835 37.32 MB/sec warmup 15 sec latency 165.044 ms 2023-10-18T12:39:47.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:46 smithi070 ceph-mon[98639]: pgmap v741: 129 pgs: 129 active+clean; 631 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 273 op/s 2023-10-18T12:39:47.421 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:46 smithi152 ceph-mon[134767]: pgmap v741: 129 pgs: 129 active+clean; 631 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 273 op/s 2023-10-18T12:39:47.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:46 smithi186 ceph-mon[135253]: pgmap v741: 129 pgs: 129 active+clean; 631 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 273 op/s 2023-10-18T12:39:48.174 INFO:tasks.workunit.client.0.smithi070.stdout: 10 7072 35.66 MB/sec warmup 16 sec latency 156.894 ms 2023-10-18T12:39:49.174 INFO:tasks.workunit.client.0.smithi070.stdout: 10 7215 33.71 MB/sec warmup 17 sec latency 143.157 ms 2023-10-18T12:39:49.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:48 smithi070 ceph-mon[98639]: pgmap v742: 129 pgs: 129 active+clean; 633 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 272 op/s 2023-10-18T12:39:49.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:48 smithi152 ceph-mon[134767]: pgmap v742: 129 pgs: 129 active+clean; 633 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 272 op/s 2023-10-18T12:39:49.469 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:48 smithi186 ceph-mon[135253]: pgmap v742: 129 pgs: 129 active+clean; 633 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 272 op/s 2023-10-18T12:39:50.174 INFO:tasks.workunit.client.0.smithi070.stdout: 10 7383 31.92 MB/sec warmup 18 sec latency 167.207 ms 2023-10-18T12:39:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:50 smithi152 ceph-mon[134767]: pgmap v743: 129 pgs: 129 active+clean; 645 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2023-10-18T12:39:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:50 smithi186 ceph-mon[135253]: pgmap v743: 129 pgs: 129 active+clean; 645 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2023-10-18T12:39:50.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:50 smithi070 ceph-mon[98639]: pgmap v743: 129 pgs: 129 active+clean; 645 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 326 op/s 2023-10-18T12:39:51.174 INFO:tasks.workunit.client.0.smithi070.stdout: 10 7629 30.56 MB/sec warmup 19 sec latency 109.888 ms 2023-10-18T12:39:51.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:51 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:39:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:51 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:39:51.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:51 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:39:52.174 INFO:tasks.workunit.client.0.smithi070.stdout: 10 7984 29.91 MB/sec warmup 20 sec latency 243.123 ms 2023-10-18T12:39:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:52 smithi152 ceph-mon[134767]: pgmap v744: 129 pgs: 129 active+clean; 648 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 275 op/s 2023-10-18T12:39:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:39:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:39:52.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:39:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:52 smithi186 ceph-mon[135253]: pgmap v744: 129 pgs: 129 active+clean; 648 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 275 op/s 2023-10-18T12:39:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:39:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:39:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:39:52.854 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:52 smithi070 ceph-mon[98639]: pgmap v744: 129 pgs: 129 active+clean; 648 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 275 op/s 2023-10-18T12:39:52.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:39:52.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:39:52.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:39:53.174 INFO:tasks.workunit.client.0.smithi070.stdout: 10 8637 29.85 MB/sec warmup 21 sec latency 53.394 ms 2023-10-18T12:39:54.175 INFO:tasks.workunit.client.0.smithi070.stdout: 10 9286 29.91 MB/sec warmup 22 sec latency 176.775 ms 2023-10-18T12:39:55.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:54 smithi152 ceph-mon[134767]: pgmap v745: 129 pgs: 129 active+clean; 659 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 298 op/s 2023-10-18T12:39:55.175 INFO:tasks.workunit.client.0.smithi070.stdout: 10 9473 29.53 MB/sec warmup 23 sec latency 620.432 ms 2023-10-18T12:39:55.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:54 smithi186 ceph-mon[135253]: pgmap v745: 129 pgs: 129 active+clean; 659 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 298 op/s 2023-10-18T12:39:55.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:54 smithi070 ceph-mon[98639]: pgmap v745: 129 pgs: 129 active+clean; 659 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 298 op/s 2023-10-18T12:39:56.175 INFO:tasks.workunit.client.0.smithi070.stdout: 10 10012 30.03 MB/sec warmup 24 sec latency 225.213 ms 2023-10-18T12:39:57.175 INFO:tasks.workunit.client.0.smithi070.stdout: 10 10348 29.88 MB/sec warmup 25 sec latency 224.442 ms 2023-10-18T12:39:57.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:56 smithi186 ceph-mon[135253]: pgmap v746: 129 pgs: 129 active+clean; 684 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 275 op/s 2023-10-18T12:39:57.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:56 smithi070 ceph-mon[98639]: pgmap v746: 129 pgs: 129 active+clean; 684 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 275 op/s 2023-10-18T12:39:57.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:56 smithi152 ceph-mon[134767]: pgmap v746: 129 pgs: 129 active+clean; 684 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 275 op/s 2023-10-18T12:39:58.175 INFO:tasks.workunit.client.0.smithi070.stdout: 10 10549 28.90 MB/sec warmup 26 sec latency 163.902 ms 2023-10-18T12:39:59.176 INFO:tasks.workunit.client.0.smithi070.stdout: 10 10713 27.92 MB/sec warmup 27 sec latency 135.949 ms 2023-10-18T12:39:59.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:39:58 smithi186 ceph-mon[135253]: pgmap v747: 129 pgs: 129 active+clean; 722 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 293 op/s 2023-10-18T12:39:59.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:39:58 smithi070 ceph-mon[98639]: pgmap v747: 129 pgs: 129 active+clean; 722 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 293 op/s 2023-10-18T12:39:59.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:39:58 smithi152 ceph-mon[134767]: pgmap v747: 129 pgs: 129 active+clean; 722 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 293 op/s 2023-10-18T12:40:00.177 INFO:tasks.workunit.client.0.smithi070.stdout: 10 10915 27.18 MB/sec warmup 28 sec latency 286.465 ms 2023-10-18T12:40:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:00 smithi152 ceph-mon[134767]: pgmap v748: 129 pgs: 129 active+clean; 741 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 350 op/s 2023-10-18T12:40:00.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:00 smithi152 ceph-mon[134767]: overall HEALTH_OK 2023-10-18T12:40:00.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:00 smithi186 ceph-mon[135253]: pgmap v748: 129 pgs: 129 active+clean; 741 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 350 op/s 2023-10-18T12:40:00.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:00 smithi186 ceph-mon[135253]: overall HEALTH_OK 2023-10-18T12:40:00.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:00 smithi070 ceph-mon[98639]: pgmap v748: 129 pgs: 129 active+clean; 741 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 350 op/s 2023-10-18T12:40:00.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:00 smithi070 ceph-mon[98639]: overall HEALTH_OK 2023-10-18T12:40:01.177 INFO:tasks.workunit.client.0.smithi070.stdout: 10 10923 26.25 MB/sec warmup 29 sec latency 751.590 ms 2023-10-18T12:40:02.177 INFO:tasks.workunit.client.0.smithi070.stdout: 10 11016 25.48 MB/sec warmup 30 sec latency 1169.856 ms 2023-10-18T12:40:02.586 INFO:tasks.workunit.client.0.smithi070.stdout:[11921] write failed on handle 12399 (Resource temporarily unavailable) 2023-10-18T12:40:02.586 INFO:tasks.workunit.client.0.smithi070.stdout:Child failed with status 1 2023-10-18T12:40:02.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:40:02.660 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-10-18T12:40:02.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-18T12:40:03.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:02 smithi152 ceph-mon[134767]: pgmap v749: 129 pgs: 129 active+clean; 771 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 19 MiB/s wr, 342 op/s 2023-10-18T12:40:03.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:02 smithi186 ceph-mon[135253]: pgmap v749: 129 pgs: 129 active+clean; 771 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 19 MiB/s wr, 342 op/s 2023-10-18T12:40:03.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:02 smithi070 ceph-mon[98639]: pgmap v749: 129 pgs: 129 active+clean; 771 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 19 MiB/s wr, 342 op/s 2023-10-18T12:40:04.109 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7420d7f8e55ceea786dcbf536c338ce920c21eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-10-18T12:40:04.178 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7cf3bacb860a46b0aa8a70058575c397 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7420d7f8e55ceea786dcbf536c338ce920c21eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-10-18T12:40:04.181 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-10-18T12:40:04.193 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-10-18T12:40:04.194 INFO:teuthology.task.exec:Executing custom commands... 2023-10-18T12:40:04.194 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:40:04.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-10-18T12:40:04.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-10-18T12:40:04.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-18T12:40:04.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 -- 172.21.15.70:0/4199989793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 msgr2=0x7ff6b0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:04.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/4199989793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 0x7ff6b0100b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009990 tx=0x7ff69c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 -- 172.21.15.70:0/4199989793 shutdown_connections 2023-10-18T12:40:04.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/4199989793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b0101080 0x7ff6b010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/4199989793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6b01066d0 0x7ff6b0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/4199989793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 0x7ff6b0100b40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 -- 172.21.15.70:0/4199989793 >> 172.21.15.70:0/4199989793 conn(0x7ff6b00fc190 msgr2=0x7ff6b00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:04.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 -- 172.21.15.70:0/4199989793 shutdown_connections 2023-10-18T12:40:04.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.540+0000 7ff6b71ff700 1 -- 172.21.15.70:0/4199989793 wait complete. 2023-10-18T12:40:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.541+0000 7ff6b71ff700 1 Processor -- start 2023-10-18T12:40:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 -- start start 2023-10-18T12:40:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 0x7ff6b019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6b0101080 0x7ff6b019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 0x7ff6b0196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0196fa0 con 0x7ff6b01066d0 2023-10-18T12:40:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0197110 con 0x7ff6b0101080 2023-10-18T12:40:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.542+0000 7ff6b71ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b01972d0 con 0x7ff6b01006c0 2023-10-18T12:40:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 0x7ff6b0196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 0x7ff6b0196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54350/0 (socket says 172.21.15.70:54350) 2023-10-18T12:40:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 -- 172.21.15.70:0/2824969322 learned_addr learned my addr 172.21.15.70:0/2824969322 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b4f9b700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 0x7ff6b019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6affff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6b0101080 0x7ff6b019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 -- 172.21.15.70:0/2824969322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 msgr2=0x7ff6b019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 0x7ff6b019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 -- 172.21.15.70:0/2824969322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6b0101080 msgr2=0x7ff6b019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6b0101080 0x7ff6b019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 -- 172.21.15.70:0/2824969322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0095f0 con 0x7ff6b01066d0 2023-10-18T12:40:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b579c700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 0x7ff6b0196940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800ef30 tx=0x7ff6a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8018070 con 0x7ff6b01066d0 2023-10-18T12:40:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6a800f040 con 0x7ff6b01066d0 2023-10-18T12:40:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b0197550 con 0x7ff6b01066d0 2023-10-18T12:40:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.543+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800cd00 con 0x7ff6b01066d0 2023-10-18T12:40:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.544+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b01a3800 con 0x7ff6b01066d0 2023-10-18T12:40:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.545+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b0108ce0 con 0x7ff6b01066d0 2023-10-18T12:40:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.546+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff6a8003a00 con 0x7ff6b01066d0 2023-10-18T12:40:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.548+0000 7ff6adffb700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff6a00a15f0 0x7ff6a00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.548+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7ff6a80cd090 con 0x7ff6b01066d0 2023-10-18T12:40:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.548+0000 7ff6b4f9b700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff6a00a15f0 0x7ff6a00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.549+0000 7ff6b4f9b700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff6a00a15f0 0x7ff6a00a3ab0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009be0 tx=0x7ff6a4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.552+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6a8096030 con 0x7ff6b01066d0 2023-10-18T12:40:04.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.784+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6b0197e00 con 0x7ff6a00a15f0 2023-10-18T12:40:04.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.794+0000 7ff6adffb700 1 -- 172.21.15.70:0/2824969322 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7ff6b0197e00 con 0x7ff6a00a15f0 2023-10-18T12:40:04.798 INFO:teuthology.orchestra.run.smithi070.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T12:26:26", "first": "2023-10-18T12:27:00", "last": "2023-10-18T12:40:00", "last_pruned": "2023-10-18T12:40:01", "created_count": 14, "pruned_count": 8, "active": true} 2023-10-18T12:40:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.796+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff6a00a15f0 msgr2=0x7ff6a00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.796+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff6a00a15f0 0x7ff6a00a3ab0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009be0 tx=0x7ff6a4009260 comp rx=0 tx=0).stop 2023-10-18T12:40:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.796+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 msgr2=0x7ff6b0196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:04.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.796+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 0x7ff6b0196940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800ef30 tx=0x7ff6a800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 shutdown_connections 2023-10-18T12:40:04.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b01066d0 0x7ff6b0196940 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6b01006c0 0x7ff6b019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7ff6a00a15f0 0x7ff6a00a3ab0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 --2- 172.21.15.70:0/2824969322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6b0101080 0x7ff6b019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:04.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 >> 172.21.15.70:0/2824969322 conn(0x7ff6b00fc190 msgr2=0x7ff6b00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:04.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 shutdown_connections 2023-10-18T12:40:04.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:04.797+0000 7ff6b71ff700 1 -- 172.21.15.70:0/2824969322 wait complete. 2023-10-18T12:40:04.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-18T12:40:05.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 -- 172.21.15.70:0/234197528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01066f0 msgr2=0x7f6eb0106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:05.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/234197528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01066f0 0x7f6eb0106ad0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009990 tx=0x7f6ea0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 -- 172.21.15.70:0/234197528 shutdown_connections 2023-10-18T12:40:05.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/234197528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 0x7f6eb010d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/234197528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01066f0 0x7f6eb0106ad0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/234197528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb01006e0 0x7f6eb0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 -- 172.21.15.70:0/234197528 >> 172.21.15.70:0/234197528 conn(0x7f6eb00fc1f0 msgr2=0x7f6eb00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 -- 172.21.15.70:0/234197528 shutdown_connections 2023-10-18T12:40:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.011+0000 7f6eb49ff700 1 -- 172.21.15.70:0/234197528 wait complete. 2023-10-18T12:40:05.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 Processor -- start 2023-10-18T12:40:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 -- start start 2023-10-18T12:40:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 0x7f6eb019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 0x7f6eb019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:05.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb01066f0 0x7f6eb01969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:05.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0197010 con 0x7f6eb01006e0 2023-10-18T12:40:05.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0197180 con 0x7f6eb01010a0 2023-10-18T12:40:05.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.012+0000 7f6eb49ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0197340 con 0x7f6eb01066f0 2023-10-18T12:40:05.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 0x7f6eb019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:05.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 0x7f6eb019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54364/0 (socket says 172.21.15.70:54364) 2023-10-18T12:40:05.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 -- 172.21.15.70:0/1433717073 learned_addr learned my addr 172.21.15.70:0/1433717073 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eaed9d700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb01066f0 0x7f6eb01969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eadd9b700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 0x7f6eb019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 -- 172.21.15.70:0/1433717073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb01066f0 msgr2=0x7f6eb01969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb01066f0 0x7f6eb01969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 -- 172.21.15.70:0/1433717073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 msgr2=0x7f6eb019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 0x7f6eb019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 -- 172.21.15.70:0/1433717073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea00095f0 con 0x7f6eb01006e0 2023-10-18T12:40:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eadd9b700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 0x7f6eb019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:40:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.013+0000 7f6eae59c700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 0x7f6eb019c880 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005910 tx=0x7f6ea0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.014+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea001d070 con 0x7f6eb01006e0 2023-10-18T12:40:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.014+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea0022470 con 0x7f6eb01006e0 2023-10-18T12:40:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.014+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0017650 con 0x7f6eb01006e0 2023-10-18T12:40:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.014+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb01975c0 con 0x7f6eb01006e0 2023-10-18T12:40:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.014+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01a3870 con 0x7f6eb01006e0 2023-10-18T12:40:05.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.015+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb0108ce0 con 0x7f6eb01006e0 2023-10-18T12:40:05.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.017+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6ea00225e0 con 0x7f6eb01006e0 2023-10-18T12:40:05.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.019+0000 7f6e9f7fe700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e980a15f0 0x7f6e980a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:05.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.019+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f6ea00ce840 con 0x7f6eb01006e0 2023-10-18T12:40:05.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.019+0000 7f6eadd9b700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e980a15f0 0x7f6e980a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:05.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.020+0000 7f6eadd9b700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e980a15f0 0x7f6e980a3ab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0197b70 tx=0x7f6ea4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:05.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.023+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6ea00977e0 con 0x7f6eb01006e0 2023-10-18T12:40:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:04 smithi070 ceph-mon[98639]: pgmap v750: 129 pgs: 129 active+clean; 810 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 429 op/s 2023-10-18T12:40:05.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:04 smithi070 ceph-mon[98639]: osdmap e108: 12 total, 12 up, 12 in 2023-10-18T12:40:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:04 smithi186 ceph-mon[135253]: pgmap v750: 129 pgs: 129 active+clean; 810 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 429 op/s 2023-10-18T12:40:05.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:04 smithi186 ceph-mon[135253]: osdmap e108: 12 total, 12 up, 12 in 2023-10-18T12:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.247+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 --> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb0197e70 con 0x7f6e980a15f0 2023-10-18T12:40:05.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.257+0000 7f6e9f7fe700 1 -- 172.21.15.70:0/1433717073 <== mgr.14214 v2:172.21.15.70:6800/223035468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6eb0197e70 con 0x7f6e980a15f0 2023-10-18T12:40:05.260 INFO:teuthology.orchestra.run.smithi070.stdout:/ 1m 6m3h 2023-10-18T12:40:05.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e980a15f0 msgr2=0x7f6e980a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:05.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e980a15f0 0x7f6e980a3ab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0197b70 tx=0x7f6ea4009260 comp rx=0 tx=0).stop 2023-10-18T12:40:05.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 msgr2=0x7f6eb019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:05.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 0x7f6eb019c880 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005910 tx=0x7f6ea0004730 comp rx=0 tx=0).stop 2023-10-18T12:40:05.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 shutdown_connections 2023-10-18T12:40:05.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eb01066f0 0x7f6eb01969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6eb01006e0 0x7f6eb019c880 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f6e980a15f0 0x7f6e980a3ab0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 --2- 172.21.15.70:0/1433717073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb01010a0 0x7f6eb019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 >> 172.21.15.70:0/1433717073 conn(0x7f6eb00fc1f0 msgr2=0x7f6eb00fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 shutdown_connections 2023-10-18T12:40:05.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:05.259+0000 7f6eb49ff700 1 -- 172.21.15.70:0/1433717073 wait complete. 2023-10-18T12:40:05.281 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-18T12:40:05.292 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-18T12:40:05.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:04 smithi152 ceph-mon[134767]: pgmap v750: 129 pgs: 129 active+clean; 810 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 429 op/s 2023-10-18T12:40:05.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:04 smithi152 ceph-mon[134767]: osdmap e108: 12 total, 12 up, 12 in 2023-10-18T12:40:06.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:05 smithi186 ceph-mon[135253]: from='client.17556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:40:06.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:05 smithi186 ceph-mon[135253]: from='client.17562 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:40:06.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:05 smithi070 ceph-mon[98639]: from='client.17556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:40:06.356 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:05 smithi070 ceph-mon[98639]: from='client.17562 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:40:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:05 smithi152 ceph-mon[134767]: from='client.17556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:40:06.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:05 smithi152 ceph-mon[134767]: from='client.17562 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T12:40:06.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T12:40:07.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 -- 172.21.15.70:0/1508420033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 msgr2=0x7f32d010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 --2- 172.21.15.70:0/1508420033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d010d5f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b0a0 tx=0x7f32c800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 -- 172.21.15.70:0/1508420033 shutdown_connections 2023-10-18T12:40:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 --2- 172.21.15.70:0/1508420033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d010d5f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 --2- 172.21.15.70:0/1508420033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0106730 0x7f32d0106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 --2- 172.21.15.70:0/1508420033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32d0100720 0x7f32d0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.068+0000 7f32d7fff700 1 -- 172.21.15.70:0/1508420033 >> 172.21.15.70:0/1508420033 conn(0x7f32d00fc1b0 msgr2=0x7f32d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.069+0000 7f32d7fff700 1 -- 172.21.15.70:0/1508420033 shutdown_connections 2023-10-18T12:40:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.069+0000 7f32d7fff700 1 -- 172.21.15.70:0/1508420033 wait complete. 2023-10-18T12:40:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.070+0000 7f32d7fff700 1 Processor -- start 2023-10-18T12:40:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.070+0000 7f32d7fff700 1 -- start start 2023-10-18T12:40:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 0x7f32d0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32d0106730 0x7f32d019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d7fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d019dc90 con 0x7f32d01010e0 2023-10-18T12:40:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d019de00 con 0x7f32d0106730 2023-10-18T12:40:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d7fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d019dfc0 con 0x7f32d0100720 2023-10-18T12:40:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 0x7f32d0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d559a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 0x7f32d0198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:51926/0 (socket says 172.21.15.70:51926) 2023-10-18T12:40:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32d0106730 0x7f32d019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 -- 172.21.15.70:0/3283925585 learned_addr learned my addr 172.21.15.70:0/3283925585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 -- 172.21.15.70:0/3283925585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32d0106730 msgr2=0x7f32d019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32d0106730 0x7f32d019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 -- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 msgr2=0x7f32d01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d5d9b700 1 -- 172.21.15.70:0/3283925585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8009d00 con 0x7f32d0100720 2023-10-18T12:40:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.071+0000 7f32d559a700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.072+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 0x7f32d0198d70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00e9a0 tx=0x7f32cc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.072+0000 7f32c6ffd700 1 -- 172.21.15.70:0/3283925585 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc009790 con 0x7f32d0100720 2023-10-18T12:40:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.072+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d019e2a0 con 0x7f32d0100720 2023-10-18T12:40:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.072+0000 7f32c6ffd700 1 -- 172.21.15.70:0/3283925585 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32cc010430 con 0x7f32d0100720 2023-10-18T12:40:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.072+0000 7f32c6ffd700 1 -- 172.21.15.70:0/3283925585 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc018680 con 0x7f32d0100720 2023-10-18T12:40:07.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.072+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01a5ae0 con 0x7f32d0100720 2023-10-18T12:40:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.075+0000 7f32c6ffd700 1 -- 172.21.15.70:0/3283925585 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f32cc0105a0 con 0x7f32d0100720 2023-10-18T12:40:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.076+0000 7f32eb8beb80 1 Processor -- start 2023-10-18T12:40:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.076+0000 7f32eb8beb80 1 -- start start 2023-10-18T12:40:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32c6ffd700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f32bc0a15f0 0x7f32bc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41305090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41305a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c412786e0 con 0x564c413055d0 2023-10-18T12:40:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4125ec60 con 0x564c4130abe0 2023-10-18T12:40:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4125c580 con 0x564c4130b500 2023-10-18T12:40:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32c6ffd700 1 -- 172.21.15.70:0/3283925585 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f32cc014070 con 0x7f32d0100720 2023-10-18T12:40:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32d559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41305090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32d559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41305090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:51942/0 (socket says 172.21.15.70:51942) 2023-10-18T12:40:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32d559a700 1 -- 172.21.15.70:0/4066080765 learned_addr learned my addr 172.21.15.70:0/4066080765 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32d659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41305a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32d659c700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f32bc0a15f0 0x7f32bc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.077+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d559a700 1 -- 172.21.15.70:0/4066080765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 msgr2=0x564c4130afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d559a700 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d559a700 1 -- 172.21.15.70:0/4066080765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 msgr2=0x564c41305a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d559a700 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41305a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d559a700 1 -- 172.21.15.70:0/4066080765 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c412153f0 con 0x564c4130b500 2023-10-18T12:40:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d659c700 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41305a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:40:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d659c700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f32bc0a15f0 0x7f32bc0a3ab0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b780 tx=0x7f32c8006230 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32d559a700 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41305090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32c00115f0 tx=0x7f32c0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf82056a045e33fb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32bbfff700 1 -- 172.21.15.70:0/4066080765 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004230 con 0x564c4130b500 2023-10-18T12:40:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32bbfff700 1 -- 172.21.15.70:0/4066080765 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32c0004390 con 0x564c4130b500 2023-10-18T12:40:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.078+0000 7f32bbfff700 1 -- 172.21.15.70:0/4066080765 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00198c0 con 0x564c4130b500 2023-10-18T12:40:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 -- 172.21.15.70:0/4066080765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 msgr2=0x564c41305090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41305090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32c00115f0 tx=0x7f32c0011900 comp rx=0 tx=0).stop 2023-10-18T12:40:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 -- 172.21.15.70:0/4066080765 shutdown_connections 2023-10-18T12:40:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41305a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/4066080765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41305090 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 -- 172.21.15.70:0/4066080765 >> 172.21.15.70:0/4066080765 conn(0x564c41279720 msgr2=0x564c412ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 -- 172.21.15.70:0/4066080765 shutdown_connections 2023-10-18T12:40:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.079+0000 7f32eb8beb80 1 -- 172.21.15.70:0/4066080765 wait complete. 2023-10-18T12:40:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.080+0000 7f32eb8beb80 1 -- start start 2023-10-18T12:40:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.080+0000 7f32eb8beb80 1 Processor -- start 2023-10-18T12:40:07.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41311470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c4130ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c41257e70 con 0x564c4130b500 2023-10-18T12:40:07.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c411f4710 con 0x564c4130abe0 2023-10-18T12:40:07.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c411f43b0 con 0x564c413055d0 2023-10-18T12:40:07.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32eb8beb80 10 client.0 _ll_register_callbacks cb 0x564c4113ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:40:07.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32d659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c4130ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41311470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32d659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c4130ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54390/0 (socket says 172.21.15.70:54390) 2023-10-18T12:40:07.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32d659c700 1 -- 172.21.15.70:0/3095787339 learned_addr learned my addr 172.21.15.70:0/3095787339 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.081+0000 7f32d559a700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d659c700 1 -- 172.21.15.70:0/3095787339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 msgr2=0x564c41311470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d659c700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41311470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d659c700 1 -- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 msgr2=0x564c4130e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d659c700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d659c700 1 -- 172.21.15.70:0/3095787339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01a5ae0 con 0x564c4130b500 2023-10-18T12:40:07.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41311470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:07.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32d659c700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c4130ed00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32c8015000 tx=0x7f32c800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05cc803952ef151 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:40:07.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32b97fa700 1 -- 172.21.15.70:0/3095787339 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8027070 con 0x564c4130b500 2023-10-18T12:40:07.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c4125cef0 con 0x564c4130b500 2023-10-18T12:40:07.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32b97fa700 1 -- 172.21.15.70:0/3095787339 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f32c8003d50 con 0x564c4130b500 2023-10-18T12:40:07.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.082+0000 7f32b97fa700 1 -- 172.21.15.70:0/3095787339 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c800e040 con 0x564c4130b500 2023-10-18T12:40:07.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 10 client.17574 fetch_fsmap learned FSMap version 20 2023-10-18T12:40:07.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f32c0009d50 con 0x564c4130b500 2023-10-18T12:40:07.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32b97fa700 1 -- 172.21.15.70:0/3095787339 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c4125cef0 con 0x564c4130b500 2023-10-18T12:40:07.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32b97fa700 1 -- 172.21.15.70:0/3095787339 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32c80214f0 con 0x564c4130b500 2023-10-18T12:40:07.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 10 client.17574 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:40:07.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:40:07.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 20 client.17574 populate_metadata read hostname 'smithi070' 2023-10-18T12:40:07.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c4131edd0 0x564c413211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 4 client.17574 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:40:07.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c412d48b0 con 0x564c4131edd0 2023-10-18T12:40:07.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.083+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c4131edd0 0x564c413211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.084+0000 7f32b97fa700 10 client.17574 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.084+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c4131edd0 0x564c413211f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb934a0fd7b5f85 server_cookie=44afad96c09d7e0e in_seq=0 out_seq=0 2023-10-18T12:40:07.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32b97fa700 1 -- 172.21.15.70:0/3095787339 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c412d48b0 con 0x564c4131edd0 2023-10-18T12:40:07.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32b97fa700 10 client.17574 handle_command_reply: tid=0 2023-10-18T12:40:07.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 client.17574 shutdown 2023-10-18T12:40:07.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 msgr2=0x564c4130ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c4130ed00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32c8015000 tx=0x7f32c800b3b0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 shutdown_connections 2023-10-18T12:40:07.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c4131edd0 0x564c413211f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c4130ed00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41311470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/3095787339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4130e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32b97fa700 0 client.17574 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 >> 172.21.15.70:0/3095787339 conn(0x564c41279720 msgr2=0x564c412ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 shutdown_connections 2023-10-18T12:40:07.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 1 -- 172.21.15.70:0/3095787339 wait complete. 2023-10-18T12:40:07.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.085+0000 7f32eb8beb80 20 client.17574 trim_cache size 0 max 16384 2023-10-18T12:40:07.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.102+0000 7f32eb8beb80 1 Processor -- start 2023-10-18T12:40:07.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.102+0000 7f32eb8beb80 1 -- start start 2023-10-18T12:40:07.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41305090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4131ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c41310fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c41300130 con 0x564c4130b500 2023-10-18T12:40:07.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c410d8920 con 0x564c4130abe0 2023-10-18T12:40:07.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4130ccb0 con 0x564c413055d0 2023-10-18T12:40:07.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41305090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c41310fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41305090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:51958/0 (socket says 172.21.15.70:51958) 2023-10-18T12:40:07.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d5d9b700 1 -- 172.21.15.70:0/2622483227 learned_addr learned my addr 172.21.15.70:0/2622483227 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d559a700 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4131ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d5d9b700 1 -- 172.21.15.70:0/2622483227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 msgr2=0x564c4131ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.103+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4131ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32d5d9b700 1 -- 172.21.15.70:0/2622483227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 msgr2=0x564c41310fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c41310fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32d5d9b700 1 -- 172.21.15.70:0/2622483227 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c41082de0 con 0x564c413055d0 2023-10-18T12:40:07.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32d659c700 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c41310fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:07.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41305090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32cc09a1d0 tx=0x7f32cc010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57b29933be3ea9b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32c4ff9700 1 -- 172.21.15.70:0/2622483227 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc09e170 con 0x564c413055d0 2023-10-18T12:40:07.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32c4ff9700 1 -- 172.21.15.70:0/2622483227 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32cc09e390 con 0x564c413055d0 2023-10-18T12:40:07.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.104+0000 7f32c4ff9700 1 -- 172.21.15.70:0/2622483227 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc09a700 con 0x564c413055d0 2023-10-18T12:40:07.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 -- 172.21.15.70:0/2622483227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 msgr2=0x564c41305090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41305090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32cc09a1d0 tx=0x7f32cc010c80 comp rx=0 tx=0).stop 2023-10-18T12:40:07.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 -- 172.21.15.70:0/2622483227 shutdown_connections 2023-10-18T12:40:07.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c4130b500 0x564c41310fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c413055d0 0x564c41305090 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/2622483227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c4131ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 -- 172.21.15.70:0/2622483227 >> 172.21.15.70:0/2622483227 conn(0x564c41279720 msgr2=0x564c4131a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 -- 172.21.15.70:0/2622483227 shutdown_connections 2023-10-18T12:40:07.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 -- 172.21.15.70:0/2622483227 wait complete. 2023-10-18T12:40:07.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 -- start start 2023-10-18T12:40:07.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.105+0000 7f32eb8beb80 1 Processor -- start 2023-10-18T12:40:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41316100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c41306c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41307370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c41307a80 con 0x564c413055d0 2023-10-18T12:40:07.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c41372c20 con 0x564c4130abe0 2023-10-18T12:40:07.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c41372d60 con 0x564c4130b500 2023-10-18T12:40:07.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32eb8beb80 10 client.0 _ll_register_callbacks cb 0x564c4113ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:40:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41316100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d559a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c41306c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41316100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54414/0 (socket says 172.21.15.70:54414) 2023-10-18T12:40:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 -- 172.21.15.70:0/760908506 learned_addr learned my addr 172.21.15.70:0/760908506 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d659c700 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41307370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 -- 172.21.15.70:0/760908506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 msgr2=0x564c41307370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41307370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 -- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 msgr2=0x564c41306c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c41306c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.106+0000 7f32d5d9b700 1 -- 172.21.15.70:0/760908506 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c0009d50 con 0x564c413055d0 2023-10-18T12:40:07.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32d5d9b700 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41316100 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f32cc099f40 tx=0x7f32cc0966a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff5bbe13c2f85c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:40:07.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32b97fa700 1 -- 172.21.15.70:0/760908506 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc098380 con 0x564c413055d0 2023-10-18T12:40:07.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32b97fa700 1 -- 172.21.15.70:0/760908506 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f32cc0a25d0 con 0x564c413055d0 2023-10-18T12:40:07.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c0004390 con 0x564c413055d0 2023-10-18T12:40:07.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32b97fa700 1 -- 172.21.15.70:0/760908506 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32cc08a070 con 0x564c413055d0 2023-10-18T12:40:07.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 10 client.17580 fetch_fsmap learned FSMap version 20 2023-10-18T12:40:07.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f32d019e2a0 con 0x564c413055d0 2023-10-18T12:40:07.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32b97fa700 1 -- 172.21.15.70:0/760908506 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32c0004390 con 0x564c413055d0 2023-10-18T12:40:07.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32b97fa700 1 -- 172.21.15.70:0/760908506 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32cc09a4a0 con 0x564c413055d0 2023-10-18T12:40:07.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 10 client.17580 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:40:07.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:40:07.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 20 client.17580 populate_metadata read hostname 'smithi070' 2023-10-18T12:40:07.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c41374030 0x564c41376450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 4 client.17580 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T12:40:07.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.107+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c41250e50 con 0x564c41374030 2023-10-18T12:40:07.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.108+0000 7f32d559a700 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c41374030 0x564c41376450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.108+0000 7f32b97fa700 10 client.17580 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.108+0000 7f32d559a700 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c41374030 0x564c41376450 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafe4466ca8399d6 server_cookie=bd766e5c5eb80bc5 in_seq=0 out_seq=0 2023-10-18T12:40:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.108+0000 7f32b97fa700 1 -- 172.21.15.70:0/760908506 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564c41250e50 con 0x564c41374030 2023-10-18T12:40:07.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.108+0000 7f32b97fa700 10 client.17580 handle_command_reply: tid=0 2023-10-18T12:40:07.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.108+0000 7f32eb8beb80 1 client.17580 shutdown 2023-10-18T12:40:07.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 msgr2=0x564c41316100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41316100 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f32cc099f40 tx=0x7f32cc0966a0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 shutdown_connections 2023-10-18T12:40:07.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x564c41374030 0x564c41376450 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c4130b500 0x564c41307370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564c413055d0 0x564c41316100 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 --2- 172.21.15.70:0/760908506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c4130abe0 0x564c41306c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32b97fa700 0 client.17580 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 >> 172.21.15.70:0/760908506 conn(0x564c41279720 msgr2=0x564c412f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 shutdown_connections 2023-10-18T12:40:07.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 1 -- 172.21.15.70:0/760908506 wait complete. 2023-10-18T12:40:07.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32eb8beb80 20 client.17580 trim_cache size 0 max 16384 2023-10-18T12:40:07.145 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:40:07.145 INFO:teuthology.orchestra.run.smithi070.stdout: "status": "no active scrubs running", 2023-10-18T12:40:07.145 INFO:teuthology.orchestra.run.smithi070.stdout: "scrubs": {} 2023-10-18T12:40:07.146 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:40:07.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f32bc0a15f0 msgr2=0x7f32bc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32d7fff700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f32bc0a15f0 0x7f32bc0a3ab0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b780 tx=0x7f32c8006230 comp rx=0 tx=0).stop 2023-10-18T12:40:07.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 msgr2=0x7f32d0198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.109+0000 7f32d7fff700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 0x7f32d0198d70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00e9a0 tx=0x7f32cc00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 shutdown_connections 2023-10-18T12:40:07.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f32bc0a15f0 0x7f32bc0a3ab0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32d0106730 0x7f32d019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d0100720 0x7f32d0198d70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 --2- 172.21.15.70:0/3283925585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32d01010e0 0x7f32d01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 >> 172.21.15.70:0/3283925585 conn(0x7f32d00fc1b0 msgr2=0x7f32d010be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 shutdown_connections 2023-10-18T12:40:07.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.110+0000 7f32d7fff700 1 -- 172.21.15.70:0/3283925585 wait complete. 2023-10-18T12:40:07.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T12:40:07.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:40:07.211 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:06 smithi070 ceph-mon[98639]: pgmap v752: 129 pgs: 129 active+clean; 817 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 29 MiB/s wr, 440 op/s 2023-10-18T12:40:07.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:06 smithi186 ceph-mon[135253]: pgmap v752: 129 pgs: 129 active+clean; 817 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 29 MiB/s wr, 440 op/s 2023-10-18T12:40:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 -- 172.21.15.70:0/3345984965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 msgr2=0x7f2fd0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/3345984965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd0100b20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009990 tx=0x7f2fb8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 -- 172.21.15.70:0/3345984965 shutdown_connections 2023-10-18T12:40:07.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/3345984965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd0101060 0x7f2fd010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/3345984965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fd01066b0 0x7f2fd0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/3345984965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd0100b20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 -- 172.21.15.70:0/3345984965 >> 172.21.15.70:0/3345984965 conn(0x7f2fd00fc170 msgr2=0x7f2fd00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 -- 172.21.15.70:0/3345984965 shutdown_connections 2023-10-18T12:40:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.346+0000 7f2fd63ff700 1 -- 172.21.15.70:0/3345984965 wait complete. 2023-10-18T12:40:07.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 Processor -- start 2023-10-18T12:40:07.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 -- start start 2023-10-18T12:40:07.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd0101060 0x7f2fd019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fd01066b0 0x7f2fd0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0199190 con 0x7f2fd01006a0 2023-10-18T12:40:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0199300 con 0x7f2fd01066b0 2023-10-18T12:40:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fd63ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01994c0 con 0x7f2fd0101060 2023-10-18T12:40:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.347+0000 7f2fcffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54430/0 (socket says 172.21.15.70:54430) 2023-10-18T12:40:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 -- 172.21.15.70:0/4209255785 learned_addr learned my addr 172.21.15.70:0/4209255785 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fc7fff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd0101060 0x7f2fd019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fd499c700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fd01066b0 0x7f2fd0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 -- 172.21.15.70:0/4209255785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd0101060 msgr2=0x7f2fd019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd0101060 0x7f2fd019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 -- 172.21.15.70:0/4209255785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fd01066b0 msgr2=0x7f2fd0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fd01066b0 0x7f2fd0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 -- 172.21.15.70:0/4209255785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb80095f0 con 0x7f2fd01006a0 2023-10-18T12:40:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcffff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd019e9a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e9a0 tx=0x7f2fc000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0009790 con 0x7f2fd01006a0 2023-10-18T12:40:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc000cbd0 con 0x7f2fd01006a0 2023-10-18T12:40:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd01997a0 con 0x7f2fd01006a0 2023-10-18T12:40:07.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.348+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0018680 con 0x7f2fd01006a0 2023-10-18T12:40:07.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.349+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01a5a40 con 0x7f2fd01006a0 2023-10-18T12:40:07.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.350+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd0101de0 con 0x7f2fd01006a0 2023-10-18T12:40:07.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.352+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2fc000cd40 con 0x7f2fd01006a0 2023-10-18T12:40:07.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.354+0000 7f2fcdffb700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.354+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f2fc0014070 con 0x7f2fd01006a0 2023-10-18T12:40:07.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.354+0000 7f2fc7fff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.354+0000 7f2fc7fff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b2e0 tx=0x7f2fb8005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.358+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2fc0052100 con 0x7f2fd01006a0 2023-10-18T12:40:07.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:06 smithi152 ceph-mon[134767]: pgmap v752: 129 pgs: 129 active+clean; 817 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 29 MiB/s wr, 440 op/s 2023-10-18T12:40:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.602+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd0067530 con 0x7f2fd01006a0 2023-10-18T12:40:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.602+0000 7f2fcdffb700 1 -- 172.21.15.70:0/4209255785 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f2fc009a140 con 0x7f2fd01006a0 2023-10-18T12:40:07.606 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:07.606 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:40:07.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2fbc0a1670 msgr2=0x7f2fbc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b2e0 tx=0x7f2fb8005fd0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 msgr2=0x7f2fd019e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd019e9a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e9a0 tx=0x7f2fc000ed60 comp rx=0 tx=0).stop 2023-10-18T12:40:07.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 shutdown_connections 2023-10-18T12:40:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fd01066b0 0x7f2fd0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd01006a0 0x7f2fd019e9a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 --2- 172.21.15.70:0/4209255785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd0101060 0x7f2fd019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 >> 172.21.15.70:0/4209255785 conn(0x7f2fd00fc170 msgr2=0x7f2fd010bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 shutdown_connections 2023-10-18T12:40:07.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.605+0000 7f2fd63ff700 1 -- 172.21.15.70:0/4209255785 wait complete. 2023-10-18T12:40:07.613 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:40:07.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T12:40:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 -- 172.21.15.70:0/1693592486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4073540 msgr2=0x7f83a410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/1693592486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4073540 0x7f83a410ca10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8394009990 tx=0x7f8394009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 -- 172.21.15.70:0/1693592486 shutdown_connections 2023-10-18T12:40:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/1693592486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a410cf50 0x7f83a410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/1693592486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4072c20 0x7f83a4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/1693592486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4073540 0x7f83a410ca10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 -- 172.21.15.70:0/1693592486 >> 172.21.15.70:0/1693592486 conn(0x7f83a40fc200 msgr2=0x7f83a40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 -- 172.21.15.70:0/1693592486 shutdown_connections 2023-10-18T12:40:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.825+0000 7f83ac1ff700 1 -- 172.21.15.70:0/1693592486 wait complete. 2023-10-18T12:40:07.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 Processor -- start 2023-10-18T12:40:07.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 -- start start 2023-10-18T12:40:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 0x7f83a4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 0x7f83a4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a410cf50 0x7f83a419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a419dd10 con 0x7f83a4072c20 2023-10-18T12:40:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a419de80 con 0x7f83a4073540 2023-10-18T12:40:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83ac1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a419e040 con 0x7f83a410cf50 2023-10-18T12:40:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 0x7f83a4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83a979a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 0x7f83a4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83a979a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 0x7f83a4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34302/0 (socket says 172.21.15.70:34302) 2023-10-18T12:40:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.826+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 0x7f83a4198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43340/0 (socket says 172.21.15.70:43340) 2023-10-18T12:40:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a979a700 1 -- 172.21.15.70:0/5784039 learned_addr learned my addr 172.21.15.70:0/5784039 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83aa79c700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a410cf50 0x7f83a419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a9f9b700 1 -- 172.21.15.70:0/5784039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a410cf50 msgr2=0x7f83a419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a410cf50 0x7f83a419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a9f9b700 1 -- 172.21.15.70:0/5784039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 msgr2=0x7f83a4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 0x7f83a4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a9f9b700 1 -- 172.21.15.70:0/5784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83940095f0 con 0x7f83a4072c20 2023-10-18T12:40:07.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a979a700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 0x7f83a4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:07.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 0x7f83a4198df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f83a000d750 tx=0x7f83a000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f839affd700 1 -- 172.21.15.70:0/5784039 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0004020 con 0x7f83a4072c20 2023-10-18T12:40:07.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f839affd700 1 -- 172.21.15.70:0/5784039 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a0004830 con 0x7f83a4072c20 2023-10-18T12:40:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a419e320 con 0x7f83a4072c20 2023-10-18T12:40:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f839affd700 1 -- 172.21.15.70:0/5784039 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00054c0 con 0x7f83a4072c20 2023-10-18T12:40:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.827+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a41a5b60 con 0x7f83a4072c20 2023-10-18T12:40:07.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.831+0000 7f839affd700 1 -- 172.21.15.70:0/5784039 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f83a00049a0 con 0x7f83a4072c20 2023-10-18T12:40:07.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 Processor -- start 2023-10-18T12:40:07.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f839affd700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83900a15f0 0x7f83900a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 -- start start 2023-10-18T12:40:07.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f839affd700 1 -- 172.21.15.70:0/5784039 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f83a00cce00 con 0x7f83a4072c20 2023-10-18T12:40:07.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9f78090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f78a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9eeb6e0 con 0x561ef9f7dbe0 2023-10-18T12:40:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9ed1c60 con 0x561ef9f785d0 2023-10-18T12:40:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9ecf580 con 0x561ef9f7e500 2023-10-18T12:40:07.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.832+0000 7f83aa79c700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83900a15f0 0x7f83900a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f7dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43342/0 (socket says 172.21.15.70:43342) 2023-10-18T12:40:07.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 -- 172.21.15.70:0/3336062677 learned_addr learned my addr 172.21.15.70:0/3336062677 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a979a700 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9f78090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83aa79c700 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f78a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 -- 172.21.15.70:0/3336062677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 msgr2=0x561ef9f78090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9f78090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 -- 172.21.15.70:0/3336062677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 msgr2=0x561ef9f78a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f78a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 -- 172.21.15.70:0/3336062677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef9e883f0 con 0x561ef9f7dbe0 2023-10-18T12:40:07.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83aa79c700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83900a15f0 0x7f83900a3ab0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f839c00f870 tx=0x7f839c00fc30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.833+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f7dfc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f83a00a1830 tx=0x7f83a009a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f68cc1b94e5a6c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f838ffff700 1 -- 172.21.15.70:0/3336062677 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a009db20 con 0x561ef9f7dbe0 2023-10-18T12:40:07.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f838ffff700 1 -- 172.21.15.70:0/3336062677 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a00a1d60 con 0x561ef9f7dbe0 2023-10-18T12:40:07.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f838ffff700 1 -- 172.21.15.70:0/3336062677 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0098ec0 con 0x561ef9f7dbe0 2023-10-18T12:40:07.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/3336062677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 msgr2=0x561ef9f7dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f7dfc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f83a00a1830 tx=0x7f83a009a150 comp rx=0 tx=0).stop 2023-10-18T12:40:07.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/3336062677 shutdown_connections 2023-10-18T12:40:07.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f78a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f7dfc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/3336062677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9f78090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.834+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/3336062677 >> 172.21.15.70:0/3336062677 conn(0x561ef9eec720 msgr2=0x561ef9f72b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.835+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/3336062677 shutdown_connections 2023-10-18T12:40:07.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.835+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/3336062677 wait complete. 2023-10-18T12:40:07.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.835+0000 7f83bfbf1b80 1 -- start start 2023-10-18T12:40:07.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.835+0000 7f83bfbf1b80 1 Processor -- start 2023-10-18T12:40:07.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f785d0 0x561ef9f81870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 0x561ef9f822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9ecae70 con 0x561ef9f7e500 2023-10-18T12:40:07.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9e67710 con 0x561ef9f7dbe0 2023-10-18T12:40:07.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9e673b0 con 0x561ef9f785d0 2023-10-18T12:40:07.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83bfbf1b80 10 client.0 _ll_register_callbacks cb 0x561ef9db1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:40:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 0x561ef9f822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83a979a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83a979a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34320/0 (socket says 172.21.15.70:34320) 2023-10-18T12:40:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 0x561ef9f822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43356/0 (socket says 172.21.15.70:43356) 2023-10-18T12:40:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 -- 172.21.15.70:0/1695321126 learned_addr learned my addr 172.21.15.70:0/1695321126 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f785d0 0x561ef9f81870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 -- 172.21.15.70:0/1695321126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f785d0 msgr2=0x561ef9f81870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f785d0 0x561ef9f81870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 -- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7dbe0 msgr2=0x561ef9f81db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.836+0000 7f83aa79c700 1 -- 172.21.15.70:0/1695321126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a41a5b60 con 0x561ef9f7e500 2023-10-18T12:40:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f83aa79c700 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 0x561ef9f822f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f839c018b00 tx=0x7f839c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa2598be6c23c18 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f838d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-18T12:40:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f838d7fa700 1 -- 172.21.15.70:0/1695321126 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c00e040 con 0x561ef9f7e500 2023-10-18T12:40:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ef9ecfef0 con 0x561ef9f7e500 2023-10-18T12:40:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f838d7fa700 1 -- 172.21.15.70:0/1695321126 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f839c008920 con 0x561ef9f7e500 2023-10-18T12:40:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f838d7fa700 1 -- 172.21.15.70:0/1695321126 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f839c008ae0 con 0x561ef9f7e500 2023-10-18T12:40:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f83bfbf1b80 10 client.17598 fetch_fsmap learned FSMap version 20 2023-10-18T12:40:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x561ef9cf5de0 con 0x561ef9f7e500 2023-10-18T12:40:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.837+0000 7f838d7fa700 1 -- 172.21.15.70:0/1695321126 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ef9ecfef0 con 0x561ef9f7e500 2023-10-18T12:40:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f838d7fa700 1 -- 172.21.15.70:0/1695321126 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f839c024c20 con 0x561ef9f7e500 2023-10-18T12:40:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83bfbf1b80 10 client.17598 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:40:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83bfbf1b80 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:40:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83bfbf1b80 20 client.17598 populate_metadata read hostname 'smithi070' 2023-10-18T12:40:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9f91f30 0x561ef9f94350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83bfbf1b80 4 client.17598 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:40:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ef9f478b0 con 0x561ef9f91f30 2023-10-18T12:40:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.838+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9f91f30 0x561ef9f94350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.839+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9f91f30 0x561ef9f94350 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bbd123dc4ff6b37 server_cookie=51b097bd7ab32fab in_seq=0 out_seq=0 2023-10-18T12:40:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.839+0000 7f838d7fa700 10 client.17598 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.839+0000 7f838d7fa700 1 -- 172.21.15.70:0/1695321126 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561ef9f478b0 con 0x561ef9f91f30 2023-10-18T12:40:07.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.839+0000 7f838d7fa700 10 client.17598 handle_command_reply: tid=0 2023-10-18T12:40:07.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.839+0000 7f83bfbf1b80 1 client.17598 shutdown 2023-10-18T12:40:07.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 msgr2=0x561ef9f822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 0x561ef9f822f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f839c018b00 tx=0x7f839c004b40 comp rx=0 tx=0).stop 2023-10-18T12:40:07.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 shutdown_connections 2023-10-18T12:40:07.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9f91f30 0x561ef9f94350 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7e500 0x561ef9f822f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f785d0 0x561ef9f81870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1695321126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f838d7fa700 0 client.17598 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 >> 172.21.15.70:0/1695321126 conn(0x561ef9eec720 msgr2=0x561ef9f77080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 shutdown_connections 2023-10-18T12:40:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1695321126 wait complete. 2023-10-18T12:40:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.840+0000 7f83bfbf1b80 20 client.17598 trim_cache size 0 max 16384 2023-10-18T12:40:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.853+0000 7f83bfbf1b80 1 Processor -- start 2023-10-18T12:40:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.853+0000 7f83bfbf1b80 1 -- start start 2023-10-18T12:40:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.853+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 0x561ef9f88e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.853+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.853+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7e500 0x561ef9f82300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9f73130 con 0x561ef9f785d0 2023-10-18T12:40:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9d4b920 con 0x561ef9f7e500 2023-10-18T12:40:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9f8e9b0 con 0x561ef9f7dbe0 2023-10-18T12:40:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 0x561ef9f88e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 0x561ef9f88e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43372/0 (socket says 172.21.15.70:43372) 2023-10-18T12:40:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 -- 172.21.15.70:0/2842447974 learned_addr learned my addr 172.21.15.70:0/2842447974 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a979a700 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83aa79c700 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7e500 0x561ef9f82300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 -- 172.21.15.70:0/2842447974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7dbe0 msgr2=0x561ef9f81dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 -- 172.21.15.70:0/2842447974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7e500 msgr2=0x561ef9f82300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7e500 0x561ef9f82300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 -- 172.21.15.70:0/2842447974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef9cdec60 con 0x561ef9f785d0 2023-10-18T12:40:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 0x561ef9f88e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f83a0095ed0 tx=0x7f83a009a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ad0fb92b1390da server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.854+0000 7f8398ff9700 1 -- 172.21.15.70:0/2842447974 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0005620 con 0x561ef9f785d0 2023-10-18T12:40:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f8398ff9700 1 -- 172.21.15.70:0/2842447974 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a00b2110 con 0x561ef9f785d0 2023-10-18T12:40:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f8398ff9700 1 -- 172.21.15.70:0/2842447974 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0099c30 con 0x561ef9f785d0 2023-10-18T12:40:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/2842447974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 msgr2=0x561ef9f88e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 0x561ef9f88e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f83a0095ed0 tx=0x7f83a009a240 comp rx=0 tx=0).stop 2023-10-18T12:40:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/2842447974 shutdown_connections 2023-10-18T12:40:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f7e500 0x561ef9f82300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f785d0 0x561ef9f88e50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/2842447974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7dbe0 0x561ef9f81dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/2842447974 >> 172.21.15.70:0/2842447974 conn(0x561ef9eec720 msgr2=0x561ef9f8d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/2842447974 shutdown_connections 2023-10-18T12:40:07.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.855+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/2842447974 wait complete. 2023-10-18T12:40:07.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.856+0000 7f83bfbf1b80 1 -- start start 2023-10-18T12:40:07.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.856+0000 7f83bfbf1b80 1 Processor -- start 2023-10-18T12:40:07.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f87020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f87560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9fdeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9fdf2e0 con 0x561ef9f7dbe0 2023-10-18T12:40:07.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9fdf450 con 0x561ef9f785d0 2023-10-18T12:40:07.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef9fdf610 con 0x561ef9f7e500 2023-10-18T12:40:07.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83bfbf1b80 10 client.0 _ll_register_callbacks cb 0x561ef9db1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:40:07.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f87020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f87020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:34334/0 (socket says 172.21.15.70:34334) 2023-10-18T12:40:07.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 -- 172.21.15.70:0/1943968537 learned_addr learned my addr 172.21.15.70:0/1943968537 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83aa79c700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9fdeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a979a700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f87560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 -- 172.21.15.70:0/1943968537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 msgr2=0x561ef9fdeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9fdeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 -- 172.21.15.70:0/1943968537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 msgr2=0x561ef9f87560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:40:07.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f87560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.857+0000 7f83a9f9b700 1 -- 172.21.15.70:0/1943968537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ef9e883f0 con 0x561ef9f785d0 2023-10-18T12:40:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f83a9f9b700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f87020 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f83a00996d0 tx=0x7f83a0004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafc9b14d0636010 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f838d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2023-10-18T12:40:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f838d7fa700 1 -- 172.21.15.70:0/1943968537 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a009a0d0 con 0x561ef9f785d0 2023-10-18T12:40:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83a00a1d60 con 0x561ef9f785d0 2023-10-18T12:40:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f838d7fa700 1 -- 172.21.15.70:0/1943968537 <== mon.2 v2:172.21.15.152:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f83a0098fd0 con 0x561ef9f785d0 2023-10-18T12:40:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f838d7fa700 1 -- 172.21.15.70:0/1943968537 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a0099190 con 0x561ef9f785d0 2023-10-18T12:40:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f83bfbf1b80 10 client.27662 fetch_fsmap learned FSMap version 20 2023-10-18T12:40:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.858+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x561ef9d1c250 con 0x561ef9f785d0 2023-10-18T12:40:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f838d7fa700 1 -- 172.21.15.70:0/1943968537 <== mon.2 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a00a1d60 con 0x561ef9f785d0 2023-10-18T12:40:07.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f838d7fa700 1 -- 172.21.15.70:0/1943968537 <== mon.2 v2:172.21.15.152:3300/0 5 ==== fsmap(e 20) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f83a00a18d0 con 0x561ef9f785d0 2023-10-18T12:40:07.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83bfbf1b80 10 client.27662 fetch_fsmap finished waiting for FSMap version 20 2023-10-18T12:40:07.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83bfbf1b80 10 client.27662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T12:40:07.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83bfbf1b80 20 client.27662 populate_metadata read hostname 'smithi070' 2023-10-18T12:40:07.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9fe6930 0x561ef9fe8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:07.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83bfbf1b80 4 client.27662 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T12:40:07.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 --> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ef9ec3e50 con 0x561ef9fe6930 2023-10-18T12:40:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.859+0000 7f83a979a700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9fe6930 0x561ef9fe8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f838d7fa700 10 client.27662 ms_handle_connect on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83a979a700 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9fe6930 0x561ef9fe8d20 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff666d773dcc27a server_cookie=2c4375ead0d807a2 in_seq=0 out_seq=0 2023-10-18T12:40:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f838d7fa700 1 -- 172.21.15.70:0/1943968537 <== mds.0 v2:172.21.15.152:6838/1810263953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ef9ec3e50 con 0x561ef9fe6930 2023-10-18T12:40:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f838d7fa700 10 client.27662 handle_command_reply: tid=0 2023-10-18T12:40:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 client.27662 shutdown 2023-10-18T12:40:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 msgr2=0x561ef9f87020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f87020 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f83a00996d0 tx=0x7f83a0004e40 comp rx=0 tx=0).stop 2023-10-18T12:40:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 shutdown_connections 2023-10-18T12:40:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:6838/1810263953,v1:172.21.15.152:6839/1810263953] conn(0x561ef9fe6930 0x561ef9fe8d20 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561ef9f7e500 0x561ef9fdeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561ef9f785d0 0x561ef9f87020 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 --2- 172.21.15.70:0/1943968537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561ef9f7dbe0 0x561ef9f87560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f838d7fa700 0 client.27662 ms_handle_reset on v2:172.21.15.152:6838/1810263953 2023-10-18T12:40:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 >> 172.21.15.70:0/1943968537 conn(0x561ef9eec720 msgr2=0x561ef9f80100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 shutdown_connections 2023-10-18T12:40:07.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 1 -- 172.21.15.70:0/1943968537 wait complete. 2023-10-18T12:40:07.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.860+0000 7f83bfbf1b80 20 client.27662 trim_cache size 0 max 16384 2023-10-18T12:40:07.904 INFO:teuthology.orchestra.run.smithi070.stdout:[] 2023-10-18T12:40:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83900a15f0 msgr2=0x7f83900a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83900a15f0 0x7f83900a3ab0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f839c00f870 tx=0x7f839c00fc30 comp rx=0 tx=0).stop 2023-10-18T12:40:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 msgr2=0x7f83a4198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 0x7f83a4198df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f83a000d750 tx=0x7f83a000da60 comp rx=0 tx=0).stop 2023-10-18T12:40:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 shutdown_connections 2023-10-18T12:40:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a410cf50 0x7f83a419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83a4072c20 0x7f83a4198df0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f83900a15f0 0x7f83900a3ab0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 --2- 172.21.15.70:0/5784039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a4073540 0x7f83a4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:07.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 >> 172.21.15.70:0/5784039 conn(0x7f83a40fc200 msgr2=0x7f83a40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:07.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 shutdown_connections 2023-10-18T12:40:07.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:07.861+0000 7f83ac1ff700 1 -- 172.21.15.70:0/5784039 wait complete. 2023-10-18T12:40:08.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:07 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/4209255785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:08.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:07 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/4209255785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:08.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:07 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/4209255785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:08.910 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-10-18T12:40:08.911 INFO:tasks.fwd_scrub:done joining 2023-10-18T12:40:08.911 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-18T12:40:08.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:40:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 -- 172.21.15.70:0/979673464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804101060 msgr2=0x7fe80410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 --2- 172.21.15.70:0/979673464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804101060 0x7fe80410d570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00b0a0 tx=0x7fe7fc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 -- 172.21.15.70:0/979673464 shutdown_connections 2023-10-18T12:40:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 --2- 172.21.15.70:0/979673464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804101060 0x7fe80410d570 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 --2- 172.21.15.70:0/979673464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8041066b0 0x7fe804106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 --2- 172.21.15.70:0/979673464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8041006a0 0x7fe804100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 -- 172.21.15.70:0/979673464 >> 172.21.15.70:0/979673464 conn(0x7fe8040fc170 msgr2=0x7fe8040fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 -- 172.21.15.70:0/979673464 shutdown_connections 2023-10-18T12:40:09.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.123+0000 7fe808dff700 1 -- 172.21.15.70:0/979673464 wait complete. 2023-10-18T12:40:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.124+0000 7fe808dff700 1 Processor -- start 2023-10-18T12:40:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.124+0000 7fe808dff700 1 -- start start 2023-10-18T12:40:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe808dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 0x7fe80419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe808dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe804101060 0x7fe80419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe808dff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8041066b0 0x7fe8041968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe808dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804196f50 con 0x7fe8041006a0 2023-10-18T12:40:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe808dff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041970c0 con 0x7fe8041066b0 2023-10-18T12:40:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe808dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804197280 con 0x7fe804101060 2023-10-18T12:40:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 0x7fe80419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe7fbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe804101060 0x7fe80419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 0x7fe80419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43400/0 (socket says 172.21.15.70:43400) 2023-10-18T12:40:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 -- 172.21.15.70:0/1885517174 learned_addr learned my addr 172.21.15.70:0/1885517174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe7fbfff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe804101060 0x7fe80419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60634/0 (socket says 172.21.15.70:60634) 2023-10-18T12:40:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe802d9d700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8041066b0 0x7fe8041968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 -- 172.21.15.70:0/1885517174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe804101060 msgr2=0x7fe80419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe804101060 0x7fe80419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 -- 172.21.15.70:0/1885517174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8041066b0 msgr2=0x7fe8041968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8041066b0 0x7fe8041968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.125+0000 7fe80259c700 1 -- 172.21.15.70:0/1885517174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc009d00 con 0x7fe8041006a0 2023-10-18T12:40:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.126+0000 7fe80259c700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 0x7fe80419c7c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e9a0 tx=0x7fe7f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.126+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4009790 con 0x7fe8041006a0 2023-10-18T12:40:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.126+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f400cbd0 con 0x7fe8041006a0 2023-10-18T12:40:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.126+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe804197560 con 0x7fe8041006a0 2023-10-18T12:40:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.126+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4018680 con 0x7fe8041006a0 2023-10-18T12:40:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.126+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8041a37b0 con 0x7fe8041006a0 2023-10-18T12:40:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.127+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe804108ca0 con 0x7fe8041006a0 2023-10-18T12:40:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.129+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe7f400cd40 con 0x7fe8041006a0 2023-10-18T12:40:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.130+0000 7fe7fb7fe700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7f00a15f0 0x7fe7f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.130+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fe7f4014070 con 0x7fe8041006a0 2023-10-18T12:40:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.130+0000 7fe7fbfff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7f00a15f0 0x7fe7f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.131+0000 7fe7fbfff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7f00a15f0 0x7fe7f00a3ab0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe804197ab0 tx=0x7fe7ec009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.134+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7f4052100 con 0x7fe8041006a0 2023-10-18T12:40:09.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:08 smithi186 ceph-mon[135253]: pgmap v753: 129 pgs: 129 active+clean; 781 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 24 MiB/s wr, 482 op/s 2023-10-18T12:40:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.338+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe804197db0 con 0x7fe8041006a0 2023-10-18T12:40:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.340+0000 7fe7fb7fe700 1 -- 172.21.15.70:0/1885517174 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+25229 (secure 0 0 0) 0x7fe7f4096640 con 0x7fe8041006a0 2023-10-18T12:40:09.343 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:09.344 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":108,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:40:03.918074+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:40:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.342+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7f00a15f0 msgr2=0x7fe7f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.342+0000 7fe808dff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7f00a15f0 0x7fe7f00a3ab0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe804197ab0 tx=0x7fe7ec009260 comp rx=0 tx=0).stop 2023-10-18T12:40:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.342+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 msgr2=0x7fe80419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.342+0000 7fe808dff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 0x7fe80419c7c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e9a0 tx=0x7fe7f400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 shutdown_connections 2023-10-18T12:40:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fe7f00a15f0 0x7fe7f00a3ab0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8041066b0 0x7fe8041968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8041006a0 0x7fe80419c7c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 --2- 172.21.15.70:0/1885517174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe804101060 0x7fe80419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 >> 172.21.15.70:0/1885517174 conn(0x7fe8040fc170 msgr2=0x7fe8040fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 shutdown_connections 2023-10-18T12:40:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.343+0000 7fe808dff700 1 -- 172.21.15.70:0/1885517174 wait complete. 2023-10-18T12:40:09.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:08 smithi070 ceph-mon[98639]: pgmap v753: 129 pgs: 129 active+clean; 781 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 24 MiB/s wr, 482 op/s 2023-10-18T12:40:09.365 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:40:09.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:40:09.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:08 smithi152 ceph-mon[134767]: pgmap v753: 129 pgs: 129 active+clean; 781 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 24 MiB/s wr, 482 op/s 2023-10-18T12:40:09.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 -- 172.21.15.70:0/646792023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b0106730 msgr2=0x7f98b0106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 --2- 172.21.15.70:0/646792023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b0106730 0x7f98b0106b10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009990 tx=0x7f98a0009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 -- 172.21.15.70:0/646792023 shutdown_connections 2023-10-18T12:40:09.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 --2- 172.21.15.70:0/646792023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b01010e0 0x7f98b010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 --2- 172.21.15.70:0/646792023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b0106730 0x7f98b0106b10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 --2- 172.21.15.70:0/646792023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98b0100720 0x7f98b0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 -- 172.21.15.70:0/646792023 >> 172.21.15.70:0/646792023 conn(0x7f98b00fc1b0 msgr2=0x7f98b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:09.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 -- 172.21.15.70:0/646792023 shutdown_connections 2023-10-18T12:40:09.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.564+0000 7f98b4bff700 1 -- 172.21.15.70:0/646792023 wait complete. 2023-10-18T12:40:09.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.565+0000 7f98b4bff700 1 Processor -- start 2023-10-18T12:40:09.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.565+0000 7f98b4bff700 1 -- start start 2023-10-18T12:40:09.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.565+0000 7f98b4bff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b0100720 0x7f98b0198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.565+0000 7f98b4bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 0x7f98b0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.565+0000 7f98b4bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98b0106730 0x7f98b019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.565+0000 7f98b4bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b019dcf0 con 0x7f98b01010e0 2023-10-18T12:40:09.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98b4bff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b019de60 con 0x7f98b0100720 2023-10-18T12:40:09.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98b4bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b019e020 con 0x7f98b0106730 2023-10-18T12:40:09.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 0x7f98b0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 0x7f98b0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43418/0 (socket says 172.21.15.70:43418) 2023-10-18T12:40:09.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 -- 172.21.15.70:0/1428670142 learned_addr learned my addr 172.21.15.70:0/1428670142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98aed9d700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98b0106730 0x7f98b019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98ae59c700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b0100720 0x7f98b0198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 -- 172.21.15.70:0/1428670142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98b0106730 msgr2=0x7f98b019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98b0106730 0x7f98b019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 -- 172.21.15.70:0/1428670142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b0100720 msgr2=0x7f98b0198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b0100720 0x7f98b0198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 -- 172.21.15.70:0/1428670142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00095f0 con 0x7f98b01010e0 2023-10-18T12:40:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.566+0000 7f98add9b700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 0x7f98b0199310 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e9a0 tx=0x7f98a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:09.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.567+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4009790 con 0x7f98b01010e0 2023-10-18T12:40:09.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.567+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98a4010430 con 0x7f98b01010e0 2023-10-18T12:40:09.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.567+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b019e300 con 0x7f98b01010e0 2023-10-18T12:40:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.567+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4018680 con 0x7f98b01010e0 2023-10-18T12:40:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.567+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b01a5b40 con 0x7f98b01010e0 2023-10-18T12:40:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.568+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b0101e60 con 0x7f98b01010e0 2023-10-18T12:40:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.570+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f98a40105a0 con 0x7f98b01010e0 2023-10-18T12:40:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.571+0000 7f989f7fe700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98980a15f0 0x7f98980a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:09.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.571+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f98a4014070 con 0x7f98b01010e0 2023-10-18T12:40:09.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.571+0000 7f98ae59c700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98980a15f0 0x7f98980a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:09.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.572+0000 7f98ae59c700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98980a15f0 0x7f98980a3ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f98a000b2e0 tx=0x7f98a0005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:09.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.575+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f98a4052100 con 0x7f98b01010e0 2023-10-18T12:40:09.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.779+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98b0110690 con 0x7f98b01010e0 2023-10-18T12:40:09.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.780+0000 7f989f7fe700 1 -- 172.21.15.70:0/1428670142 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98a40965e0 con 0x7f98b01010e0 2023-10-18T12:40:09.783 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:40:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98980a15f0 msgr2=0x7f98980a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98980a15f0 0x7f98980a3ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f98a000b2e0 tx=0x7f98a0005fd0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 msgr2=0x7f98b0199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 0x7f98b0199310 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e9a0 tx=0x7f98a400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 shutdown_connections 2023-10-18T12:40:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98b0106730 0x7f98b019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f98980a15f0 0x7f98980a3ab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b0100720 0x7f98b0198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 --2- 172.21.15.70:0/1428670142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b01010e0 0x7f98b0199310 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:09.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 >> 172.21.15.70:0/1428670142 conn(0x7f98b00fc1b0 msgr2=0x7f98b010be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:09.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 shutdown_connections 2023-10-18T12:40:09.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:09.782+0000 7f98b4bff700 1 -- 172.21.15.70:0/1428670142 wait complete. 2023-10-18T12:40:09.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:40:10.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 -- 172.21.15.70:0/705269131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec106720 msgr2=0x7f5bec106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/705269131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec106720 0x7f5bec106b00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5be80098f0 tx=0x7f5be8009c00 comp rx=0 tx=0).stop 2023-10-18T12:40:10.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 -- 172.21.15.70:0/705269131 shutdown_connections 2023-10-18T12:40:10.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/705269131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bec1010d0 0x7f5bec10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/705269131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec106720 0x7f5bec106b00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/705269131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bec100710 0x7f5bec100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.006+0000 7f5bf3fff700 1 -- 172.21.15.70:0/705269131 >> 172.21.15.70:0/705269131 conn(0x7f5bec0fc200 msgr2=0x7f5bec0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:10.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.007+0000 7f5bf3fff700 1 -- 172.21.15.70:0/705269131 shutdown_connections 2023-10-18T12:40:10.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.007+0000 7f5bf3fff700 1 -- 172.21.15.70:0/705269131 wait complete. 2023-10-18T12:40:10.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 Processor -- start 2023-10-18T12:40:10.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 -- start start 2023-10-18T12:40:10.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bec100710 0x7f5bec198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 0x7f5bec1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bec106720 0x7f5bec19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19dd70 con 0x7f5bec1010d0 2023-10-18T12:40:10.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19dee0 con 0x7f5bec100710 2023-10-18T12:40:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19e0a0 con 0x7f5bec106720 2023-10-18T12:40:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf159a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 0x7f5bec1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf159a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 0x7f5bec1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43434/0 (socket says 172.21.15.70:43434) 2023-10-18T12:40:10.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.008+0000 7f5bf159a700 1 -- 172.21.15.70:0/874912500 learned_addr learned my addr 172.21.15.70:0/874912500 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:10.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf1d9b700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bec100710 0x7f5bec198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf259c700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bec106720 0x7f5bec19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf159a700 1 -- 172.21.15.70:0/874912500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bec106720 msgr2=0x7f5bec19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf159a700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bec106720 0x7f5bec19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf159a700 1 -- 172.21.15.70:0/874912500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bec100710 msgr2=0x7f5bec198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf159a700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bec100710 0x7f5bec198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf159a700 1 -- 172.21.15.70:0/874912500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8009520 con 0x7f5bec1010d0 2023-10-18T12:40:10.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf159a700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 0x7f5bec1993e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e9c0 tx=0x7f5bdc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:10.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc004a30 con 0x7f5bec1010d0 2023-10-18T12:40:10.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bdc00cc80 con 0x7f5bec1010d0 2023-10-18T12:40:10.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec19e380 con 0x7f5bec1010d0 2023-10-18T12:40:10.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0054e0 con 0x7f5bec1010d0 2023-10-18T12:40:10.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.009+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec1a5bc0 con 0x7f5bec1010d0 2023-10-18T12:40:10.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.010+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec101e50 con 0x7f5bec1010d0 2023-10-18T12:40:10.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.012+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5bdc004b90 con 0x7f5bec1010d0 2023-10-18T12:40:10.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.014+0000 7f5be2ffd700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5bd80a1670 0x7f5bd80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.014+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f5bdc014070 con 0x7f5bec1010d0 2023-10-18T12:40:10.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.014+0000 7f5bf1d9b700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5bd80a1670 0x7f5bd80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.015+0000 7f5bf1d9b700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5bd80a1670 0x7f5bd80a3b30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5be80098c0 tx=0x7f5be8019040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:10.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.016+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5bdc096690 con 0x7f5bec1010d0 2023-10-18T12:40:10.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:09 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1885517174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:10.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:09 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1428670142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:40:10.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:09 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1885517174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:10.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:09 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1428670142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:40:10.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.220+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5bec19a150 con 0x7f5bec1010d0 2023-10-18T12:40:10.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.220+0000 7f5be2ffd700 1 -- 172.21.15.70:0/874912500 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5bdc099cb0 con 0x7f5bec1010d0 2023-10-18T12:40:10.224 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:40:10.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.222+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5bd80a1670 msgr2=0x7f5bd80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.222+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5bd80a1670 0x7f5bd80a3b30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5be80098c0 tx=0x7f5be8019040 comp rx=0 tx=0).stop 2023-10-18T12:40:10.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.222+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 msgr2=0x7f5bec1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.222+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 0x7f5bec1993e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e9c0 tx=0x7f5bdc00ecd0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 shutdown_connections 2023-10-18T12:40:10.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bec106720 0x7f5bec19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f5bd80a1670 0x7f5bd80a3b30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bec100710 0x7f5bec198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 --2- 172.21.15.70:0/874912500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec1010d0 0x7f5bec1993e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 >> 172.21.15.70:0/874912500 conn(0x7f5bec0fc200 msgr2=0x7f5bec0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:10.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 shutdown_connections 2023-10-18T12:40:10.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.223+0000 7f5bf3fff700 1 -- 172.21.15.70:0/874912500 wait complete. 2023-10-18T12:40:10.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:40:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:09 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1885517174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:10.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:09 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1428670142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:40:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 -- 172.21.15.70:0/2635368238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc1010e0 msgr2=0x7fecfc10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 --2- 172.21.15.70:0/2635368238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc1010e0 0x7fecfc10d5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009f90 tx=0x7fecf400b330 comp rx=0 tx=0).stop 2023-10-18T12:40:10.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 -- 172.21.15.70:0/2635368238 shutdown_connections 2023-10-18T12:40:10.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 --2- 172.21.15.70:0/2635368238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc1010e0 0x7fecfc10d5f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 --2- 172.21.15.70:0/2635368238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecfc106730 0x7fecfc106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 --2- 172.21.15.70:0/2635368238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 -- 172.21.15.70:0/2635368238 >> 172.21.15.70:0/2635368238 conn(0x7fecfc0fc1b0 msgr2=0x7fecfc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 -- 172.21.15.70:0/2635368238 shutdown_connections 2023-10-18T12:40:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.446+0000 7fed015ff700 1 -- 172.21.15.70:0/2635368238 wait complete. 2023-10-18T12:40:10.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.447+0000 7fed015ff700 1 Processor -- start 2023-10-18T12:40:10.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.447+0000 7fed015ff700 1 -- start start 2023-10-18T12:40:10.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fed015ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fed015ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecfc1010e0 0x7fecfc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fed015ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc106730 0x7fecfc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fed015ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19dcd0 con 0x7fecfc100720 2023-10-18T12:40:10.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fed015ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19de40 con 0x7fecfc1010e0 2023-10-18T12:40:10.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fed015ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19e000 con 0x7fecfc106730 2023-10-18T12:40:10.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfaffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfaffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43448/0 (socket says 172.21.15.70:43448) 2023-10-18T12:40:10.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfaffd700 1 -- 172.21.15.70:0/665142357 learned_addr learned my addr 172.21.15.70:0/665142357 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:10.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfa7fc700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecfc1010e0 0x7fecfc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfb7fe700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc106730 0x7fecfc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfaffd700 1 -- 172.21.15.70:0/665142357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc106730 msgr2=0x7fecfc19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.448+0000 7fecfaffd700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc106730 0x7fecfc19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fecfaffd700 1 -- 172.21.15.70:0/665142357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecfc1010e0 msgr2=0x7fecfc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fecfaffd700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecfc1010e0 0x7fecfc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fecfaffd700 1 -- 172.21.15.70:0/665142357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4009c40 con 0x7fecfc100720 2023-10-18T12:40:10.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fecfaffd700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc198db0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e9a0 tx=0x7fecec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:10.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec009790 con 0x7fecfc100720 2023-10-18T12:40:10.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecec00ce80 con 0x7fecfc100720 2023-10-18T12:40:10.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc19e2e0 con 0x7fecfc100720 2023-10-18T12:40:10.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec018680 con 0x7fecfc100720 2023-10-18T12:40:10.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.449+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc1a5b20 con 0x7fecfc100720 2023-10-18T12:40:10.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.450+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecfc101e60 con 0x7fecfc100720 2023-10-18T12:40:10.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.452+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fecec010430 con 0x7fecfc100720 2023-10-18T12:40:10.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.454+0000 7fece3fff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fece40a15f0 0x7fece40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.454+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fecec014070 con 0x7fecfc100720 2023-10-18T12:40:10.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.455+0000 7fecfa7fc700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fece40a15f0 0x7fece40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.455+0000 7fecfa7fc700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fece40a15f0 0x7fece40a3ab0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fecfc19a300 tx=0x7fecf0009310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:10.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.458+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fecec052100 con 0x7fecfc100720 2023-10-18T12:40:10.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.660+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecfc19a0b0 con 0x7fecfc100720 2023-10-18T12:40:10.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.661+0000 7fece3fff700 1 -- 172.21.15.70:0/665142357 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecec09a000 con 0x7fecfc100720 2023-10-18T12:40:10.665 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:40:10.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fece40a15f0 msgr2=0x7fece40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fece40a15f0 0x7fece40a3ab0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fecfc19a300 tx=0x7fecf0009310 comp rx=0 tx=0).stop 2023-10-18T12:40:10.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 msgr2=0x7fecfc198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc198db0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e9a0 tx=0x7fecec00ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 shutdown_connections 2023-10-18T12:40:10.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecfc106730 0x7fecfc19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecfc100720 0x7fecfc198db0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fece40a15f0 0x7fece40a3ab0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 --2- 172.21.15.70:0/665142357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecfc1010e0 0x7fecfc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.663+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 >> 172.21.15.70:0/665142357 conn(0x7fecfc0fc1b0 msgr2=0x7fecfc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.664+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 shutdown_connections 2023-10-18T12:40:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.664+0000 7fed015ff700 1 -- 172.21.15.70:0/665142357 wait complete. 2023-10-18T12:40:10.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:40:10.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 -- 172.21.15.70:0/393471166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70072c20 msgr2=0x7fed70073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 --2- 172.21.15.70:0/393471166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70072c20 0x7fed70073000 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fed64009990 tx=0x7fed64009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 -- 172.21.15.70:0/393471166 shutdown_connections 2023-10-18T12:40:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 --2- 172.21.15.70:0/393471166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7010cf60 0x7fed7010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 --2- 172.21.15.70:0/393471166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70072c20 0x7fed70073000 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 --2- 172.21.15.70:0/393471166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed70073540 0x7fed7010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 -- 172.21.15.70:0/393471166 >> 172.21.15.70:0/393471166 conn(0x7fed700fc1b0 msgr2=0x7fed700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 -- 172.21.15.70:0/393471166 shutdown_connections 2023-10-18T12:40:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.885+0000 7fed76fff700 1 -- 172.21.15.70:0/393471166 wait complete. 2023-10-18T12:40:10.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.886+0000 7fed76fff700 1 Processor -- start 2023-10-18T12:40:10.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.886+0000 7fed76fff700 1 -- start start 2023-10-18T12:40:10.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed76fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70072c20 0x7fed70198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed76fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed70073540 0x7fed701993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed76fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7010cf60 0x7fed7019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed76fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7019dd30 con 0x7fed7010cf60 2023-10-18T12:40:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed76fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7019dea0 con 0x7fed70073540 2023-10-18T12:40:10.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed76fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7019e060 con 0x7fed70072c20 2023-10-18T12:40:10.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed7559c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7010cf60 0x7fed7019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed74d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70072c20 0x7fed70198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed74d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70072c20 0x7fed70198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:60708/0 (socket says 172.21.15.70:60708) 2023-10-18T12:40:10.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed74d9b700 1 -- 172.21.15.70:0/723736819 learned_addr learned my addr 172.21.15.70:0/723736819 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:10.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed6ffff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed70073540 0x7fed701993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed7559c700 1 -- 172.21.15.70:0/723736819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70072c20 msgr2=0x7fed70198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed7559c700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70072c20 0x7fed70198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed7559c700 1 -- 172.21.15.70:0/723736819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed70073540 msgr2=0x7fed701993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:10.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed7559c700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed70073540 0x7fed701993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:10.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.887+0000 7fed7559c700 1 -- 172.21.15.70:0/723736819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640095f0 con 0x7fed7010cf60 2023-10-18T12:40:10.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.888+0000 7fed7559c700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7010cf60 0x7fed7019d6f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fed6800efc0 tx=0x7fed6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:10.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.888+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68018070 con 0x7fed7010cf60 2023-10-18T12:40:10.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.888+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7019e2e0 con 0x7fed7010cf60 2023-10-18T12:40:10.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.888+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6800f040 con 0x7fed7010cf60 2023-10-18T12:40:10.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.888+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800cce0 con 0x7fed7010cf60 2023-10-18T12:40:10.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.888+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed701a5b80 con 0x7fed7010cf60 2023-10-18T12:40:10.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.889+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed70108170 con 0x7fed7010cf60 2023-10-18T12:40:10.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.891+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fed68003a00 con 0x7fed7010cf60 2023-10-18T12:40:10.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.892+0000 7fed6dffb700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed600a15f0 0x7fed600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:10.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.892+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fed680ccf80 con 0x7fed7010cf60 2023-10-18T12:40:10.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.893+0000 7fed74d9b700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed600a15f0 0x7fed600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:10.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.893+0000 7fed74d9b700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed600a15f0 0x7fed600a3ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fed64006010 tx=0x7fed6400b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:10.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:10.896+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fed68100100 con 0x7fed7010cf60 2023-10-18T12:40:11.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.100+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fed7019a110 con 0x7fed7010cf60 2023-10-18T12:40:11.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.101+0000 7fed6dffb700 1 -- 172.21.15.70:0/723736819 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) v1 ==== 103+0+11 (secure 0 0 0) 0x7fed68095f80 con 0x7fed7010cf60 2023-10-18T12:40:11.104 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:40:11.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed600a15f0 msgr2=0x7fed600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:11.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed600a15f0 0x7fed600a3ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fed64006010 tx=0x7fed6400b260 comp rx=0 tx=0).stop 2023-10-18T12:40:11.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7010cf60 msgr2=0x7fed7019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7010cf60 0x7fed7019d6f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fed6800efc0 tx=0x7fed6800c3a0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 shutdown_connections 2023-10-18T12:40:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7010cf60 0x7fed7019d6f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed70072c20 0x7fed70198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fed600a15f0 0x7fed600a3ab0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 --2- 172.21.15.70:0/723736819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed70073540 0x7fed701993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 >> 172.21.15.70:0/723736819 conn(0x7fed700fc1b0 msgr2=0x7fed700fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 shutdown_connections 2023-10-18T12:40:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.103+0000 7fed76fff700 1 -- 172.21.15.70:0/723736819 wait complete. 2023-10-18T12:40:11.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-18T12:40:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:10 smithi186 ceph-mon[135253]: pgmap v754: 129 pgs: 129 active+clean; 765 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 19 MiB/s wr, 393 op/s 2023-10-18T12:40:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:10 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/874912500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:40:11.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:10 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/665142357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:40:11.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/342822806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 msgr2=0x7fbb18106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/342822806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb18106a90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009990 tx=0x7fbb08009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/342822806 shutdown_connections 2023-10-18T12:40:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/342822806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb18101060 0x7fbb1810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/342822806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb18106a90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/342822806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb181006a0 0x7fbb18100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/342822806 >> 172.21.15.70:0/342822806 conn(0x7fbb180fc1d0 msgr2=0x7fbb180fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/342822806 shutdown_connections 2023-10-18T12:40:11.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.324+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/342822806 wait complete. 2023-10-18T12:40:11.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 Processor -- start 2023-10-18T12:40:11.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 -- start start 2023-10-18T12:40:11.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb181006a0 0x7fbb18198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:11.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb18101060 0x7fbb18199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:11.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb1819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:11.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1819cb40 con 0x7fbb181066b0 2023-10-18T12:40:11.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1819ccb0 con 0x7fbb181006a0 2023-10-18T12:40:11.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.325+0000 7fbb1e7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1819ce70 con 0x7fbb18101060 2023-10-18T12:40:11.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb1819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:11.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb1819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43490/0 (socket says 172.21.15.70:43490) 2023-10-18T12:40:11.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 -- 172.21.15.70:0/488299454 learned_addr learned my addr 172.21.15.70:0/488299454 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:11.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb17fff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb181006a0 0x7fbb18198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:11.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb0ffff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb18101060 0x7fbb18199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:11.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 -- 172.21.15.70:0/488299454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb18101060 msgr2=0x7fbb18199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:11.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb18101060 0x7fbb18199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 -- 172.21.15.70:0/488299454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb181006a0 msgr2=0x7fbb18198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:11.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb181006a0 0x7fbb18198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:11.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 -- 172.21.15.70:0/488299454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080095f0 con 0x7fbb181066b0 2023-10-18T12:40:11.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb17fff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb181006a0 0x7fbb18198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:11.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.326+0000 7fbb1cd9c700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb1819c540 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000ef30 tx=0x7fbb1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:11.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.327+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10018070 con 0x7fbb181066b0 2023-10-18T12:40:11.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.327+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1000f040 con 0x7fbb181066b0 2023-10-18T12:40:11.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.327+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000cd00 con 0x7fbb181066b0 2023-10-18T12:40:11.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.327+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1819d0f0 con 0x7fbb181066b0 2023-10-18T12:40:11.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.327+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb181a5ab0 con 0x7fbb181066b0 2023-10-18T12:40:11.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.328+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb18101de0 con 0x7fbb181066b0 2023-10-18T12:40:11.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.330+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbb10003a00 con 0x7fbb181066b0 2023-10-18T12:40:11.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.331+0000 7fbb15ffb700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbb040a1670 0x7fbb040a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:11.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.331+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fbb100ccfa0 con 0x7fbb181066b0 2023-10-18T12:40:11.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.331+0000 7fbb17fff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbb040a1670 0x7fbb040a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:11.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.332+0000 7fbb17fff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbb040a1670 0x7fbb040a3b30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb0800b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:11.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.335+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb10095f40 con 0x7fbb181066b0 2023-10-18T12:40:11.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:10 smithi070 ceph-mon[98639]: pgmap v754: 129 pgs: 129 active+clean; 765 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 19 MiB/s wr, 393 op/s 2023-10-18T12:40:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:10 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/874912500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:40:11.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:10 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/665142357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:40:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:10 smithi152 ceph-mon[134767]: pgmap v754: 129 pgs: 129 active+clean; 765 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 19 MiB/s wr, 393 op/s 2023-10-18T12:40:11.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:10 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/874912500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:40:11.423 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:10 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/665142357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:40:11.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.598+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fbb18110620 con 0x7fbb181066b0 2023-10-18T12:40:11.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.609+0000 7fbb15ffb700 1 -- 172.21.15.70:0/488299454 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+734405 (secure 0 0 0) 0x7fbb10099560 con 0x7fbb181066b0 2023-10-18T12:40:11.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:11.622 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:40:11.623 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 24, 2023-10-18T12:40:11.623 INFO:teuthology.orchestra.run.smithi070.stdout: "active_gid": 14214, 2023-10-18T12:40:11.623 INFO:teuthology.orchestra.run.smithi070.stdout: "active_name": "x", 2023-10-18T12:40:11.624 INFO:teuthology.orchestra.run.smithi070.stdout: "active_addrs": { 2023-10-18T12:40:11.624 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:11.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:11.624 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:11.625 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-10-18T12:40:11.625 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 223035468 2023-10-18T12:40:11.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:11.626 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-10-18T12:40:11.626 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-10-18T12:40:11.626 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 223035468 2023-10-18T12:40:11.627 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:11.627 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:11.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.627 INFO:teuthology.orchestra.run.smithi070.stdout: "active_addr": "172.21.15.70:6801/223035468", 2023-10-18T12:40:11.628 INFO:teuthology.orchestra.run.smithi070.stdout: "active_change": "2023-10-18T12:18:13.515456+0000", 2023-10-18T12:40:11.628 INFO:teuthology.orchestra.run.smithi070.stdout: "active_mgr_features": 4540138322906710015, 2023-10-18T12:40:11.628 INFO:teuthology.orchestra.run.smithi070.stdout: "available": true, 2023-10-18T12:40:11.629 INFO:teuthology.orchestra.run.smithi070.stdout: "standbys": [ 2023-10-18T12:40:11.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:11.629 INFO:teuthology.orchestra.run.smithi070.stdout: "gid": 24145, 2023-10-18T12:40:11.629 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "z", 2023-10-18T12:40:11.630 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_features": 4540138322906710015, 2023-10-18T12:40:11.630 INFO:teuthology.orchestra.run.smithi070.stdout: "available_modules": [ 2023-10-18T12:40:11.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:11.631 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "alerts", 2023-10-18T12:40:11.631 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:11.631 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:11.631 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:11.632 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:11.632 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:11.632 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.632 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.633 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.633 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:11.633 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.634 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.634 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.634 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How frequently to reexamine health status", 2023-10-18T12:40:11.634 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.635 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.635 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.635 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:11.636 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:11.636 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.636 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.637 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.637 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.637 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.637 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.638 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.638 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:11.638 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:11.639 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:11.639 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:11.639 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:11.639 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:11.640 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.640 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.640 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.640 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.641 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.641 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:11.642 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:11.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.642 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.642 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.643 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.643 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.643 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.643 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.644 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.644 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.644 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.645 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.645 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:11.645 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:11.646 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.646 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.646 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.646 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:11.647 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.647 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.647 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.648 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:11.648 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:11.648 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:11.648 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:11.649 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:11.649 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:11.649 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.649 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.650 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.650 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.650 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.651 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:11.651 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:11.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.652 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.652 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.652 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.652 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.653 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.653 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.653 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.653 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.654 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.654 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.655 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_destination": { 2023-10-18T12:40:11.655 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_destination", 2023-10-18T12:40:11.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.656 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.656 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.656 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.656 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.657 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.657 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.657 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Email address to send alerts to", 2023-10-18T12:40:11.658 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.658 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.658 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.659 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_from_name": { 2023-10-18T12:40:11.659 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_from_name", 2023-10-18T12:40:11.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.660 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.660 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.660 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "Ceph", 2023-10-18T12:40:11.660 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.661 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.661 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.661 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Email From: name", 2023-10-18T12:40:11.661 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.662 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.662 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.663 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_host": { 2023-10-18T12:40:11.663 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_host", 2023-10-18T12:40:11.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.663 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.664 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.664 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.664 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.664 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.665 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.665 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP server", 2023-10-18T12:40:11.665 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.666 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.666 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.666 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_password": { 2023-10-18T12:40:11.667 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_password", 2023-10-18T12:40:11.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.667 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.667 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.668 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.668 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.668 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.669 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.669 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Password to authenticate with", 2023-10-18T12:40:11.669 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.669 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.670 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.670 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_port": { 2023-10-18T12:40:11.670 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_port", 2023-10-18T12:40:11.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.671 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.671 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.672 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "465", 2023-10-18T12:40:11.672 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.672 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.673 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.673 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP port", 2023-10-18T12:40:11.673 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.673 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.674 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.674 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_sender": { 2023-10-18T12:40:11.674 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_sender", 2023-10-18T12:40:11.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.675 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.675 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.676 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.676 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.676 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.676 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.677 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP envelope sender", 2023-10-18T12:40:11.677 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.677 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.677 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.678 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_ssl": { 2023-10-18T12:40:11.679 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_ssl", 2023-10-18T12:40:11.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.679 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.679 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.680 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.680 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.680 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.680 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.681 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-18T12:40:11.681 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.681 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.682 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.682 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_user": { 2023-10-18T12:40:11.682 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_user", 2023-10-18T12:40:11.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.683 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.683 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.684 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.684 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.684 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.684 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.685 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "User to authenticate as", 2023-10-18T12:40:11.685 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.685 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.686 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.686 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:11.686 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:11.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:11.687 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "balancer", 2023-10-18T12:40:11.687 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:11.687 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:11.688 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:11.688 INFO:teuthology.orchestra.run.smithi070.stdout: "active": { 2023-10-18T12:40:11.688 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "active", 2023-10-18T12:40:11.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.689 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.689 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.689 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.690 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.690 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.690 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.690 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "automatically balance PGs across cluster", 2023-10-18T12:40:11.691 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.691 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.691 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.692 INFO:teuthology.orchestra.run.smithi070.stdout: "begin_time": { 2023-10-18T12:40:11.692 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "begin_time", 2023-10-18T12:40:11.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.693 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.693 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0000", 2023-10-18T12:40:11.694 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.694 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.694 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.694 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "beginning time of day to automatically balance", 2023-10-18T12:40:11.695 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T12:40:11.695 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.695 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.696 INFO:teuthology.orchestra.run.smithi070.stdout: "begin_weekday": { 2023-10-18T12:40:11.696 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "begin_weekday", 2023-10-18T12:40:11.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:11.697 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.697 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.697 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:11.697 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0", 2023-10-18T12:40:11.698 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "6", 2023-10-18T12:40:11.698 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.698 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-18T12:40:11.698 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T12:40:11.699 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.699 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_max_iterations": { 2023-10-18T12:40:11.700 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_max_iterations", 2023-10-18T12:40:11.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:11.700 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.701 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.701 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "25", 2023-10-18T12:40:11.701 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:11.701 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "250", 2023-10-18T12:40:11.702 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.702 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-18T12:40:11.702 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.703 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.703 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.703 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_metrics": { 2023-10-18T12:40:11.704 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_metrics", 2023-10-18T12:40:11.704 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.704 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.705 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.705 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "pgs,objects,bytes", 2023-10-18T12:40:11.705 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.705 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.706 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.706 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-18T12:40:11.706 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-18T12:40:11.707 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.707 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.708 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_step": { 2023-10-18T12:40:11.708 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_step", 2023-10-18T12:40:11.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:11.708 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.709 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.709 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0.5", 2023-10-18T12:40:11.709 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0.001", 2023-10-18T12:40:11.709 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "0.999", 2023-10-18T12:40:11.710 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.710 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "aggressiveness of optimization", 2023-10-18T12:40:11.710 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-18T12:40:11.711 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.711 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.711 INFO:teuthology.orchestra.run.smithi070.stdout: "end_time": { 2023-10-18T12:40:11.712 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "end_time", 2023-10-18T12:40:11.712 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.712 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.713 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.713 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "2359", 2023-10-18T12:40:11.713 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.713 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.714 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.714 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "ending time of day to automatically balance", 2023-10-18T12:40:11.714 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T12:40:11.715 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.715 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.715 INFO:teuthology.orchestra.run.smithi070.stdout: "end_weekday": { 2023-10-18T12:40:11.716 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "end_weekday", 2023-10-18T12:40:11.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:11.716 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.716 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.717 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:11.717 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0", 2023-10-18T12:40:11.717 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "6", 2023-10-18T12:40:11.718 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.718 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-18T12:40:11.718 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T12:40:11.718 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.719 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.719 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:11.720 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:11.720 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.720 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.720 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.721 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.721 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.721 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.722 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.722 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:11.722 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:11.723 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:11.723 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:11.723 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:11.724 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:11.724 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.724 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.724 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.725 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.725 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.726 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:11.726 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:11.726 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.726 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.727 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.727 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.727 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.728 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.728 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.728 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.728 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.729 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.729 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.730 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:11.730 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:11.730 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.730 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.731 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.731 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:11.731 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.732 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.732 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.732 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:11.732 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:11.733 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:11.733 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:11.733 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:11.733 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:11.734 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.734 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.734 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.735 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.735 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.735 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:11.736 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:11.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.736 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.736 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.737 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.737 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.737 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.738 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.738 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.738 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.738 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.739 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.739 INFO:teuthology.orchestra.run.smithi070.stdout: "min_score": { 2023-10-18T12:40:11.739 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "min_score", 2023-10-18T12:40:11.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:11.740 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.740 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.741 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:11.741 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.741 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.741 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.742 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-18T12:40:11.742 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.742 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.742 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": { 2023-10-18T12:40:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mode", 2023-10-18T12:40:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "upmap", 2023-10-18T12:40:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.746 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-compat", 2023-10-18T12:40:11.746 INFO:teuthology.orchestra.run.smithi070.stdout: "none", 2023-10-18T12:40:11.746 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap" 2023-10-18T12:40:11.747 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.747 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Balancer mode", 2023-10-18T12:40:11.747 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.747 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.748 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_ids": { 2023-10-18T12:40:11.749 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pool_ids", 2023-10-18T12:40:11.749 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.749 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.749 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.750 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.750 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.750 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.751 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.751 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-18T12:40:11.751 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.751 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.752 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.752 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:11.753 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:11.753 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.753 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.754 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.754 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:11.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.754 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.755 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.755 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-18T12:40:11.755 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.756 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.756 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.756 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap_max_deviation": { 2023-10-18T12:40:11.757 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "upmap_max_deviation", 2023-10-18T12:40:11.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.757 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.758 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.758 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:11.758 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:11.758 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.759 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.759 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-18T12:40:11.759 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-18T12:40:11.759 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.760 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.760 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap_max_optimizations": { 2023-10-18T12:40:11.761 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "upmap_max_optimizations", 2023-10-18T12:40:11.761 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:11.761 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.762 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.762 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:11.762 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.762 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.763 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.763 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-18T12:40:11.763 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.764 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.764 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.764 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:11.764 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:11.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:11.765 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephadm", 2023-10-18T12:40:11.766 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:11.766 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:11.766 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:11.766 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_down_multiplier": { 2023-10-18T12:40:11.767 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_down_multiplier", 2023-10-18T12:40:11.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:11.767 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.767 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.768 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3.0", 2023-10-18T12:40:11.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.768 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.769 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.769 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-18T12:40:11.769 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.769 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.770 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.770 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_refresh_rate": { 2023-10-18T12:40:11.771 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_refresh_rate", 2023-10-18T12:40:11.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.771 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.771 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.772 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "20", 2023-10-18T12:40:11.772 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.772 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.772 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.773 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-18T12:40:11.773 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.773 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.774 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.774 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_starting_port": { 2023-10-18T12:40:11.774 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_starting_port", 2023-10-18T12:40:11.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.775 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.775 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.776 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "4721", 2023-10-18T12:40:11.776 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.776 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.776 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.777 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-18T12:40:11.777 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.777 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.778 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.778 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_ptrace": { 2023-10-18T12:40:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_ptrace", 2023-10-18T12:40:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.781 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-18T12:40:11.781 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-18T12:40:11.781 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.782 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.782 INFO:teuthology.orchestra.run.smithi070.stdout: "autotune_interval": { 2023-10-18T12:40:11.782 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "autotune_interval", 2023-10-18T12:40:11.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.783 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.785 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.785 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:11.786 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.786 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.786 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.786 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-18T12:40:11.787 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.787 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.787 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.788 INFO:teuthology.orchestra.run.smithi070.stdout: "autotune_memory_target_ratio": { 2023-10-18T12:40:11.788 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "autotune_memory_target_ratio", 2023-10-18T12:40:11.788 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:11.789 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.789 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.789 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0.7", 2023-10-18T12:40:11.789 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.790 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.790 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.790 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-18T12:40:11.791 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.791 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.791 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.792 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm_log_destination": { 2023-10-18T12:40:11.792 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephadm_log_destination", 2023-10-18T12:40:11.792 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.793 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.793 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.793 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.793 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.794 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.794 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.794 INFO:teuthology.orchestra.run.smithi070.stdout: "file", 2023-10-18T12:40:11.795 INFO:teuthology.orchestra.run.smithi070.stdout: "file,syslog", 2023-10-18T12:40:11.795 INFO:teuthology.orchestra.run.smithi070.stdout: "syslog" 2023-10-18T12:40:11.795 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.795 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-18T12:40:11.796 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.796 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.796 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.797 INFO:teuthology.orchestra.run.smithi070.stdout: "cgroups_split": { 2023-10-18T12:40:11.797 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cgroups_split", 2023-10-18T12:40:11.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.798 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.798 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.798 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.799 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.799 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.799 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.799 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-18T12:40:11.800 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.800 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.800 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.801 INFO:teuthology.orchestra.run.smithi070.stdout: "config_checks_enabled": { 2023-10-18T12:40:11.801 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_checks_enabled", 2023-10-18T12:40:11.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.803 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.803 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.803 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-18T12:40:11.804 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.804 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.804 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.805 INFO:teuthology.orchestra.run.smithi070.stdout: "config_dashboard": { 2023-10-18T12:40:11.805 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_dashboard", 2023-10-18T12:40:11.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.805 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.806 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.806 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.807 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.807 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.807 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-18T12:40:11.807 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.808 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.808 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.809 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_alertmanager": { 2023-10-18T12:40:11.809 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_alertmanager", 2023-10-18T12:40:11.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.809 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.810 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-18T12:40:11.810 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.810 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.811 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.811 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:11.811 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.812 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.812 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.812 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_base": { 2023-10-18T12:40:11.813 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_base", 2023-10-18T12:40:11.813 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.813 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.814 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.814 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-18T12:40:11.814 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.814 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.815 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.815 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Container image name, without the tag", 2023-10-18T12:40:11.815 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.815 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.816 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.816 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_elasticsearch": { 2023-10-18T12:40:11.817 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_elasticsearch", 2023-10-18T12:40:11.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.817 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.817 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.818 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-18T12:40:11.818 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.818 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.818 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.819 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "elasticsearch container image", 2023-10-18T12:40:11.819 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.819 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.820 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.820 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_grafana": { 2023-10-18T12:40:11.820 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_grafana", 2023-10-18T12:40:11.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.821 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.821 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.822 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-18T12:40:11.822 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.822 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.822 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.823 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:11.823 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.823 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.824 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.824 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_haproxy": { 2023-10-18T12:40:11.824 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_haproxy", 2023-10-18T12:40:11.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.825 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.826 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-18T12:40:11.826 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.826 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.827 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.827 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "HAproxy container image", 2023-10-18T12:40:11.827 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.827 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.828 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.828 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_agent": { 2023-10-18T12:40:11.828 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_agent", 2023-10-18T12:40:11.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.829 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.829 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.829 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-18T12:40:11.830 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.830 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.830 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.831 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger agent container image", 2023-10-18T12:40:11.831 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.831 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.831 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.832 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_collector": { 2023-10-18T12:40:11.832 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_collector", 2023-10-18T12:40:11.832 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.833 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.833 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.833 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-18T12:40:11.834 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.834 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.834 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.834 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger collector container image", 2023-10-18T12:40:11.835 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.835 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.835 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.836 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_query": { 2023-10-18T12:40:11.836 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_query", 2023-10-18T12:40:11.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.837 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.837 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.837 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-18T12:40:11.838 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.838 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.838 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.838 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger query container image", 2023-10-18T12:40:11.839 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.839 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.839 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.840 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_keepalived": { 2023-10-18T12:40:11.840 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_keepalived", 2023-10-18T12:40:11.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.841 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.841 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.841 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-18T12:40:11.841 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.842 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.842 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.842 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Keepalived container image", 2023-10-18T12:40:11.843 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.843 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.843 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.844 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_loki": { 2023-10-18T12:40:11.844 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_loki", 2023-10-18T12:40:11.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.845 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.845 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.845 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-18T12:40:11.845 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.846 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.846 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.846 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Loki container image", 2023-10-18T12:40:11.847 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.847 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.847 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.848 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_node_exporter": { 2023-10-18T12:40:11.848 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_node_exporter", 2023-10-18T12:40:11.848 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.849 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.849 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.849 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-18T12:40:11.849 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.850 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.850 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.850 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:11.850 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.851 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.851 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.852 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_nvmeof": { 2023-10-18T12:40:11.852 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_nvmeof", 2023-10-18T12:40:11.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.852 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.853 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.853 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-18T12:40:11.853 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.853 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.854 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.854 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Nvme-of container image", 2023-10-18T12:40:11.854 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.855 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.855 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.856 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_prometheus": { 2023-10-18T12:40:11.856 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_prometheus", 2023-10-18T12:40:11.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.856 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.857 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.857 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-18T12:40:11.857 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.857 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.858 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.858 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:11.858 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.859 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.859 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.859 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_promtail": { 2023-10-18T12:40:11.860 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_promtail", 2023-10-18T12:40:11.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.860 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.861 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.861 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-18T12:40:11.861 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Promtail container image", 2023-10-18T12:40:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_snmp_gateway": { 2023-10-18T12:40:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_snmp_gateway", 2023-10-18T12:40:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.865 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-18T12:40:11.865 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.866 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.866 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.866 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SNMP Gateway container image", 2023-10-18T12:40:11.866 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.867 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.867 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.868 INFO:teuthology.orchestra.run.smithi070.stdout: "container_init": { 2023-10-18T12:40:11.868 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_init", 2023-10-18T12:40:11.868 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.868 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.869 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.869 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.869 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.870 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.870 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.870 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Run podman/docker with `--init`", 2023-10-18T12:40:11.871 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.871 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.871 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.872 INFO:teuthology.orchestra.run.smithi070.stdout: "daemon_cache_timeout": { 2023-10-18T12:40:11.872 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "daemon_cache_timeout", 2023-10-18T12:40:11.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.873 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.873 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.873 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:11.873 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.874 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.874 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.874 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-18T12:40:11.875 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.875 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.875 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.876 INFO:teuthology.orchestra.run.smithi070.stdout: "default_cephadm_command_timeout": { 2023-10-18T12:40:11.876 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "default_cephadm_command_timeout", 2023-10-18T12:40:11.877 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.877 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.877 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.878 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "900", 2023-10-18T12:40:11.878 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.878 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.878 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.879 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-18T12:40:11.879 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.879 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.880 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.880 INFO:teuthology.orchestra.run.smithi070.stdout: "default_registry": { 2023-10-18T12:40:11.880 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "default_registry", 2023-10-18T12:40:11.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.881 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.881 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.882 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io", 2023-10-18T12:40:11.882 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.882 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.883 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.883 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-18T12:40:11.883 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.883 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.884 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.884 INFO:teuthology.orchestra.run.smithi070.stdout: "device_cache_timeout": { 2023-10-18T12:40:11.885 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_cache_timeout", 2023-10-18T12:40:11.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.885 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.885 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.886 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1800", 2023-10-18T12:40:11.886 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.886 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.887 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.887 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache device inventory", 2023-10-18T12:40:11.887 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.887 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: "device_enhanced_scan": { 2023-10-18T12:40:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_enhanced_scan", 2023-10-18T12:40:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-18T12:40:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: "facts_cache_timeout": { 2023-10-18T12:40:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "facts_cache_timeout", 2023-10-18T12:40:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache host facts data", 2023-10-18T12:40:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: "host_check_interval": { 2023-10-18T12:40:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "host_check_interval", 2023-10-18T12:40:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.899 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.899 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to perform a host check", 2023-10-18T12:40:11.899 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.899 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.900 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.900 INFO:teuthology.orchestra.run.smithi070.stdout: "inventory_list_all": { 2023-10-18T12:40:11.901 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "inventory_list_all", 2023-10-18T12:40:11.901 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.901 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.902 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.902 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.902 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.902 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.903 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.903 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-18T12:40:11.903 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.904 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.904 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.904 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:11.905 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:11.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.905 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.906 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.906 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.906 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.906 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.907 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.907 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:11.907 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:11.907 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:11.908 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:11.908 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:11.908 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:11.909 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.909 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.909 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.910 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.910 INFO:teuthology.orchestra.run.smithi070.stdout: "log_refresh_metadata": { 2023-10-18T12:40:11.911 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_refresh_metadata", 2023-10-18T12:40:11.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.911 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.912 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.912 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.912 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.913 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.913 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.913 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-18T12:40:11.913 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.914 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.914 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.915 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:11.915 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:11.915 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.916 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.916 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.916 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.916 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.917 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.917 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.917 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-18T12:40:11.917 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.918 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.918 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.919 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:11.919 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:11.919 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.919 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.920 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.920 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:11.920 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.920 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.921 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.921 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:11.921 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:11.922 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:11.922 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:11.922 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:11.923 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:11.923 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.923 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.923 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.924 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.924 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.924 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:11.925 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:11.925 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.925 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.926 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:11.926 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.926 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.927 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.927 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.927 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:11.928 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.928 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.928 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.929 INFO:teuthology.orchestra.run.smithi070.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-18T12:40:11.929 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-18T12:40:11.929 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.930 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.930 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.930 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.931 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.931 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.931 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.931 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-18T12:40:11.932 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.932 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.932 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.933 INFO:teuthology.orchestra.run.smithi070.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-18T12:40:11.933 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-18T12:40:11.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.934 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.934 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.934 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "*", 2023-10-18T12:40:11.934 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.935 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.935 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.935 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-18T12:40:11.936 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.936 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.936 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.937 INFO:teuthology.orchestra.run.smithi070.stdout: "max_count_per_host": { 2023-10-18T12:40:11.937 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_count_per_host", 2023-10-18T12:40:11.937 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.938 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.938 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.938 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:11.939 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.939 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.939 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.939 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "max number of daemons per service per host", 2023-10-18T12:40:11.940 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.940 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.940 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.941 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd_draining_count": { 2023-10-18T12:40:11.941 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_osd_draining_count", 2023-10-18T12:40:11.941 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.942 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.942 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.942 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:11.943 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.943 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.943 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.944 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-18T12:40:11.944 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.944 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.944 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.945 INFO:teuthology.orchestra.run.smithi070.stdout: "migration_current": { 2023-10-18T12:40:11.945 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "migration_current", 2023-10-18T12:40:11.945 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.946 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.946 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.946 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.947 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.947 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.947 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.948 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "internal - do not modify", 2023-10-18T12:40:11.948 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.948 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.948 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": { 2023-10-18T12:40:11.949 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mode", 2023-10-18T12:40:11.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.950 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.950 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.950 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "root", 2023-10-18T12:40:11.951 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.951 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.951 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:11.952 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm-package", 2023-10-18T12:40:11.952 INFO:teuthology.orchestra.run.smithi070.stdout: "root" 2023-10-18T12:40:11.952 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:11.952 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "mode for remote execution of cephadm", 2023-10-18T12:40:11.953 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.953 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.954 INFO:teuthology.orchestra.run.smithi070.stdout: "prometheus_alerts_path": { 2023-10-18T12:40:11.954 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prometheus_alerts_path", 2023-10-18T12:40:11.954 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.955 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.955 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.955 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-18T12:40:11.956 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.956 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.956 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.957 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-18T12:40:11.957 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.957 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.957 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.958 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_insecure": { 2023-10-18T12:40:11.958 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_insecure", 2023-10-18T12:40:11.959 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.959 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.959 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.960 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.960 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.960 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.961 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.961 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-18T12:40:11.961 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_password": { 2023-10-18T12:40:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_password", 2023-10-18T12:40:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.964 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.964 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.964 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.964 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.965 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.965 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.965 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-18T12:40:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.967 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_url": { 2023-10-18T12:40:11.967 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_url", 2023-10-18T12:40:11.967 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.968 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.968 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.968 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.969 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.969 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.969 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.969 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-18T12:40:11.970 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.970 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.970 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.971 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_username": { 2023-10-18T12:40:11.971 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_username", 2023-10-18T12:40:11.971 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.972 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.972 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.972 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.972 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.973 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.973 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.973 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-18T12:40:11.973 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.974 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.974 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.975 INFO:teuthology.orchestra.run.smithi070.stdout: "secure_monitoring_stack": { 2023-10-18T12:40:11.975 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "secure_monitoring_stack", 2023-10-18T12:40:11.975 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.975 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.976 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.976 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.976 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.977 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.977 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.977 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-18T12:40:11.977 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.978 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.978 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.979 INFO:teuthology.orchestra.run.smithi070.stdout: "service_discovery_port": { 2023-10-18T12:40:11.979 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "service_discovery_port", 2023-10-18T12:40:11.979 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:11.979 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.980 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.980 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8765", 2023-10-18T12:40:11.980 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.981 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.981 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.981 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "cephadm service discovery port", 2023-10-18T12:40:11.981 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.982 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.982 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.983 INFO:teuthology.orchestra.run.smithi070.stdout: "ssh_config_file": { 2023-10-18T12:40:11.983 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssh_config_file", 2023-10-18T12:40:11.983 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:11.983 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.984 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.984 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:11.984 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.985 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.985 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.985 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-18T12:40:11.985 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.986 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.986 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.986 INFO:teuthology.orchestra.run.smithi070.stdout: "use_agent": { 2023-10-18T12:40:11.987 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "use_agent", 2023-10-18T12:40:11.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.987 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.988 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.988 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:11.988 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.988 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.989 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.989 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-18T12:40:11.989 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.990 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.990 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.993 INFO:teuthology.orchestra.run.smithi070.stdout: "use_repo_digest": { 2023-10-18T12:40:11.993 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "use_repo_digest", 2023-10-18T12:40:11.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.994 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.994 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.995 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.995 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.995 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.995 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:11.996 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-18T12:40:11.996 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:11.996 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:11.997 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:11.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:11.997 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_failed_host_check": { 2023-10-18T12:40:11.997 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_failed_host_check", 2023-10-18T12:40:11.998 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:11.998 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:11.998 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:11.999 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:11.999 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:11.999 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:11.999 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.000 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if the host check fails", 2023-10-18T12:40:12.000 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.001 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.001 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_stray_daemons": { 2023-10-18T12:40:12.001 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_stray_daemons", 2023-10-18T12:40:12.002 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.002 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.002 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.002 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.003 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.003 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.003 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.003 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-18T12:40:12.004 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.004 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.004 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.005 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_stray_hosts": { 2023-10-18T12:40:12.005 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_stray_hosts", 2023-10-18T12:40:12.005 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.006 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.006 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.006 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.007 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.007 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.007 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-18T12:40:12.008 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.008 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.008 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.009 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.009 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.010 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crash", 2023-10-18T12:40:12.010 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.010 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.011 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.011 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.011 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.011 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.012 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.012 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.012 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.012 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.013 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.013 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.013 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.014 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.014 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.014 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.014 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.015 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.015 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.015 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.016 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.016 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.016 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.017 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.017 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.018 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.018 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.018 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.019 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.019 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.019 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.020 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.020 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.020 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.021 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.021 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.022 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.022 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.022 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.022 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.023 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.023 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.023 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.024 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.024 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.024 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.024 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.025 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.025 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.025 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.025 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.026 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.026 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.027 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.027 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.027 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.028 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.028 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.028 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.028 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.029 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.029 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.030 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.030 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.030 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.031 INFO:teuthology.orchestra.run.smithi070.stdout: "retain_interval": { 2023-10-18T12:40:12.031 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "retain_interval", 2023-10-18T12:40:12.031 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.031 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.032 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.032 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "31536000", 2023-10-18T12:40:12.032 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.033 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.033 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.033 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-18T12:40:12.033 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.034 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.034 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.035 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_recent_interval": { 2023-10-18T12:40:12.035 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_recent_interval", 2023-10-18T12:40:12.035 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.035 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.036 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.036 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1209600", 2023-10-18T12:40:12.036 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.036 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.037 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.037 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-18T12:40:12.037 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.038 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.038 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.039 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "dashboard", 2023-10-18T12:40:12.039 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.040 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.040 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.040 INFO:teuthology.orchestra.run.smithi070.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-18T12:40:12.041 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-18T12:40:12.041 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.041 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.041 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.042 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:12.042 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.042 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.043 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.043 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.043 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.043 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.044 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.044 INFO:teuthology.orchestra.run.smithi070.stdout: "ALERTMANAGER_API_HOST": { 2023-10-18T12:40:12.044 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-18T12:40:12.045 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.045 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.045 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.046 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.046 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.046 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.046 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.047 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.047 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.047 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.048 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.048 INFO:teuthology.orchestra.run.smithi070.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-18T12:40:12.048 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-18T12:40:12.049 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.049 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.049 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.050 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.050 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.050 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.050 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.051 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.051 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.051 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.052 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.052 INFO:teuthology.orchestra.run.smithi070.stdout: "AUDIT_API_ENABLED": { 2023-10-18T12:40:12.052 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "AUDIT_API_ENABLED", 2023-10-18T12:40:12.053 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.053 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.053 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.054 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.054 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.054 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.055 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.055 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.055 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.055 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.056 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.056 INFO:teuthology.orchestra.run.smithi070.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-18T12:40:12.056 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-18T12:40:12.057 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.057 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.057 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.057 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.058 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.058 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.058 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.059 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.059 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.059 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.059 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.060 INFO:teuthology.orchestra.run.smithi070.stdout: "ENABLE_BROWSABLE_API": { 2023-10-18T12:40:12.060 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-18T12:40:12.061 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.061 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.061 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.061 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.062 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.062 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.062 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.063 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.063 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.063 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.064 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.064 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-18T12:40:12.065 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-18T12:40:12.065 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.065 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.065 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.066 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.066 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.066 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.067 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.067 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.067 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.068 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.068 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.068 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-18T12:40:12.069 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-18T12:40:12.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.069 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.070 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.070 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.070 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.070 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.071 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.071 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.071 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.071 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.072 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.072 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-18T12:40:12.073 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-18T12:40:12.073 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.073 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.073 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.074 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.074 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.074 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.075 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.075 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.075 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.076 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.076 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.076 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-18T12:40:12.077 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-18T12:40:12.077 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.077 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.077 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.078 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.078 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.078 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.079 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.079 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.079 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.079 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.080 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.080 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-18T12:40:12.080 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-18T12:40:12.081 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.081 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.081 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.082 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.082 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.082 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.083 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.083 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.083 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.083 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.084 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.084 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-18T12:40:12.085 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-18T12:40:12.085 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.085 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.085 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.086 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.086 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.086 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.086 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.087 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.087 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.087 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.088 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.088 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-18T12:40:12.089 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-18T12:40:12.089 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.089 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.090 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.090 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.090 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.091 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.091 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.091 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.091 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.092 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.092 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.093 INFO:teuthology.orchestra.run.smithi070.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-18T12:40:12.093 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-18T12:40:12.093 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.094 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.094 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.094 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.095 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.095 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.095 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.096 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.096 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.096 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.097 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.097 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_PASSWORD": { 2023-10-18T12:40:12.097 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-18T12:40:12.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.098 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.098 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.098 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:12.099 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.099 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.099 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.100 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.100 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.100 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.100 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.101 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-18T12:40:12.101 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-18T12:40:12.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.102 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.102 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.102 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.103 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.103 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.103 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.104 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.104 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.104 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.104 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.105 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_URL": { 2023-10-18T12:40:12.105 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_URL", 2023-10-18T12:40:12.105 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.106 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.106 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.106 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.106 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.107 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.107 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.107 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.108 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.108 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.108 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.109 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_USERNAME": { 2023-10-18T12:40:12.109 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-18T12:40:12.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.110 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.110 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.110 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:12.111 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.111 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.111 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.111 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.112 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.112 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.112 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.113 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-18T12:40:12.113 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-18T12:40:12.114 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.114 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.114 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.115 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.115 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.115 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.116 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.116 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.116 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.117 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.117 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.118 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-18T12:40:12.118 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-18T12:40:12.118 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.118 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.119 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.119 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.119 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.120 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.120 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.120 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.121 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.121 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.121 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.122 INFO:teuthology.orchestra.run.smithi070.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-18T12:40:12.122 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-18T12:40:12.123 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.123 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.123 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.124 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.124 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.124 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.125 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.125 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.125 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.126 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.126 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.127 INFO:teuthology.orchestra.run.smithi070.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-18T12:40:12.127 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-18T12:40:12.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.128 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.128 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.128 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.129 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.129 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.129 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.129 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.130 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.130 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.130 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.131 INFO:teuthology.orchestra.run.smithi070.stdout: "PROMETHEUS_API_HOST": { 2023-10-18T12:40:12.131 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-18T12:40:12.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.132 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.132 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.132 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.132 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.133 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.133 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.133 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.134 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.134 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.134 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.135 INFO:teuthology.orchestra.run.smithi070.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-18T12:40:12.135 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-18T12:40:12.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.136 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.136 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.136 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.136 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.137 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.137 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.137 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.138 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.138 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.138 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.139 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-18T12:40:12.139 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-18T12:40:12.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-18T12:40:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-18T12:40:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-18T12:40:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-18T12:40:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-18T12:40:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-18T12:40:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-18T12:40:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-18T12:40:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-18T12:40:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-18T12:40:12.160 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.160 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.160 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.160 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.161 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.161 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.161 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.161 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.162 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.162 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.163 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-18T12:40:12.163 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-18T12:40:12.163 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.164 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.164 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.164 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.165 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.165 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.165 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.165 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.166 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.166 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.166 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.167 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_ENABLED": { 2023-10-18T12:40:12.167 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-18T12:40:12.167 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.168 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.168 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.168 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.169 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.169 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.169 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.169 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.170 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.170 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.170 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.171 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-18T12:40:12.171 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-18T12:40:12.172 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.172 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.172 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.172 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-18T12:40:12.173 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.173 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.173 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.174 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.174 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.174 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.174 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-18T12:40:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-18T12:40:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:12.177 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.177 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.177 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.178 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.178 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.178 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.179 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.179 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-18T12:40:12.179 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-18T12:40:12.180 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.180 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.180 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.180 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8", 2023-10-18T12:40:12.181 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.181 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.181 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.182 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.182 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.182 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.182 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.183 INFO:teuthology.orchestra.run.smithi070.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-18T12:40:12.183 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-18T12:40:12.184 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.184 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.184 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.185 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "45", 2023-10-18T12:40:12.185 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.185 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.185 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.186 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.186 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.187 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.187 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_ACCESS_KEY": { 2023-10-18T12:40:12.188 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-18T12:40:12.188 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.188 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.188 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.189 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.189 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.189 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.190 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.190 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.190 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.190 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.191 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.191 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-18T12:40:12.192 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-18T12:40:12.192 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.192 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.192 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.193 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:12.193 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.193 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.193 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.194 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.194 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.195 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.195 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_SECRET_KEY": { 2023-10-18T12:40:12.195 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-18T12:40:12.197 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.197 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.198 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.198 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.198 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.199 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.199 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.199 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.200 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.200 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.201 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_SSL_VERIFY": { 2023-10-18T12:40:12.201 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-18T12:40:12.201 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.202 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.202 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.202 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.203 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.203 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.203 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.203 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.204 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.204 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.204 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.205 INFO:teuthology.orchestra.run.smithi070.stdout: "UNSAFE_TLS_v1_2": { 2023-10-18T12:40:12.205 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-18T12:40:12.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.206 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.206 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.206 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.207 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.207 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.207 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.208 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.208 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.208 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.208 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.209 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-18T12:40:12.209 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-18T12:40:12.209 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.210 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.210 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.210 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:12.211 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.211 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.211 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.211 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.212 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.212 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.212 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.213 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-18T12:40:12.213 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-18T12:40:12.213 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.214 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.214 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.214 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:12.215 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.215 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.215 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.216 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.216 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.216 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.216 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.217 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-18T12:40:12.217 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-18T12:40:12.218 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.218 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.218 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.219 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:12.219 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.219 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.219 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.220 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.220 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.220 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.221 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.221 INFO:teuthology.orchestra.run.smithi070.stdout: "cross_origin_url": { 2023-10-18T12:40:12.221 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cross_origin_url", 2023-10-18T12:40:12.222 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.222 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.222 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.223 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.223 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.223 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.223 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.224 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.224 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.224 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.225 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.225 INFO:teuthology.orchestra.run.smithi070.stdout: "crt_file": { 2023-10-18T12:40:12.225 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crt_file", 2023-10-18T12:40:12.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.226 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.226 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.227 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.227 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.227 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.227 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.228 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.228 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.228 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.228 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.229 INFO:teuthology.orchestra.run.smithi070.stdout: "debug": { 2023-10-18T12:40:12.229 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "debug", 2023-10-18T12:40:12.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.230 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.230 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.230 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.231 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.231 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.231 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.232 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable/disable debug options", 2023-10-18T12:40:12.232 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.232 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.232 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.233 INFO:teuthology.orchestra.run.smithi070.stdout: "jwt_token_ttl": { 2023-10-18T12:40:12.233 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "jwt_token_ttl", 2023-10-18T12:40:12.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.234 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.234 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.234 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "28800", 2023-10-18T12:40:12.235 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.235 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.235 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.236 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.236 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.236 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.236 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.237 INFO:teuthology.orchestra.run.smithi070.stdout: "key_file": { 2023-10-18T12:40:12.237 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "key_file", 2023-10-18T12:40:12.238 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.238 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.239 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.239 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.239 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.240 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.240 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.240 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.240 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.241 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.241 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.242 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.242 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.242 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.242 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.243 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.243 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.243 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.244 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.244 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.244 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.244 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.245 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.245 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.245 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.246 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.246 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.246 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.247 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.247 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.247 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.248 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.248 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.248 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.248 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.249 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.249 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.249 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.250 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.250 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.250 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.250 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.251 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.251 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.251 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.252 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.252 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.252 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.253 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.253 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.253 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.254 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.254 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.254 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.254 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.255 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.255 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.255 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.255 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.256 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.256 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.256 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.257 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.257 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.257 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.258 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.258 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.258 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.259 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.259 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.259 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.260 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.260 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.260 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.261 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.261 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.261 INFO:teuthology.orchestra.run.smithi070.stdout: "motd": { 2023-10-18T12:40:12.262 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "motd", 2023-10-18T12:40:12.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.262 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.263 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.263 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.263 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.263 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.264 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.264 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "The message of the day", 2023-10-18T12:40:12.264 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.264 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.265 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.265 INFO:teuthology.orchestra.run.smithi070.stdout: "redirect_resolve_ip_addr": { 2023-10-18T12:40:12.266 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "redirect_resolve_ip_addr", 2023-10-18T12:40:12.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.266 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.266 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.267 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.267 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.267 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.268 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.268 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.268 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.268 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.269 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.269 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:12.269 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:12.270 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.270 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.270 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.271 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "::", 2023-10-18T12:40:12.271 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.271 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.272 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.272 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.272 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.272 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.273 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.273 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:12.273 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:12.274 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.274 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.274 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.275 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8080", 2023-10-18T12:40:12.275 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.275 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.276 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.276 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.276 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.276 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.277 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.277 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl": { 2023-10-18T12:40:12.277 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl", 2023-10-18T12:40:12.278 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.278 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.278 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.279 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.279 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.279 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.279 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.280 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.280 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.280 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.281 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.281 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl_server_port": { 2023-10-18T12:40:12.282 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl_server_port", 2023-10-18T12:40:12.282 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.282 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.282 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.283 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8443", 2023-10-18T12:40:12.283 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.283 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.283 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.284 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.284 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.284 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.284 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.285 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_behaviour": { 2023-10-18T12:40:12.285 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_behaviour", 2023-10-18T12:40:12.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.286 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.286 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "redirect", 2023-10-18T12:40:12.287 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.287 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.288 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.288 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.288 INFO:teuthology.orchestra.run.smithi070.stdout: "redirect" 2023-10-18T12:40:12.288 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.289 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.289 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.289 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.290 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_error_status_code": { 2023-10-18T12:40:12.290 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_error_status_code", 2023-10-18T12:40:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "500", 2023-10-18T12:40:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "400", 2023-10-18T12:40:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "599", 2023-10-18T12:40:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.293 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.293 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.293 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.293 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.294 INFO:teuthology.orchestra.run.smithi070.stdout: "url_prefix": { 2023-10-18T12:40:12.294 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "url_prefix", 2023-10-18T12:40:12.295 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.295 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.295 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.295 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.296 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.296 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.296 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.296 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.297 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.297 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.297 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.298 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.298 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.299 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "devicehealth", 2023-10-18T12:40:12.299 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.299 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_monitoring": { 2023-10-18T12:40:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enable_monitoring", 2023-10-18T12:40:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "monitor device health metrics", 2023-10-18T12:40:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.304 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.304 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.304 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.305 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.305 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.305 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.306 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.306 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.306 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.306 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.307 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.307 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.307 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.307 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.308 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.308 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.308 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.309 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.309 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.309 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.310 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.310 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.311 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.311 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.311 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.311 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.312 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.312 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.312 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.312 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.313 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.313 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.313 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.314 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.314 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.315 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.315 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.315 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.315 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.316 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.316 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.316 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.316 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.317 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.317 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.317 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.318 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.318 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.318 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.319 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.319 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.319 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.320 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.320 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.321 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.321 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.321 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.322 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.322 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.322 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.322 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.323 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.323 INFO:teuthology.orchestra.run.smithi070.stdout: "mark_out_threshold": { 2023-10-18T12:40:12.323 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mark_out_threshold", 2023-10-18T12:40:12.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.324 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.324 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.324 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "2419200", 2023-10-18T12:40:12.325 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.325 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.325 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.325 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-18T12:40:12.326 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.326 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.326 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": { 2023-10-18T12:40:12.327 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pool_name", 2023-10-18T12:40:12.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.328 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.328 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.328 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "device_health_metrics", 2023-10-18T12:40:12.329 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.329 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.329 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.330 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-18T12:40:12.330 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.330 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.330 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.331 INFO:teuthology.orchestra.run.smithi070.stdout: "retention_period": { 2023-10-18T12:40:12.331 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "retention_period", 2023-10-18T12:40:12.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.332 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.332 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.332 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15552000", 2023-10-18T12:40:12.333 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.333 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.333 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.333 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long to retain device health metrics", 2023-10-18T12:40:12.334 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.334 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.334 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.335 INFO:teuthology.orchestra.run.smithi070.stdout: "scrape_frequency": { 2023-10-18T12:40:12.335 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "scrape_frequency", 2023-10-18T12:40:12.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.336 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.336 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.336 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "86400", 2023-10-18T12:40:12.337 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.337 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.337 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.337 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-18T12:40:12.338 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.338 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.338 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.339 INFO:teuthology.orchestra.run.smithi070.stdout: "self_heal": { 2023-10-18T12:40:12.339 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "self_heal", 2023-10-18T12:40:12.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.340 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.340 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.340 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.340 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.341 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.341 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.341 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-18T12:40:12.342 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.342 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.342 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.343 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:12.343 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:12.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.344 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.344 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.344 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:12.344 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.345 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.345 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.345 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to wake up and check device health", 2023-10-18T12:40:12.346 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.346 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.346 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.347 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_threshold": { 2023-10-18T12:40:12.347 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_threshold", 2023-10-18T12:40:12.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.347 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.348 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.348 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "7257600", 2023-10-18T12:40:12.348 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.349 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.349 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.349 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-18T12:40:12.349 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.350 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.350 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.351 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "diskprediction_local", 2023-10-18T12:40:12.352 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.352 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.352 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.352 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.353 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.353 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.353 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.354 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.354 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.354 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.354 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.355 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.355 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.355 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.355 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.356 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.356 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.356 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.357 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.357 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.357 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.357 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.358 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.358 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.359 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.359 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.359 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.360 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.360 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.360 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.360 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.361 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.361 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.361 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.361 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.362 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.362 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.363 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.363 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.363 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.363 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.364 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.364 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.364 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.365 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.365 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.365 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.365 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.366 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.366 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.366 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.367 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.367 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.367 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.367 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: "predict_interval": { 2023-10-18T12:40:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "predict_interval", 2023-10-18T12:40:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "86400", 2023-10-18T12:40:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: "predictor_model": { 2023-10-18T12:40:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "predictor_model", 2023-10-18T12:40:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "prophetstor", 2023-10-18T12:40:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.380 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:12.380 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:12.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.381 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.381 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.381 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:12.382 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.382 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.382 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.383 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.383 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.383 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.383 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.385 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "influx", 2023-10-18T12:40:12.385 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": false, 2023-10-18T12:40:12.385 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "influxdb python module not found", 2023-10-18T12:40:12.386 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.386 INFO:teuthology.orchestra.run.smithi070.stdout: "batch_size": { 2023-10-18T12:40:12.386 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "batch_size", 2023-10-18T12:40:12.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.387 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.387 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.387 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5000", 2023-10-18T12:40:12.387 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.388 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.388 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.388 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-18T12:40:12.389 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.389 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.389 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.390 INFO:teuthology.orchestra.run.smithi070.stdout: "database": { 2023-10-18T12:40:12.390 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "database", 2023-10-18T12:40:12.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.391 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.391 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.391 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "ceph", 2023-10-18T12:40:12.391 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.392 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.392 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.392 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-18T12:40:12.392 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.393 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.393 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.394 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": { 2023-10-18T12:40:12.394 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "hostname", 2023-10-18T12:40:12.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.394 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.395 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.395 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.395 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.396 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.396 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.396 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB server hostname", 2023-10-18T12:40:12.396 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.397 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.397 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.398 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:12.398 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:12.398 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.398 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.399 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.399 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "30", 2023-10-18T12:40:12.399 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "5", 2023-10-18T12:40:12.400 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.400 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.400 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-18T12:40:12.400 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.401 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.401 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.402 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.402 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.402 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.405 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.405 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.406 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.406 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.406 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.406 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.407 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.407 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.407 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.408 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.408 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.408 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.408 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.409 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.409 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.409 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.410 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.410 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.410 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.411 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.411 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.411 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.411 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.412 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.412 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.412 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.413 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.413 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.413 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.414 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.414 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.414 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.414 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.415 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.415 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.415 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.416 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.416 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.416 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.416 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.417 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.417 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.417 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.417 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.418 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.418 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.418 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.419 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.419 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.419 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.420 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.420 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.420 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.421 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.421 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.421 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.422 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.422 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.422 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.422 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.423 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.423 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.423 INFO:teuthology.orchestra.run.smithi070.stdout: "password": { 2023-10-18T12:40:12.424 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "password", 2023-10-18T12:40:12.424 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.424 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.425 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.425 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.425 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.425 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.426 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.426 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "password of InfluxDB server user", 2023-10-18T12:40:12.426 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.427 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.427 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.427 INFO:teuthology.orchestra.run.smithi070.stdout: "port": { 2023-10-18T12:40:12.428 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "port", 2023-10-18T12:40:12.428 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.428 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.429 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8086", 2023-10-18T12:40:12.429 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.429 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.429 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.430 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB server port", 2023-10-18T12:40:12.430 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.431 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.431 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl": { 2023-10-18T12:40:12.431 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl", 2023-10-18T12:40:12.432 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.432 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.432 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.433 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "false", 2023-10-18T12:40:12.433 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.433 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.433 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.434 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T12:40:12.434 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.434 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.434 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.435 INFO:teuthology.orchestra.run.smithi070.stdout: "threads": { 2023-10-18T12:40:12.435 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "threads", 2023-10-18T12:40:12.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.436 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.436 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:12.437 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:12.437 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "32", 2023-10-18T12:40:12.437 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.437 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-18T12:40:12.438 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.438 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.438 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.439 INFO:teuthology.orchestra.run.smithi070.stdout: "username": { 2023-10-18T12:40:12.439 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "username", 2023-10-18T12:40:12.439 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "username of InfluxDB server user", 2023-10-18T12:40:12.442 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.442 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.442 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.443 INFO:teuthology.orchestra.run.smithi070.stdout: "verify_ssl": { 2023-10-18T12:40:12.443 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "verify_ssl", 2023-10-18T12:40:12.443 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.444 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.444 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.444 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "true", 2023-10-18T12:40:12.444 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.445 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.445 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.445 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T12:40:12.446 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.446 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.446 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.448 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "insights", 2023-10-18T12:40:12.448 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.448 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.449 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.449 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.449 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.450 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.450 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.450 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.450 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.451 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.451 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.451 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.451 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.452 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.452 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.452 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.453 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.453 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.453 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.453 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.454 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.454 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.454 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.455 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.455 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.456 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.456 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.456 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.456 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.457 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.457 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.457 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.457 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.458 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.458 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.459 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.459 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.459 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.460 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.460 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.460 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.461 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.461 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.461 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.461 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.462 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.462 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.462 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.463 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.463 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.463 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.463 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.464 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.464 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.464 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.465 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.465 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.465 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.466 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.466 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.466 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.467 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.467 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.467 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.467 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.468 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.469 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "iostat", 2023-10-18T12:40:12.469 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.470 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.470 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.470 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.470 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.471 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.472 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.472 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.472 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.472 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.473 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.473 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.473 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.474 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.474 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.474 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.474 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.475 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.475 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.475 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.476 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.476 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.476 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.477 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.477 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.478 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.478 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.478 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.478 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.479 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.479 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.479 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.480 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.480 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.480 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.481 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.482 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.482 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.482 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.482 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.483 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.483 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.483 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.484 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.484 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.484 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.484 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.485 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.485 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.485 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.486 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.486 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.487 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.487 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.487 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.487 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.488 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.488 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.488 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.489 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.489 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.489 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.489 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.490 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.491 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "k8sevents", 2023-10-18T12:40:12.491 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.492 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.492 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.492 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_event_retention_days": { 2023-10-18T12:40:12.492 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ceph_event_retention_days", 2023-10-18T12:40:12.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.493 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.493 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.494 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "7", 2023-10-18T12:40:12.494 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.494 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.494 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.495 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-18T12:40:12.495 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.495 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.495 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.496 INFO:teuthology.orchestra.run.smithi070.stdout: "config_check_secs": { 2023-10-18T12:40:12.496 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_check_secs", 2023-10-18T12:40:12.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.497 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.497 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:12.498 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "10", 2023-10-18T12:40:12.498 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.498 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.498 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-18T12:40:12.499 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.499 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.499 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.500 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.500 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.501 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.501 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.501 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.501 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.502 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.502 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.502 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.503 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.503 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.503 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.503 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.504 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.504 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.504 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.505 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.505 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.506 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.506 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.506 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.507 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.507 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.507 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.507 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.508 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.508 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.508 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.508 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.509 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.509 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.509 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.510 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.510 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.510 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.511 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.511 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.511 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.511 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.514 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.514 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.514 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.515 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.515 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.515 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.516 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.516 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.516 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.517 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.517 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.517 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.517 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.518 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.518 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.518 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.519 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.519 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.520 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "localpool", 2023-10-18T12:40:12.520 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.521 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.521 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.521 INFO:teuthology.orchestra.run.smithi070.stdout: "failure_domain": { 2023-10-18T12:40:12.522 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "failure_domain", 2023-10-18T12:40:12.522 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.522 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.522 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.523 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "host", 2023-10-18T12:40:12.523 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.523 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.524 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.524 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "failure domain for any created local pool", 2023-10-18T12:40:12.524 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-18T12:40:12.524 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.525 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.525 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.525 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.526 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.526 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.526 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.527 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.527 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.527 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.527 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.528 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.528 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.528 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.528 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.529 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.529 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.529 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.530 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.530 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.530 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.530 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.531 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.531 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.532 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.532 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.533 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.533 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.533 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.534 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.534 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.534 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.534 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.535 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.535 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.536 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.536 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.536 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.536 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.537 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.537 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.537 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.538 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.538 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.538 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.538 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.539 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.539 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.539 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.539 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.540 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.540 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.541 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.541 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.541 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.542 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.542 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.542 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.543 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.543 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.543 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.543 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.544 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.544 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.544 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": { 2023-10-18T12:40:12.545 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "min_size", 2023-10-18T12:40:12.545 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.545 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.546 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.546 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.546 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.547 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.547 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.547 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default min_size for any created local pool", 2023-10-18T12:40:12.547 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-18T12:40:12.548 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.548 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.549 INFO:teuthology.orchestra.run.smithi070.stdout: "num_rep": { 2023-10-18T12:40:12.549 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "num_rep", 2023-10-18T12:40:12.549 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.549 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.550 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.550 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3", 2023-10-18T12:40:12.550 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.550 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.551 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.551 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default replica count for any created local pool", 2023-10-18T12:40:12.551 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.552 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.552 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.553 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": { 2023-10-18T12:40:12.553 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pg_num", 2023-10-18T12:40:12.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.553 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.554 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.554 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "128", 2023-10-18T12:40:12.554 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.555 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.555 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.555 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default pg_num for any created local pool", 2023-10-18T12:40:12.555 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.556 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.556 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.556 INFO:teuthology.orchestra.run.smithi070.stdout: "prefix": { 2023-10-18T12:40:12.557 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prefix", 2023-10-18T12:40:12.557 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.557 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.558 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.558 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.558 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.558 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.559 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.559 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "name prefix for any created local pool", 2023-10-18T12:40:12.559 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.559 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.560 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.560 INFO:teuthology.orchestra.run.smithi070.stdout: "subtree": { 2023-10-18T12:40:12.561 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "subtree", 2023-10-18T12:40:12.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.561 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.561 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.562 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "rack", 2023-10-18T12:40:12.562 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.562 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.563 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.563 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-18T12:40:12.563 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-18T12:40:12.563 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.564 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.564 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.564 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.565 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mds_autoscaler", 2023-10-18T12:40:12.565 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.566 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.566 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.566 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.567 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.567 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.567 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.568 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.568 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.568 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.568 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.569 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.569 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.569 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.569 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.570 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.570 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.570 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.571 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.571 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.571 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.572 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.572 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.572 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.573 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.573 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.573 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.574 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.574 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.574 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.575 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.575 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.575 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.576 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.576 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.576 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.577 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.577 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.577 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.578 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.578 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.578 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.579 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.579 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.579 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.579 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.580 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.580 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.580 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.580 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.581 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.581 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.581 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.582 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.582 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.582 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.583 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.583 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.583 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.583 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.584 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.584 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.584 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.584 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.585 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.585 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.585 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.586 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mirroring", 2023-10-18T12:40:12.587 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.587 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.587 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.588 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.588 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.588 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.588 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.589 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.589 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.589 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.590 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.590 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.590 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.590 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.591 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.591 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.591 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.591 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.592 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.592 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.592 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.593 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.593 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.593 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.594 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.594 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.595 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.595 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.595 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.595 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.596 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.596 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.596 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.597 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.597 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.597 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.598 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.598 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.598 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.599 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.599 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.599 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.599 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.600 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.600 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.600 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.600 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.601 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.601 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.601 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.602 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.602 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.602 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.602 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.603 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.603 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.604 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.604 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.604 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.604 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.605 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.605 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.605 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.605 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.606 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.606 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.606 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.607 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.607 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.609 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "nfs", 2023-10-18T12:40:12.609 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.609 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.610 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.610 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.610 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.611 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.611 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.611 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.611 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.612 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.612 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.612 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.612 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.613 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.613 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.613 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.614 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.614 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.614 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.615 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.615 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.615 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.615 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.616 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.616 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.616 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.617 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.617 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.617 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.617 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.618 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.618 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.618 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.618 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.619 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.619 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.619 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.620 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.620 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.620 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.621 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.621 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.621 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.621 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.622 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.622 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.622 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.622 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.623 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.623 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.623 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.623 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.624 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.624 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.624 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.624 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.625 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.625 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.626 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.626 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.626 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.626 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.627 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.627 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.627 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.628 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.628 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.628 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.630 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "orchestrator", 2023-10-18T12:40:12.630 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.630 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.631 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.631 INFO:teuthology.orchestra.run.smithi070.stdout: "fail_fs": { 2023-10-18T12:40:12.631 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "fail_fs", 2023-10-18T12:40:12.631 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.632 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.632 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.632 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.632 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.633 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.633 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.633 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-18T12:40:12.634 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.634 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.634 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.635 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.635 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.635 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.635 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.636 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.636 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.636 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.637 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.637 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.637 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.637 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.638 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.638 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.638 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.638 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.639 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.639 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.639 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.640 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.640 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.640 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.641 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.641 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.641 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.641 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.642 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.642 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.642 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.643 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.643 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.643 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.643 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.644 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.644 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.644 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.645 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.645 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.646 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.646 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.646 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.647 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.647 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.647 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.647 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.648 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.648 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.648 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.648 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.649 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.649 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.649 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.650 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.650 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.650 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.651 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.651 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.652 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.652 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.652 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.652 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.653 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.653 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.653 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.653 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.654 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator": { 2023-10-18T12:40:12.654 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "orchestrator", 2023-10-18T12:40:12.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.655 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.655 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.655 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.656 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.656 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.656 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm", 2023-10-18T12:40:12.657 INFO:teuthology.orchestra.run.smithi070.stdout: "rook", 2023-10-18T12:40:12.657 INFO:teuthology.orchestra.run.smithi070.stdout: "test_orchestrator" 2023-10-18T12:40:12.657 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.658 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Orchestrator backend", 2023-10-18T12:40:12.658 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.658 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.658 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.659 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.659 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.660 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "osd_perf_query", 2023-10-18T12:40:12.660 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.660 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.661 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.661 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.661 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.662 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.662 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.662 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.663 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.663 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.663 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.663 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.663 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.664 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.664 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.664 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.664 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.665 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.665 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.665 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.666 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.666 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.666 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.667 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.667 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.668 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.668 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.668 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.668 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.669 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.669 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.669 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.669 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.670 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.670 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.671 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.671 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.671 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.672 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.672 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.672 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.672 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.673 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.673 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.673 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.674 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.674 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.674 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.675 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.675 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.675 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.675 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.676 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.676 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.676 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.677 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.677 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.678 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.678 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.678 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.678 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.679 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.679 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.679 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "osd_support", 2023-10-18T12:40:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.682 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.682 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.682 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.683 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.683 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.684 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.684 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.684 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.684 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.685 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.685 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.685 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.685 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.686 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.686 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.686 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.687 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.687 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.687 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.687 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.688 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.688 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.689 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.689 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.689 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.690 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.690 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.690 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.691 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.691 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.691 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.692 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.692 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.692 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.693 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.693 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.694 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.694 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.694 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.695 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.695 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.695 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.696 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.696 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.696 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.697 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.697 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.697 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.697 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.698 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.699 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.699 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.699 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.699 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.700 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.700 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.700 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.701 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.701 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.701 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.702 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.702 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.702 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.704 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pg_autoscaler", 2023-10-18T12:40:12.704 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.704 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.705 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.705 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.705 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.705 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.706 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.706 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.706 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.706 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.707 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.707 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.707 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.708 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.708 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.708 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.708 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.709 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.709 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.709 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.709 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.710 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.710 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.711 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.711 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.711 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.711 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.712 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.712 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.712 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.713 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.713 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.713 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.713 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.714 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.714 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.715 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.715 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.715 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.716 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.716 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.716 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.717 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.717 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.717 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.717 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.718 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.718 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.718 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.719 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.719 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.719 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.719 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.720 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.720 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.721 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.721 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.721 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.722 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.722 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.723 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.723 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.723 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.723 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.724 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.724 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.725 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:12.725 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:12.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.725 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.726 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.726 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:12.726 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.727 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.727 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.727 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.727 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.728 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.728 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.729 INFO:teuthology.orchestra.run.smithi070.stdout: "threshold": { 2023-10-18T12:40:12.729 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "threshold", 2023-10-18T12:40:12.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:12.730 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.730 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.730 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3.0", 2023-10-18T12:40:12.731 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1.0", 2023-10-18T12:40:12.731 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.731 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.732 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "scaling threshold", 2023-10-18T12:40:12.732 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-18T12:40:12.732 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.732 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.734 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "progress", 2023-10-18T12:40:12.734 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.735 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.735 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.735 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_pg_recovery_event": { 2023-10-18T12:40:12.735 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_pg_recovery_event", 2023-10-18T12:40:12.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.736 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.736 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.736 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.737 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.737 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.737 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.738 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-18T12:40:12.738 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.738 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.738 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.739 INFO:teuthology.orchestra.run.smithi070.stdout: "enabled": { 2023-10-18T12:40:12.739 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enabled", 2023-10-18T12:40:12.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.740 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.740 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.740 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.741 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.741 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.741 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.742 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.742 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.742 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.742 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.743 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.743 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.744 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.744 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.744 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.744 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.745 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.745 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.745 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.746 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.746 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.746 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.746 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.747 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.747 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.747 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.747 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.748 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.748 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.748 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.749 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.749 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.749 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.750 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.750 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.750 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.751 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.751 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.751 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.751 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.752 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.752 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.752 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.753 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.753 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.753 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.754 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.754 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.754 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.755 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.755 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.755 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.756 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.756 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.756 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.756 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.757 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.757 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.757 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.757 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.758 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.758 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.759 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.759 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.759 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.760 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.760 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.760 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.760 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.761 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.761 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.761 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.762 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.762 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.763 INFO:teuthology.orchestra.run.smithi070.stdout: "max_completed_events": { 2023-10-18T12:40:12.763 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_completed_events", 2023-10-18T12:40:12.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.763 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.764 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.764 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "50", 2023-10-18T12:40:12.764 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.764 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.765 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.765 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "number of past completed events to remember", 2023-10-18T12:40:12.765 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.766 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.766 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.766 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:12.767 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:12.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:12.767 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.767 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.768 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:12.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.768 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.768 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.769 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long the module is going to sleep", 2023-10-18T12:40:12.769 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.769 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.769 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.770 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.770 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.771 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prometheus", 2023-10-18T12:40:12.771 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.771 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.772 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.772 INFO:teuthology.orchestra.run.smithi070.stdout: "cache": { 2023-10-18T12:40:12.772 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cache", 2023-10-18T12:40:12.773 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.773 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.773 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.773 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.774 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.774 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.774 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.775 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.775 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.775 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.775 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.776 INFO:teuthology.orchestra.run.smithi070.stdout: "exclude_perf_counters": { 2023-10-18T12:40:12.776 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "exclude_perf_counters", 2023-10-18T12:40:12.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.777 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.777 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.777 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.778 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.778 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.778 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.779 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-18T12:40:12.779 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-18T12:40:12.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.779 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.780 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.780 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.780 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.781 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.781 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.781 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.782 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.782 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.782 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.783 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.783 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.783 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.783 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.784 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.784 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.784 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.785 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.785 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.785 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.785 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.786 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.786 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.787 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.787 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.788 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.788 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.788 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.788 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.789 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.789 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.789 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.790 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.790 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.791 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.791 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.791 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.792 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.792 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.792 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.792 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.793 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.793 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.793 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.794 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.794 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.794 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.794 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.796 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.796 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.796 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.797 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.797 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.797 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.798 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.798 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.798 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.799 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.799 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_stats_pools": { 2023-10-18T12:40:12.800 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_stats_pools", 2023-10-18T12:40:12.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.801 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.801 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.801 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.802 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.802 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.802 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.802 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.803 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.803 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.803 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-18T12:40:12.804 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-18T12:40:12.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.805 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.805 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.805 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "300", 2023-10-18T12:40:12.806 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.806 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.806 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.806 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.807 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.807 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.807 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.808 INFO:teuthology.orchestra.run.smithi070.stdout: "scrape_interval": { 2023-10-18T12:40:12.808 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "scrape_interval", 2023-10-18T12:40:12.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:12.809 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.809 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.809 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15.0", 2023-10-18T12:40:12.810 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.810 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.810 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.811 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.811 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.811 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.812 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:12.812 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:12.813 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.813 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.813 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.813 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "::", 2023-10-18T12:40:12.814 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.814 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.814 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.815 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-18T12:40:12.815 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.815 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.815 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.816 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:12.816 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:12.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.817 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.817 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.817 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "9283", 2023-10-18T12:40:12.818 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.818 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.818 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.819 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-18T12:40:12.819 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.819 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.819 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.820 INFO:teuthology.orchestra.run.smithi070.stdout: "stale_cache_strategy": { 2023-10-18T12:40:12.820 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "stale_cache_strategy", 2023-10-18T12:40:12.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.821 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.821 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.821 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "log", 2023-10-18T12:40:12.822 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.822 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.822 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.823 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.823 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.823 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.824 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.824 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_behaviour": { 2023-10-18T12:40:12.824 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_behaviour", 2023-10-18T12:40:12.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.825 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.825 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "default", 2023-10-18T12:40:12.826 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.826 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.826 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.827 INFO:teuthology.orchestra.run.smithi070.stdout: "default", 2023-10-18T12:40:12.827 INFO:teuthology.orchestra.run.smithi070.stdout: "error" 2023-10-18T12:40:12.827 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.828 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.828 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.828 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.828 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.829 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_error_status_code": { 2023-10-18T12:40:12.829 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_error_status_code", 2023-10-18T12:40:12.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.830 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.830 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.830 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "500", 2023-10-18T12:40:12.831 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "400", 2023-10-18T12:40:12.831 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "599", 2023-10-18T12:40:12.831 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.832 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.832 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.832 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.832 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.834 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_support", 2023-10-18T12:40:12.834 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.834 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.835 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.835 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.835 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.836 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.836 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.836 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.837 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.837 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.837 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.838 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.838 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.838 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.838 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.839 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.839 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.839 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.840 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.840 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.840 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.840 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.841 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.841 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.842 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.842 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.842 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.843 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.843 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.843 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.844 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.844 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.844 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.845 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.845 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.846 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.846 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.847 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.847 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.847 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.847 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.847 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.848 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.848 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.848 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.849 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.849 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.849 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.849 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.850 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.850 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.850 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.851 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.851 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.851 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.852 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.852 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.852 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.853 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.853 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.853 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.853 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.854 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.854 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.854 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.855 INFO:teuthology.orchestra.run.smithi070.stdout: "max_concurrent_snap_create": { 2023-10-18T12:40:12.855 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_concurrent_snap_create", 2023-10-18T12:40:12.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.856 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.856 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.857 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:12.857 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.857 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.858 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.858 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.858 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.858 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.859 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mirror_snapshot_schedule": { 2023-10-18T12:40:12.860 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mirror_snapshot_schedule", 2023-10-18T12:40:12.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.860 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.860 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.861 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.861 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.861 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.861 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.862 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.862 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.862 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.863 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.863 INFO:teuthology.orchestra.run.smithi070.stdout: "trash_purge_schedule": { 2023-10-18T12:40:12.863 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "trash_purge_schedule", 2023-10-18T12:40:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.865 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.865 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.865 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.865 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.866 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.866 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.866 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.867 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.867 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "restful", 2023-10-18T12:40:12.868 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.868 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.868 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.869 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_auth": { 2023-10-18T12:40:12.869 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enable_auth", 2023-10-18T12:40:12.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.869 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.870 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.870 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:12.870 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.871 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.871 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.871 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.871 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.872 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.872 INFO:teuthology.orchestra.run.smithi070.stdout: "key_file": { 2023-10-18T12:40:12.873 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "key_file", 2023-10-18T12:40:12.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.873 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.874 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.874 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.874 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.874 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.875 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.875 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.875 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.875 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.876 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.876 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.876 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.877 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.877 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.877 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.877 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.878 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.878 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.878 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.879 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.879 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.879 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.879 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.880 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.880 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.880 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.881 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.881 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.881 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.881 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.882 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.882 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.883 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.883 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.883 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.884 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.884 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.884 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.884 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.885 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.885 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.885 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.886 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.886 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.886 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.887 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.887 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.887 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.888 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.888 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.888 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.888 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.889 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.889 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.889 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.890 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.890 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.890 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.891 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.891 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.892 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.892 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.893 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.893 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.893 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.894 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.894 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.894 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.894 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.895 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.895 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.895 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.896 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:12.896 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:12.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.897 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.897 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.897 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.897 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.898 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.898 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.898 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.898 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.899 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.899 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.900 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:12.900 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:12.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.900 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.901 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.901 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.901 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.902 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.902 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.902 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.902 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.903 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.903 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.903 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.904 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rgw", 2023-10-18T12:40:12.905 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.905 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.905 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.905 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.906 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.906 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.906 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.906 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.907 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.907 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.907 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.907 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.908 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.908 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.908 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.908 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.909 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.909 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.909 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.910 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.910 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.910 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.910 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.911 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.911 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.912 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.912 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.912 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.913 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.913 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.913 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.913 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.914 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.914 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.914 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.915 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.915 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.915 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.916 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.916 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.916 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.916 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.917 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.917 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.917 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.918 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.918 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.918 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.918 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.919 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.919 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.919 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.920 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.920 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.920 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.921 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.921 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.921 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.922 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.922 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.922 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.923 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.923 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.923 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.923 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.924 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.924 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.924 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.925 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rook", 2023-10-18T12:40:12.926 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.926 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.926 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.926 INFO:teuthology.orchestra.run.smithi070.stdout: "drive_group_interval": { 2023-10-18T12:40:12.927 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "drive_group_interval", 2023-10-18T12:40:12.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:12.927 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.928 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.928 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "300.0", 2023-10-18T12:40:12.928 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.928 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.929 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.929 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-18T12:40:12.929 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.929 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.930 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.930 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.931 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.931 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.931 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.932 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.932 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.932 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.932 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.933 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.933 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.933 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.934 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.934 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.934 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.934 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.935 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.935 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.935 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.935 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.936 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.936 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.937 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.937 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.937 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.937 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.938 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.938 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.938 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.939 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.939 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.939 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.939 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.940 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.940 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.941 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.941 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.941 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.941 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.942 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.942 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.942 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.943 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.943 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.943 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.943 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.944 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.944 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.944 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.945 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.945 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.945 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.946 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.946 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.947 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.947 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.947 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.948 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.948 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.948 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.948 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.949 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.949 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.949 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.950 INFO:teuthology.orchestra.run.smithi070.stdout: "storage_class": { 2023-10-18T12:40:12.950 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "storage_class", 2023-10-18T12:40:12.950 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.951 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.951 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.951 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "local", 2023-10-18T12:40:12.951 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.952 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.952 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.952 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-18T12:40:12.953 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.953 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:12.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:12.954 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "selftest", 2023-10-18T12:40:12.955 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:12.955 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:12.955 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:12.956 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:12.956 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:12.956 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.956 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.957 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.957 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.957 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.957 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.958 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.958 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.958 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.959 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.959 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.959 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.959 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.960 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.960 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.960 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.961 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.961 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.961 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:12.962 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:12.962 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.962 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.962 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.963 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.963 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.963 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.964 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.964 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.964 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.964 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.965 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.965 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:12.965 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:12.966 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.966 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.966 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.967 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:12.967 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.967 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.967 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:12.968 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:12.968 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:12.968 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:12.969 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:12.969 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:12.969 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:12.969 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:12.970 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.970 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.970 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.970 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.971 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:12.971 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:12.971 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.972 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.972 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:12.972 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:12.972 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.973 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.973 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.973 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.974 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.974 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.974 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.975 INFO:teuthology.orchestra.run.smithi070.stdout: "roption1": { 2023-10-18T12:40:12.975 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "roption1", 2023-10-18T12:40:12.975 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.975 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.976 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.976 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.976 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.976 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.977 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.977 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.977 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.978 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.978 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.978 INFO:teuthology.orchestra.run.smithi070.stdout: "roption2": { 2023-10-18T12:40:12.979 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "roption2", 2023-10-18T12:40:12.979 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.979 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.979 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.980 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "xyz", 2023-10-18T12:40:12.980 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.980 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.981 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.981 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.981 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.981 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.982 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption1": { 2023-10-18T12:40:12.983 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption1", 2023-10-18T12:40:12.983 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.983 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.984 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.984 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.984 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.984 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.985 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.985 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.985 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.985 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.986 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption2": { 2023-10-18T12:40:12.986 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption2", 2023-10-18T12:40:12.986 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:12.987 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.987 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.987 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.988 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.988 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.988 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.989 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.989 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.989 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.990 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption3": { 2023-10-18T12:40:12.990 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption3", 2023-10-18T12:40:12.990 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:12.990 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.991 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.991 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.991 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.991 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.992 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.992 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.992 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.992 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.993 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.993 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption4": { 2023-10-18T12:40:12.993 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption4", 2023-10-18T12:40:12.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:12.994 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.994 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.995 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.995 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.995 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.996 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:12.996 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:12.996 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:12.996 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:12.997 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:12.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:12.997 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption5": { 2023-10-18T12:40:12.997 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption5", 2023-10-18T12:40:12.998 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:12.998 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:12.998 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:12.999 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:12.999 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:12.999 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:12.999 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.000 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.000 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.001 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.001 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption6": { 2023-10-18T12:40:13.001 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption6", 2023-10-18T12:40:13.002 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.002 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.002 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.003 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.003 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.003 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.003 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.004 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.004 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.004 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.005 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.005 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption7": { 2023-10-18T12:40:13.005 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption7", 2023-10-18T12:40:13.006 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.006 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.006 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.007 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:13.007 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "42", 2023-10-18T12:40:13.007 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.007 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.008 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.008 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.008 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.009 INFO:teuthology.orchestra.run.smithi070.stdout: "testkey": { 2023-10-18T12:40:13.009 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testkey", 2023-10-18T12:40:13.009 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.010 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.010 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.010 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.011 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.011 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.011 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.011 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.012 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.012 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.012 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.013 INFO:teuthology.orchestra.run.smithi070.stdout: "testlkey": { 2023-10-18T12:40:13.013 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testlkey", 2023-10-18T12:40:13.013 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.014 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.014 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.014 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.015 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.015 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.015 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.015 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.016 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.016 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.016 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.017 INFO:teuthology.orchestra.run.smithi070.stdout: "testnewline": { 2023-10-18T12:40:13.017 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testnewline", 2023-10-18T12:40:13.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.018 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.018 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.019 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.019 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.019 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.019 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.020 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.020 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.020 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.021 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snap_schedule", 2023-10-18T12:40:13.022 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.022 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.022 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.023 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_m_granularity": { 2023-10-18T12:40:13.023 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_m_granularity", 2023-10-18T12:40:13.023 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.023 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.024 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.024 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.024 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.024 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.025 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.025 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow minute scheduled snapshots", 2023-10-18T12:40:13.025 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.026 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.026 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.026 INFO:teuthology.orchestra.run.smithi070.stdout: "dump_on_update": { 2023-10-18T12:40:13.027 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "dump_on_update", 2023-10-18T12:40:13.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.027 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.028 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.028 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.028 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.028 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.029 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.029 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "dump database to debug log on update", 2023-10-18T12:40:13.029 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.029 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.030 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.030 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.031 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.031 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.031 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.031 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.032 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.032 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.032 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.033 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.033 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.033 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.033 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.034 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.034 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.034 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.035 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.035 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.035 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.035 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.036 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.036 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.036 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.037 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.037 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.038 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.038 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.038 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.038 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.039 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.039 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.039 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.040 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.040 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.040 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.041 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.041 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.041 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.042 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.042 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.042 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.043 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.043 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.043 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.043 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.044 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.044 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.044 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.045 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.045 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.045 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.045 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.046 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.046 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.047 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.047 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.047 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.048 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.048 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.048 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.048 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.049 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.049 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.049 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.050 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.051 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "stats", 2023-10-18T12:40:13.051 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.052 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.052 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.052 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.053 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.053 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.053 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.053 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.057 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.057 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.057 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.057 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.058 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.058 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.059 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.059 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.060 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.060 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.060 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.060 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.061 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.061 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.061 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.063 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.063 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.063 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.063 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.064 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.064 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.064 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.064 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.065 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.065 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.065 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.065 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.066 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.066 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.066 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.067 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.067 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.067 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.067 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.068 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.068 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.069 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.069 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.069 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.070 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.070 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.070 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.070 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.071 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.071 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.071 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.071 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.072 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "status", 2023-10-18T12:40:13.073 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.073 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.073 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.074 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.074 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.075 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.075 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.075 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.075 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.076 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.076 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.076 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.076 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.077 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.077 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.077 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.078 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.078 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.078 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.079 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.079 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.079 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.080 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.080 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.080 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.081 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.081 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.081 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.081 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.082 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.082 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.082 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.083 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.083 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.083 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.084 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.084 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.084 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.085 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.085 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.085 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.085 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.086 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.086 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.086 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.087 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.087 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.087 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.088 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.088 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.088 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.088 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.089 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.089 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.089 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.090 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.090 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.090 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.091 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.091 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.091 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.091 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.092 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.092 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.092 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.092 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.093 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.093 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.093 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.093 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.094 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "telegraf", 2023-10-18T12:40:13.095 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.095 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.095 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.095 INFO:teuthology.orchestra.run.smithi070.stdout: "address": { 2023-10-18T12:40:13.096 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "address", 2023-10-18T12:40:13.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.096 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.097 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.097 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-18T12:40:13.097 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.097 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.098 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.098 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.098 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.098 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.099 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.099 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:13.100 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:13.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.100 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.100 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.101 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15", 2023-10-18T12:40:13.101 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.101 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.102 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.102 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.102 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.102 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.103 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.103 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.103 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.104 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.104 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.104 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.105 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.105 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.105 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.105 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.106 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.106 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.106 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.106 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.107 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.107 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.107 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.108 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.108 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.108 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.108 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.109 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.109 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.110 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.110 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.110 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.110 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.111 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.111 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.111 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.112 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.112 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.112 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.112 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.113 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.113 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.113 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.114 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.114 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.114 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.115 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.115 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.115 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.115 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.117 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.118 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.118 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.118 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.119 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.119 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.119 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.119 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.120 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.120 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.120 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.121 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.121 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.121 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.122 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.122 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.122 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.123 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.123 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.123 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.124 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.124 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.124 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.124 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.125 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.125 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "telemetry", 2023-10-18T12:40:13.126 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.126 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.126 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.127 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_basic": { 2023-10-18T12:40:13.127 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_basic", 2023-10-18T12:40:13.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.127 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.128 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.128 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.128 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.128 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.129 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.129 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-18T12:40:13.129 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.130 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.130 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.130 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_crash": { 2023-10-18T12:40:13.131 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_crash", 2023-10-18T12:40:13.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.131 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.132 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.132 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.132 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.132 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.133 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.133 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-18T12:40:13.133 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.133 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.134 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.134 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_device": { 2023-10-18T12:40:13.135 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_device", 2023-10-18T12:40:13.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.135 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.136 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.136 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.136 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.136 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.137 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.137 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-18T12:40:13.137 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.138 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.138 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.138 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_ident": { 2023-10-18T12:40:13.139 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_ident", 2023-10-18T12:40:13.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.139 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.140 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.140 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.140 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.140 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.141 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.141 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-18T12:40:13.141 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.142 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.142 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_perf": { 2023-10-18T12:40:13.143 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_perf", 2023-10-18T12:40:13.143 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.143 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.143 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.144 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.144 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.144 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.145 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.145 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-18T12:40:13.145 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.145 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.146 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.146 INFO:teuthology.orchestra.run.smithi070.stdout: "contact": { 2023-10-18T12:40:13.146 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "contact", 2023-10-18T12:40:13.147 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.147 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.147 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.148 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.148 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.148 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.148 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.149 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.149 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.149 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.149 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.150 INFO:teuthology.orchestra.run.smithi070.stdout: "description": { 2023-10-18T12:40:13.150 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "description", 2023-10-18T12:40:13.151 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.151 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.151 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.151 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.152 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.152 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.152 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.153 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.153 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.153 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.153 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.154 INFO:teuthology.orchestra.run.smithi070.stdout: "device_url": { 2023-10-18T12:40:13.154 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_url", 2023-10-18T12:40:13.155 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.155 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.155 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.156 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-18T12:40:13.156 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.156 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.156 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.157 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.157 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.157 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.158 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.158 INFO:teuthology.orchestra.run.smithi070.stdout: "enabled": { 2023-10-18T12:40:13.158 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enabled", 2023-10-18T12:40:13.159 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.159 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.159 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.160 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.160 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.160 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.160 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.161 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.161 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.161 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.162 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.162 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:13.163 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:13.163 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.163 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.163 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.164 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "24", 2023-10-18T12:40:13.164 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "8", 2023-10-18T12:40:13.164 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.164 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.165 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.165 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.165 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.166 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.166 INFO:teuthology.orchestra.run.smithi070.stdout: "last_opt_revision": { 2023-10-18T12:40:13.166 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "last_opt_revision", 2023-10-18T12:40:13.167 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.167 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.167 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.168 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1", 2023-10-18T12:40:13.168 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.168 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.168 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.169 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.169 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.169 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.169 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.170 INFO:teuthology.orchestra.run.smithi070.stdout: "leaderboard": { 2023-10-18T12:40:13.170 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "leaderboard", 2023-10-18T12:40:13.171 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.171 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.171 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.171 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.172 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.172 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.172 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.172 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.173 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.173 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.173 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.174 INFO:teuthology.orchestra.run.smithi070.stdout: "leaderboard_description": { 2023-10-18T12:40:13.174 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "leaderboard_description", 2023-10-18T12:40:13.175 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.175 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.175 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.175 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.176 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.176 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.176 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.177 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.177 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.177 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.177 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.178 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.178 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.179 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.179 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.179 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.179 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.180 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.180 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.180 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.181 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.181 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.181 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.181 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.182 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.182 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.182 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.183 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.183 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.183 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.183 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.184 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.184 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.185 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.185 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.185 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.185 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.186 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.186 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.186 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.186 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.187 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.187 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.187 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.188 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.188 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.188 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.189 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.189 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.189 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.190 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.190 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.190 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.190 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.191 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.191 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.191 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.192 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.192 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.192 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.192 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.193 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.193 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.193 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.194 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.194 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.194 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.195 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.195 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.195 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.196 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.196 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.196 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.196 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.197 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.197 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.197 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.198 INFO:teuthology.orchestra.run.smithi070.stdout: "organization": { 2023-10-18T12:40:13.198 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "organization", 2023-10-18T12:40:13.198 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.199 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.199 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.199 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.200 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.200 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.200 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.200 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.201 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.201 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.201 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.202 INFO:teuthology.orchestra.run.smithi070.stdout: "proxy": { 2023-10-18T12:40:13.202 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "proxy", 2023-10-18T12:40:13.202 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.203 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.203 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.203 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.203 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.204 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.204 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.204 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.205 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.205 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.205 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.206 INFO:teuthology.orchestra.run.smithi070.stdout: "url": { 2023-10-18T12:40:13.206 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "url", 2023-10-18T12:40:13.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.206 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.207 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.207 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-18T12:40:13.207 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.207 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.208 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.208 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.208 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.209 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.209 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.210 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "test_orchestrator", 2023-10-18T12:40:13.211 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.211 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.211 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.211 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.212 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.212 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.212 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.213 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.213 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.213 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.214 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.214 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.214 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.214 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.215 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.215 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.215 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.216 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.217 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.218 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.218 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.218 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.219 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.219 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.219 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.219 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.220 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.220 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.220 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.221 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.221 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.221 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.221 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.222 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.222 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.223 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.223 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.223 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.224 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.224 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.225 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.225 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.225 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.225 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.226 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.226 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.226 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.226 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.227 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.227 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.227 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.227 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.228 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.228 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.229 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.229 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.229 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.229 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.230 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.230 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.230 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.231 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.231 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.231 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.231 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.232 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.233 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "volumes", 2023-10-18T12:40:13.233 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.234 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.234 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.234 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.234 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.235 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.235 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.235 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.236 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.236 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.236 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.236 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.237 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.237 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.237 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.238 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.238 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.238 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.238 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.239 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.239 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.239 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.240 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.240 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.240 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.241 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.241 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.242 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.242 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.242 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.242 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.243 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.243 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.243 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.243 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.244 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.244 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.245 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.245 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.245 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.245 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.246 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.246 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.246 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.247 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.247 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.247 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.248 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.248 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.248 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.248 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.249 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.249 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.249 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.249 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.250 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.250 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.251 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.251 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.251 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.251 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.252 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.252 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.252 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.253 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.253 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.253 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.254 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.254 INFO:teuthology.orchestra.run.smithi070.stdout: "max_concurrent_clones": { 2023-10-18T12:40:13.254 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_concurrent_clones", 2023-10-18T12:40:13.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.255 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.255 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "4", 2023-10-18T12:40:13.256 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.256 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.256 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.257 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-18T12:40:13.257 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.257 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.257 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.258 INFO:teuthology.orchestra.run.smithi070.stdout: "periodic_async_work": { 2023-10-18T12:40:13.258 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "periodic_async_work", 2023-10-18T12:40:13.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.259 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.259 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.260 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.260 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.260 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.260 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Periodically check for async work", 2023-10-18T12:40:13.261 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.261 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.261 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.262 INFO:teuthology.orchestra.run.smithi070.stdout: "snapshot_clone_delay": { 2023-10-18T12:40:13.262 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snapshot_clone_delay", 2023-10-18T12:40:13.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.263 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.263 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.263 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-18T12:40:13.265 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.265 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.265 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.267 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix", 2023-10-18T12:40:13.267 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.267 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.268 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.268 INFO:teuthology.orchestra.run.smithi070.stdout: "discovery_interval": { 2023-10-18T12:40:13.268 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "discovery_interval", 2023-10-18T12:40:13.268 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:13.269 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.269 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.269 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "100", 2023-10-18T12:40:13.270 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.270 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.270 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.270 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.271 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.271 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.271 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.272 INFO:teuthology.orchestra.run.smithi070.stdout: "identifier": { 2023-10-18T12:40:13.272 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "identifier", 2023-10-18T12:40:13.272 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.273 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.273 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.273 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.274 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.274 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.274 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.274 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.275 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.275 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.275 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.276 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:13.276 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:13.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.277 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.277 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.277 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:13.277 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.278 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.278 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.278 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.278 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.279 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.280 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.280 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.280 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.281 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.281 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.281 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.281 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.282 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.282 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.282 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.283 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.283 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.283 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.283 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.284 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.284 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.284 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.285 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.285 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.285 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.286 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.286 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.287 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.287 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.287 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.289 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.289 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.290 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.290 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.290 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.291 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.291 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.291 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.291 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.292 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.292 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.292 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.293 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.293 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.293 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.293 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.294 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.294 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.294 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.294 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.295 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.295 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.296 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.296 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.296 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.297 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.297 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.297 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.298 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.298 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.298 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.298 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.299 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.299 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_host": { 2023-10-18T12:40:13.300 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_host", 2023-10-18T12:40:13.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.300 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.300 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.301 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.301 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.301 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.302 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.302 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.302 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.302 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.303 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.303 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_port": { 2023-10-18T12:40:13.304 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_port", 2023-10-18T12:40:13.304 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.304 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.304 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.305 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10051", 2023-10-18T12:40:13.305 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.305 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.306 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.306 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.306 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.306 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.307 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.307 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_sender": { 2023-10-18T12:40:13.308 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_sender", 2023-10-18T12:40:13.308 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.308 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.308 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.309 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-18T12:40:13.309 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.309 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.310 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.310 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.310 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.311 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.311 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.312 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:13.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.313 INFO:teuthology.orchestra.run.smithi070.stdout: "gid": 24167, 2023-10-18T12:40:13.313 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "y", 2023-10-18T12:40:13.313 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_features": 4540138322906710015, 2023-10-18T12:40:13.314 INFO:teuthology.orchestra.run.smithi070.stdout: "available_modules": [ 2023-10-18T12:40:13.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.314 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "alerts", 2023-10-18T12:40:13.315 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.315 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.315 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.315 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:13.316 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:13.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.316 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.317 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.317 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:13.317 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.317 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.318 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.318 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How frequently to reexamine health status", 2023-10-18T12:40:13.318 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.319 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.319 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.319 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.320 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.320 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.321 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.321 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.321 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.321 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.322 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.322 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.322 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.323 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.323 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.323 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.323 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.324 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.324 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.324 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.324 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.325 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.325 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.326 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.326 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.326 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.327 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.327 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.327 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.328 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.328 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.328 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.328 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.332 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.332 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.332 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.332 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.333 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.333 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.333 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.333 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.334 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.334 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.334 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.335 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.335 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.335 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.336 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.336 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.336 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.336 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.337 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.337 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.337 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.337 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.338 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.338 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.338 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.339 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_destination": { 2023-10-18T12:40:13.339 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_destination", 2023-10-18T12:40:13.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.340 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.340 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.340 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Email address to send alerts to", 2023-10-18T12:40:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_from_name": { 2023-10-18T12:40:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_from_name", 2023-10-18T12:40:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "Ceph", 2023-10-18T12:40:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Email From: name", 2023-10-18T12:40:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_host": { 2023-10-18T12:40:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_host", 2023-10-18T12:40:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP server", 2023-10-18T12:40:13.350 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.350 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.350 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.351 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_password": { 2023-10-18T12:40:13.351 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_password", 2023-10-18T12:40:13.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.352 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.352 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.352 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.353 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.353 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.353 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.354 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Password to authenticate with", 2023-10-18T12:40:13.354 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.354 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.354 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.355 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_port": { 2023-10-18T12:40:13.355 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_port", 2023-10-18T12:40:13.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.356 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.356 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.356 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "465", 2023-10-18T12:40:13.357 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.357 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.357 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.358 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP port", 2023-10-18T12:40:13.358 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.358 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.359 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.359 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_sender": { 2023-10-18T12:40:13.359 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_sender", 2023-10-18T12:40:13.360 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.360 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.360 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.360 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.361 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.361 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.361 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.362 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP envelope sender", 2023-10-18T12:40:13.362 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.362 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.362 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.363 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_ssl": { 2023-10-18T12:40:13.363 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_ssl", 2023-10-18T12:40:13.364 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.364 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.364 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.364 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.365 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.365 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.365 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.365 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-18T12:40:13.366 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.366 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.366 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.367 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_user": { 2023-10-18T12:40:13.367 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_user", 2023-10-18T12:40:13.367 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.368 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.368 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.368 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.369 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.369 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.369 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.369 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "User to authenticate as", 2023-10-18T12:40:13.370 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.370 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.370 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.372 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "balancer", 2023-10-18T12:40:13.372 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.372 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.372 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.372 INFO:teuthology.orchestra.run.smithi070.stdout: "active": { 2023-10-18T12:40:13.373 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "active", 2023-10-18T12:40:13.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.373 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.373 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.374 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.374 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.374 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.375 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.375 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "automatically balance PGs across cluster", 2023-10-18T12:40:13.375 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.376 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.376 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.376 INFO:teuthology.orchestra.run.smithi070.stdout: "begin_time": { 2023-10-18T12:40:13.377 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "begin_time", 2023-10-18T12:40:13.377 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.377 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.377 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.378 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0000", 2023-10-18T12:40:13.378 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.378 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.378 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.379 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "beginning time of day to automatically balance", 2023-10-18T12:40:13.379 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T12:40:13.379 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.380 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.380 INFO:teuthology.orchestra.run.smithi070.stdout: "begin_weekday": { 2023-10-18T12:40:13.380 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "begin_weekday", 2023-10-18T12:40:13.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:13.381 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.381 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.382 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:13.382 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0", 2023-10-18T12:40:13.382 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "6", 2023-10-18T12:40:13.383 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.383 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-18T12:40:13.383 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T12:40:13.383 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.384 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.384 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_max_iterations": { 2023-10-18T12:40:13.384 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_max_iterations", 2023-10-18T12:40:13.385 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:13.385 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.385 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.385 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "25", 2023-10-18T12:40:13.386 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:13.386 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "250", 2023-10-18T12:40:13.386 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.387 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-18T12:40:13.387 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.387 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.387 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.388 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_metrics": { 2023-10-18T12:40:13.388 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_metrics", 2023-10-18T12:40:13.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.389 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.389 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.389 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "pgs,objects,bytes", 2023-10-18T12:40:13.390 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.390 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.390 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.391 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-18T12:40:13.391 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-18T12:40:13.391 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.392 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.392 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_step": { 2023-10-18T12:40:13.392 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_step", 2023-10-18T12:40:13.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:13.393 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.393 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.394 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0.5", 2023-10-18T12:40:13.394 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0.001", 2023-10-18T12:40:13.394 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "0.999", 2023-10-18T12:40:13.394 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.395 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "aggressiveness of optimization", 2023-10-18T12:40:13.395 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-18T12:40:13.395 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.395 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.396 INFO:teuthology.orchestra.run.smithi070.stdout: "end_time": { 2023-10-18T12:40:13.396 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "end_time", 2023-10-18T12:40:13.397 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.397 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.397 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.397 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "2359", 2023-10-18T12:40:13.398 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.398 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.398 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.398 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "ending time of day to automatically balance", 2023-10-18T12:40:13.399 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T12:40:13.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.399 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.400 INFO:teuthology.orchestra.run.smithi070.stdout: "end_weekday": { 2023-10-18T12:40:13.400 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "end_weekday", 2023-10-18T12:40:13.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:13.401 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.401 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.401 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:13.401 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0", 2023-10-18T12:40:13.402 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "6", 2023-10-18T12:40:13.402 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.402 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-18T12:40:13.403 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T12:40:13.403 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.403 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.404 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.404 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.404 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.405 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.405 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.405 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.405 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.406 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.406 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.406 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.407 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.407 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.407 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.407 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.408 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.408 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.408 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.408 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.409 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.409 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.409 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.410 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.410 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.410 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.410 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.411 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.411 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.411 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.412 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.412 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.412 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.413 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.413 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.413 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.414 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.414 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.414 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.414 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.415 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.415 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.415 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.416 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.416 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.416 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.416 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.417 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.417 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.417 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.417 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.418 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.418 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.418 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.419 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.419 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.419 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.420 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.420 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.420 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.420 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.421 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.421 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.421 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.422 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.422 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.422 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.423 INFO:teuthology.orchestra.run.smithi070.stdout: "min_score": { 2023-10-18T12:40:13.425 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "min_score", 2023-10-18T12:40:13.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:13.426 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.426 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.426 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:13.427 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.427 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.427 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.427 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-18T12:40:13.428 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.428 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.428 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.429 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": { 2023-10-18T12:40:13.429 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mode", 2023-10-18T12:40:13.429 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.430 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.430 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.430 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "upmap", 2023-10-18T12:40:13.431 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.431 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.431 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.431 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-compat", 2023-10-18T12:40:13.432 INFO:teuthology.orchestra.run.smithi070.stdout: "none", 2023-10-18T12:40:13.432 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap" 2023-10-18T12:40:13.432 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Balancer mode", 2023-10-18T12:40:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.434 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_ids": { 2023-10-18T12:40:13.434 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pool_ids", 2023-10-18T12:40:13.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.435 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.435 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.435 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.436 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.436 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.436 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.436 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-18T12:40:13.437 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.437 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.437 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.438 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:13.438 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:13.438 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.439 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.439 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.439 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:13.440 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.440 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.440 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.440 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-18T12:40:13.441 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.441 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.441 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.442 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap_max_deviation": { 2023-10-18T12:40:13.442 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "upmap_max_deviation", 2023-10-18T12:40:13.442 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.443 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.443 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.443 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:13.444 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:13.444 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.444 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.444 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-18T12:40:13.445 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-18T12:40:13.445 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.445 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.446 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap_max_optimizations": { 2023-10-18T12:40:13.446 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "upmap_max_optimizations", 2023-10-18T12:40:13.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:13.447 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.447 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.447 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:13.448 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.448 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.448 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.448 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-18T12:40:13.449 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.449 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.449 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.451 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephadm", 2023-10-18T12:40:13.451 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.451 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.452 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.452 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_down_multiplier": { 2023-10-18T12:40:13.452 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_down_multiplier", 2023-10-18T12:40:13.452 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:13.453 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.453 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.453 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3.0", 2023-10-18T12:40:13.454 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.454 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.454 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.454 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-18T12:40:13.455 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.455 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.455 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.456 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_refresh_rate": { 2023-10-18T12:40:13.456 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_refresh_rate", 2023-10-18T12:40:13.456 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.457 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.457 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.457 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "20", 2023-10-18T12:40:13.457 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.458 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.458 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.458 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-18T12:40:13.459 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.459 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.459 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.460 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_starting_port": { 2023-10-18T12:40:13.460 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_starting_port", 2023-10-18T12:40:13.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.460 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.461 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "4721", 2023-10-18T12:40:13.461 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.462 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.462 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.462 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-18T12:40:13.462 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.463 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.463 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.463 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_ptrace": { 2023-10-18T12:40:13.464 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_ptrace", 2023-10-18T12:40:13.464 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.464 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.465 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.465 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.465 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.465 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.466 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.466 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-18T12:40:13.466 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-18T12:40:13.466 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.467 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.467 INFO:teuthology.orchestra.run.smithi070.stdout: "autotune_interval": { 2023-10-18T12:40:13.468 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "autotune_interval", 2023-10-18T12:40:13.468 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.468 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.468 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.469 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:13.469 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.469 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.469 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.470 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-18T12:40:13.470 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.470 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.471 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.471 INFO:teuthology.orchestra.run.smithi070.stdout: "autotune_memory_target_ratio": { 2023-10-18T12:40:13.471 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "autotune_memory_target_ratio", 2023-10-18T12:40:13.472 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:13.472 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.472 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.473 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0.7", 2023-10-18T12:40:13.473 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.473 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.473 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.474 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-18T12:40:13.474 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.474 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.474 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.475 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm_log_destination": { 2023-10-18T12:40:13.475 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephadm_log_destination", 2023-10-18T12:40:13.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.476 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.476 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.476 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.477 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.477 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.477 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.478 INFO:teuthology.orchestra.run.smithi070.stdout: "file", 2023-10-18T12:40:13.478 INFO:teuthology.orchestra.run.smithi070.stdout: "file,syslog", 2023-10-18T12:40:13.478 INFO:teuthology.orchestra.run.smithi070.stdout: "syslog" 2023-10-18T12:40:13.479 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.479 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-18T12:40:13.479 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.479 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.480 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.480 INFO:teuthology.orchestra.run.smithi070.stdout: "cgroups_split": { 2023-10-18T12:40:13.481 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cgroups_split", 2023-10-18T12:40:13.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.481 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.482 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.482 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.482 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.482 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.483 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-18T12:40:13.483 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.483 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.484 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.484 INFO:teuthology.orchestra.run.smithi070.stdout: "config_checks_enabled": { 2023-10-18T12:40:13.484 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_checks_enabled", 2023-10-18T12:40:13.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.485 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.485 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.485 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.486 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.486 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.486 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.487 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-18T12:40:13.487 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.487 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.487 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.488 INFO:teuthology.orchestra.run.smithi070.stdout: "config_dashboard": { 2023-10-18T12:40:13.488 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_dashboard", 2023-10-18T12:40:13.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.489 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.489 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.489 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.490 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.490 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.490 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.491 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-18T12:40:13.491 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.491 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.491 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.492 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_alertmanager": { 2023-10-18T12:40:13.492 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_alertmanager", 2023-10-18T12:40:13.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.493 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.493 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.493 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-18T12:40:13.494 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.494 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.494 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.495 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:13.495 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.495 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.495 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.496 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_base": { 2023-10-18T12:40:13.496 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_base", 2023-10-18T12:40:13.496 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.497 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.497 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-18T12:40:13.498 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.498 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.498 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.498 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Container image name, without the tag", 2023-10-18T12:40:13.499 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.499 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.499 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.500 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_elasticsearch": { 2023-10-18T12:40:13.500 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_elasticsearch", 2023-10-18T12:40:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-18T12:40:13.502 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.502 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.502 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.503 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "elasticsearch container image", 2023-10-18T12:40:13.503 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.503 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.504 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_grafana": { 2023-10-18T12:40:13.504 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_grafana", 2023-10-18T12:40:13.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.505 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.505 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-18T12:40:13.506 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.506 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.506 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.507 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:13.507 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.507 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.507 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.508 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_haproxy": { 2023-10-18T12:40:13.508 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_haproxy", 2023-10-18T12:40:13.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.509 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.509 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-18T12:40:13.510 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.510 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.510 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.510 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "HAproxy container image", 2023-10-18T12:40:13.511 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.511 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.511 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.512 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_agent": { 2023-10-18T12:40:13.512 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_agent", 2023-10-18T12:40:13.512 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.513 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.513 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.513 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-18T12:40:13.514 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.514 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.514 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.514 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger agent container image", 2023-10-18T12:40:13.515 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.515 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.515 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.516 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_collector": { 2023-10-18T12:40:13.516 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_collector", 2023-10-18T12:40:13.516 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.517 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.517 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.517 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-18T12:40:13.517 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.518 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.518 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.518 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger collector container image", 2023-10-18T12:40:13.519 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.519 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.519 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.520 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_query": { 2023-10-18T12:40:13.520 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_query", 2023-10-18T12:40:13.520 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.520 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.521 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.521 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-18T12:40:13.521 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.522 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.522 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.522 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger query container image", 2023-10-18T12:40:13.522 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.523 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.523 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.524 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_keepalived": { 2023-10-18T12:40:13.524 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_keepalived", 2023-10-18T12:40:13.524 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.524 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.525 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.525 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-18T12:40:13.525 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.525 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.526 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.526 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Keepalived container image", 2023-10-18T12:40:13.526 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.527 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.527 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.527 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_loki": { 2023-10-18T12:40:13.528 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_loki", 2023-10-18T12:40:13.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.528 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.529 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.529 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-18T12:40:13.529 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.529 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.530 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.530 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Loki container image", 2023-10-18T12:40:13.530 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.531 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.531 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.531 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_node_exporter": { 2023-10-18T12:40:13.532 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_node_exporter", 2023-10-18T12:40:13.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.532 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.533 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-18T12:40:13.533 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.533 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.534 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.534 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:13.534 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.535 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.535 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.535 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_nvmeof": { 2023-10-18T12:40:13.536 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_nvmeof", 2023-10-18T12:40:13.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.536 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.537 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.537 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-18T12:40:13.537 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.537 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.538 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.538 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Nvme-of container image", 2023-10-18T12:40:13.538 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.539 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.539 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.539 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_prometheus": { 2023-10-18T12:40:13.540 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_prometheus", 2023-10-18T12:40:13.540 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.540 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.541 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.541 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-18T12:40:13.541 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.541 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.542 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.542 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:13.542 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.543 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.543 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.544 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_promtail": { 2023-10-18T12:40:13.544 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_promtail", 2023-10-18T12:40:13.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.544 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.545 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.545 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-18T12:40:13.545 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.545 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.546 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.546 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Promtail container image", 2023-10-18T12:40:13.546 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.546 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.547 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.547 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_snmp_gateway": { 2023-10-18T12:40:13.547 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_snmp_gateway", 2023-10-18T12:40:13.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.548 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.548 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.548 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-18T12:40:13.549 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.549 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.549 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.550 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SNMP Gateway container image", 2023-10-18T12:40:13.550 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.550 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.550 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.551 INFO:teuthology.orchestra.run.smithi070.stdout: "container_init": { 2023-10-18T12:40:13.551 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_init", 2023-10-18T12:40:13.551 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.552 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.552 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.552 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.553 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.553 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.553 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.553 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Run podman/docker with `--init`", 2023-10-18T12:40:13.554 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.554 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.554 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.555 INFO:teuthology.orchestra.run.smithi070.stdout: "daemon_cache_timeout": { 2023-10-18T12:40:13.555 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "daemon_cache_timeout", 2023-10-18T12:40:13.555 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.556 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.556 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.556 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:13.557 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.557 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.557 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.557 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-18T12:40:13.558 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.558 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.558 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.559 INFO:teuthology.orchestra.run.smithi070.stdout: "default_cephadm_command_timeout": { 2023-10-18T12:40:13.559 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "default_cephadm_command_timeout", 2023-10-18T12:40:13.559 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.560 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.560 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.560 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "900", 2023-10-18T12:40:13.560 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.561 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.561 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.561 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-18T12:40:13.561 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.562 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.562 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.563 INFO:teuthology.orchestra.run.smithi070.stdout: "default_registry": { 2023-10-18T12:40:13.563 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "default_registry", 2023-10-18T12:40:13.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.563 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.564 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.564 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io", 2023-10-18T12:40:13.564 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.564 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.565 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.565 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-18T12:40:13.565 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.566 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.566 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.566 INFO:teuthology.orchestra.run.smithi070.stdout: "device_cache_timeout": { 2023-10-18T12:40:13.567 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_cache_timeout", 2023-10-18T12:40:13.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.567 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.568 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.568 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1800", 2023-10-18T12:40:13.568 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.568 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.568 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.569 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache device inventory", 2023-10-18T12:40:13.569 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.569 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.570 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.570 INFO:teuthology.orchestra.run.smithi070.stdout: "device_enhanced_scan": { 2023-10-18T12:40:13.571 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_enhanced_scan", 2023-10-18T12:40:13.571 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.571 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.571 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.572 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.572 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.572 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.573 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.573 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-18T12:40:13.573 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.573 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.574 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.574 INFO:teuthology.orchestra.run.smithi070.stdout: "facts_cache_timeout": { 2023-10-18T12:40:13.574 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "facts_cache_timeout", 2023-10-18T12:40:13.575 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.575 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.575 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.576 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:13.576 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.576 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.576 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.577 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache host facts data", 2023-10-18T12:40:13.577 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.577 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.578 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.578 INFO:teuthology.orchestra.run.smithi070.stdout: "host_check_interval": { 2023-10-18T12:40:13.578 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "host_check_interval", 2023-10-18T12:40:13.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.579 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.579 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.580 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:13.580 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.580 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.580 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.581 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to perform a host check", 2023-10-18T12:40:13.581 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.581 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.582 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.582 INFO:teuthology.orchestra.run.smithi070.stdout: "inventory_list_all": { 2023-10-18T12:40:13.583 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "inventory_list_all", 2023-10-18T12:40:13.583 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.583 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.583 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.584 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.584 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.584 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.585 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.585 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-18T12:40:13.585 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.586 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.586 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.587 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.587 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.587 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.587 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.588 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.588 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.588 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.589 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.589 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.589 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.590 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.590 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.590 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.590 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.591 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.591 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.591 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.591 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.592 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.592 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.592 INFO:teuthology.orchestra.run.smithi070.stdout: "log_refresh_metadata": { 2023-10-18T12:40:13.593 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_refresh_metadata", 2023-10-18T12:40:13.593 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.593 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.593 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.594 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.594 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.594 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.595 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.595 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-18T12:40:13.595 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.595 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.596 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.596 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.596 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.597 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.597 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.597 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.598 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.598 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.598 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.599 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.599 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-18T12:40:13.599 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.599 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.599 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.600 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.600 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.601 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.601 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.601 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.601 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.602 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.602 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.602 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.602 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.603 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.603 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.603 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.604 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.604 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.604 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.604 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.605 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.605 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.605 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.606 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.606 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.606 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.606 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.607 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.607 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.607 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.608 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.608 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.608 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.608 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.609 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.609 INFO:teuthology.orchestra.run.smithi070.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-18T12:40:13.610 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-18T12:40:13.610 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.610 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.610 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.611 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.611 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.611 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.611 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.612 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-18T12:40:13.612 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.612 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.613 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.613 INFO:teuthology.orchestra.run.smithi070.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-18T12:40:13.613 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-18T12:40:13.614 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.614 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.614 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.614 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "*", 2023-10-18T12:40:13.615 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.615 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.615 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.615 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-18T12:40:13.616 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.616 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.616 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.617 INFO:teuthology.orchestra.run.smithi070.stdout: "max_count_per_host": { 2023-10-18T12:40:13.617 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_count_per_host", 2023-10-18T12:40:13.617 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.618 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.618 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.618 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:13.618 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.619 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.619 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.619 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "max number of daemons per service per host", 2023-10-18T12:40:13.619 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.620 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.620 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.621 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd_draining_count": { 2023-10-18T12:40:13.621 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_osd_draining_count", 2023-10-18T12:40:13.621 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.622 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.622 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.622 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:13.622 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.623 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.623 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.623 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-18T12:40:13.624 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.624 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.624 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.625 INFO:teuthology.orchestra.run.smithi070.stdout: "migration_current": { 2023-10-18T12:40:13.625 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "migration_current", 2023-10-18T12:40:13.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.626 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.626 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.626 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.627 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.627 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.627 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.627 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "internal - do not modify", 2023-10-18T12:40:13.628 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.628 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.628 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.629 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": { 2023-10-18T12:40:13.629 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mode", 2023-10-18T12:40:13.629 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.630 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.630 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.630 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "root", 2023-10-18T12:40:13.631 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.631 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.631 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.631 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm-package", 2023-10-18T12:40:13.632 INFO:teuthology.orchestra.run.smithi070.stdout: "root" 2023-10-18T12:40:13.632 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.632 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "mode for remote execution of cephadm", 2023-10-18T12:40:13.633 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.633 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.633 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.634 INFO:teuthology.orchestra.run.smithi070.stdout: "prometheus_alerts_path": { 2023-10-18T12:40:13.634 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prometheus_alerts_path", 2023-10-18T12:40:13.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.635 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.635 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.635 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-18T12:40:13.635 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.636 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.636 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.636 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-18T12:40:13.636 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.637 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.637 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.637 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_insecure": { 2023-10-18T12:40:13.638 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_insecure", 2023-10-18T12:40:13.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.638 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.639 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.639 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.639 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.639 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.640 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.640 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-18T12:40:13.640 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.640 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.641 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.641 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_password": { 2023-10-18T12:40:13.641 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_password", 2023-10-18T12:40:13.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.642 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.642 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.643 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.643 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.643 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.643 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.644 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-18T12:40:13.644 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.644 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.644 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.645 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_url": { 2023-10-18T12:40:13.645 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_url", 2023-10-18T12:40:13.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.646 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.646 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.646 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.646 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.647 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.647 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.647 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-18T12:40:13.647 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.648 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.648 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.649 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_username": { 2023-10-18T12:40:13.649 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_username", 2023-10-18T12:40:13.649 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.649 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.650 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.650 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.650 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.650 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.651 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.651 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-18T12:40:13.651 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.652 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.652 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.652 INFO:teuthology.orchestra.run.smithi070.stdout: "secure_monitoring_stack": { 2023-10-18T12:40:13.653 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "secure_monitoring_stack", 2023-10-18T12:40:13.653 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.653 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.653 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.654 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.654 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.654 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.654 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.655 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-18T12:40:13.655 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.655 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.655 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.656 INFO:teuthology.orchestra.run.smithi070.stdout: "service_discovery_port": { 2023-10-18T12:40:13.656 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "service_discovery_port", 2023-10-18T12:40:13.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.657 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.657 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.657 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8765", 2023-10-18T12:40:13.658 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.658 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.658 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.659 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "cephadm service discovery port", 2023-10-18T12:40:13.659 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.659 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.659 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.660 INFO:teuthology.orchestra.run.smithi070.stdout: "ssh_config_file": { 2023-10-18T12:40:13.660 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssh_config_file", 2023-10-18T12:40:13.660 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.661 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.661 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.661 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.662 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.662 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.662 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.662 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-18T12:40:13.663 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.663 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.663 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.664 INFO:teuthology.orchestra.run.smithi070.stdout: "use_agent": { 2023-10-18T12:40:13.664 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "use_agent", 2023-10-18T12:40:13.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.665 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.665 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.665 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.665 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.666 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.666 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.666 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-18T12:40:13.666 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.667 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.667 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.668 INFO:teuthology.orchestra.run.smithi070.stdout: "use_repo_digest": { 2023-10-18T12:40:13.668 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "use_repo_digest", 2023-10-18T12:40:13.668 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.668 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.669 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.669 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.669 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.669 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.670 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.670 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-18T12:40:13.670 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.671 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.671 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.671 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_failed_host_check": { 2023-10-18T12:40:13.672 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_failed_host_check", 2023-10-18T12:40:13.672 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.672 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.672 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.673 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.673 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.673 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.674 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.674 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if the host check fails", 2023-10-18T12:40:13.674 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.674 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.675 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.675 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_stray_daemons": { 2023-10-18T12:40:13.676 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_stray_daemons", 2023-10-18T12:40:13.676 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.676 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.677 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.677 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.677 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.677 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.678 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.678 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-18T12:40:13.678 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.679 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.679 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.679 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_stray_hosts": { 2023-10-18T12:40:13.680 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_stray_hosts", 2023-10-18T12:40:13.680 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.680 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.681 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.681 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.681 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.681 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.682 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.682 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-18T12:40:13.682 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.683 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.683 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.684 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crash", 2023-10-18T12:40:13.684 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.685 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.685 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.685 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.685 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.686 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.686 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.686 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.687 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.687 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.687 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.687 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.688 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.688 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.688 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.688 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.689 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.689 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.689 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.689 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.690 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.690 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.690 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.691 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.691 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.691 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.692 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.692 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.692 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.692 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.693 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.693 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.693 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.693 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.694 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.694 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.694 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.695 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.695 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.695 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.696 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.696 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.696 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.696 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: "retain_interval": { 2023-10-18T12:40:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "retain_interval", 2023-10-18T12:40:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "31536000", 2023-10-18T12:40:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-18T12:40:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_recent_interval": { 2023-10-18T12:40:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_recent_interval", 2023-10-18T12:40:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1209600", 2023-10-18T12:40:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-18T12:40:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.711 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.712 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "dashboard", 2023-10-18T12:40:13.712 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.712 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.712 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.713 INFO:teuthology.orchestra.run.smithi070.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-18T12:40:13.713 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-18T12:40:13.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.713 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.714 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.714 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:13.714 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.714 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.715 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.715 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.715 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.715 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.716 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.716 INFO:teuthology.orchestra.run.smithi070.stdout: "ALERTMANAGER_API_HOST": { 2023-10-18T12:40:13.717 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-18T12:40:13.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.717 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.717 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.718 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.718 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.718 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.718 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.719 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.719 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.719 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.720 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.720 INFO:teuthology.orchestra.run.smithi070.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-18T12:40:13.720 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-18T12:40:13.721 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.721 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.721 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.721 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.722 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.722 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.723 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.723 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.723 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.723 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.724 INFO:teuthology.orchestra.run.smithi070.stdout: "AUDIT_API_ENABLED": { 2023-10-18T12:40:13.724 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "AUDIT_API_ENABLED", 2023-10-18T12:40:13.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.725 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.725 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.725 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.726 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.726 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.726 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.727 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.727 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.727 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.727 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.728 INFO:teuthology.orchestra.run.smithi070.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-18T12:40:13.728 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-18T12:40:13.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.729 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.729 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.729 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.730 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.730 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.730 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.731 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.731 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.731 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.731 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.732 INFO:teuthology.orchestra.run.smithi070.stdout: "ENABLE_BROWSABLE_API": { 2023-10-18T12:40:13.732 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-18T12:40:13.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.733 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.733 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.733 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.734 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.734 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.734 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.734 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.735 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.735 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.735 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.736 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-18T12:40:13.736 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-18T12:40:13.737 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.737 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.737 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.737 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.738 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.738 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.738 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.738 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.739 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.739 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.739 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.740 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-18T12:40:13.740 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-18T12:40:13.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.741 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.741 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.741 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.742 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.742 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.742 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.742 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.743 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.743 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.743 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.744 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-18T12:40:13.744 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-18T12:40:13.744 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.744 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.745 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.745 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.745 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.746 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.746 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.746 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.746 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-18T12:40:13.748 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-18T12:40:13.748 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.748 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.749 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.749 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.749 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.749 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.750 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.750 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.750 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.750 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.751 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.751 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-18T12:40:13.751 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-18T12:40:13.752 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.752 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.752 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.752 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.753 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.753 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.753 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.753 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.754 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.754 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.754 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.755 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-18T12:40:13.755 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-18T12:40:13.755 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.755 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.756 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.756 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.756 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.756 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.757 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.757 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.757 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.758 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.758 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.758 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-18T12:40:13.759 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-18T12:40:13.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.759 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.759 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.760 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.760 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.760 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.760 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.761 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.761 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.761 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.761 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.762 INFO:teuthology.orchestra.run.smithi070.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-18T12:40:13.762 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-18T12:40:13.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.763 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.763 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.763 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.764 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.764 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.764 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.765 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.765 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.765 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.765 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.766 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_PASSWORD": { 2023-10-18T12:40:13.766 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-18T12:40:13.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.767 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.767 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.767 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:13.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.768 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.768 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.769 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.769 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.769 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.770 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.770 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-18T12:40:13.771 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-18T12:40:13.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.771 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.771 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.772 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.772 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.772 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.773 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.773 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.773 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.773 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.774 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.774 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_URL": { 2023-10-18T12:40:13.775 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_URL", 2023-10-18T12:40:13.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.775 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.775 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.776 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.776 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.776 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.776 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.777 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.777 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.777 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.778 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.778 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_USERNAME": { 2023-10-18T12:40:13.778 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-18T12:40:13.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.779 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.779 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.780 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:13.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.780 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.780 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.781 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.781 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.781 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.781 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.782 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-18T12:40:13.782 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-18T12:40:13.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.783 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.783 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.783 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.784 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.784 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.784 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.784 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.785 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.785 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.785 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.786 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-18T12:40:13.786 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-18T12:40:13.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.787 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.787 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.788 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.788 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.788 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.789 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.789 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.789 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.790 INFO:teuthology.orchestra.run.smithi070.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-18T12:40:13.790 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-18T12:40:13.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.791 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.791 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.791 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.792 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.792 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.792 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.792 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.793 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.793 INFO:teuthology.orchestra.run.smithi070.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-18T12:40:13.794 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-18T12:40:13.794 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.794 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.795 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.795 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.795 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.795 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.796 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.796 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.796 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.796 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.797 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.797 INFO:teuthology.orchestra.run.smithi070.stdout: "PROMETHEUS_API_HOST": { 2023-10-18T12:40:13.797 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-18T12:40:13.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.798 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.798 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.799 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.799 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.799 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.799 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.800 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.800 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.800 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.800 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.801 INFO:teuthology.orchestra.run.smithi070.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-18T12:40:13.801 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-18T12:40:13.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.802 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.802 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.802 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.802 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.803 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.803 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.803 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.804 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.804 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.804 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.805 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-18T12:40:13.805 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-18T12:40:13.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.805 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.806 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.806 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.807 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.807 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.807 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.807 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.808 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.808 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.808 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-18T12:40:13.809 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-18T12:40:13.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.809 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.810 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.810 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.810 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.811 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.811 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.811 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.812 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.812 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-18T12:40:13.812 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-18T12:40:13.813 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.813 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.813 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.814 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.814 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.814 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.814 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.815 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.815 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.815 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.815 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.816 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-18T12:40:13.816 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-18T12:40:13.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.817 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.817 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.817 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.817 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.818 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.818 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.818 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.819 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.819 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.819 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.820 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-18T12:40:13.820 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-18T12:40:13.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.820 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.821 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.821 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.821 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.822 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.822 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.822 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.822 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.823 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.823 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.823 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-18T12:40:13.824 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-18T12:40:13.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.824 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.825 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.825 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.825 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.826 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.826 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.826 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.827 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.827 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.829 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-18T12:40:13.830 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-18T12:40:13.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.830 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.831 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.831 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.831 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.831 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.832 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.832 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.832 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.833 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.833 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.833 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_ENABLED": { 2023-10-18T12:40:13.834 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-18T12:40:13.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.834 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.835 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.835 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.835 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.835 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.836 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.836 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.836 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.837 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.837 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.837 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-18T12:40:13.838 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-18T12:40:13.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.838 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.838 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.839 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-18T12:40:13.839 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.839 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.839 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.840 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.840 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.840 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.840 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.841 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-18T12:40:13.841 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-18T12:40:13.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.842 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.842 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.842 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:13.843 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.843 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.843 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.844 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.844 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.844 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.845 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-18T12:40:13.845 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-18T12:40:13.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.846 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.846 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.846 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8", 2023-10-18T12:40:13.847 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.847 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.847 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.849 INFO:teuthology.orchestra.run.smithi070.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-18T12:40:13.849 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-18T12:40:13.850 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.850 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.850 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.850 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "45", 2023-10-18T12:40:13.851 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.851 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.851 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.852 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.852 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.852 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.852 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.853 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_ACCESS_KEY": { 2023-10-18T12:40:13.853 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-18T12:40:13.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.854 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.854 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.854 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.855 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.855 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.855 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.855 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-18T12:40:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-18T12:40:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_SECRET_KEY": { 2023-10-18T12:40:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-18T12:40:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_SSL_VERIFY": { 2023-10-18T12:40:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-18T12:40:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "UNSAFE_TLS_v1_2": { 2023-10-18T12:40:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-18T12:40:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-18T12:40:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-18T12:40:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-18T12:40:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-18T12:40:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-18T12:40:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-18T12:40:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "cross_origin_url": { 2023-10-18T12:40:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cross_origin_url", 2023-10-18T12:40:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "crt_file": { 2023-10-18T12:40:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crt_file", 2023-10-18T12:40:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "debug": { 2023-10-18T12:40:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "debug", 2023-10-18T12:40:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable/disable debug options", 2023-10-18T12:40:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "jwt_token_ttl": { 2023-10-18T12:40:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "jwt_token_ttl", 2023-10-18T12:40:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "28800", 2023-10-18T12:40:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "key_file": { 2023-10-18T12:40:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "key_file", 2023-10-18T12:40:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "motd": { 2023-10-18T12:40:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "motd", 2023-10-18T12:40:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "The message of the day", 2023-10-18T12:40:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "redirect_resolve_ip_addr": { 2023-10-18T12:40:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "redirect_resolve_ip_addr", 2023-10-18T12:40:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "::", 2023-10-18T12:40:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8080", 2023-10-18T12:40:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl": { 2023-10-18T12:40:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl", 2023-10-18T12:40:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl_server_port": { 2023-10-18T12:40:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl_server_port", 2023-10-18T12:40:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8443", 2023-10-18T12:40:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_behaviour": { 2023-10-18T12:40:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_behaviour", 2023-10-18T12:40:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "redirect", 2023-10-18T12:40:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "redirect" 2023-10-18T12:40:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_error_status_code": { 2023-10-18T12:40:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_error_status_code", 2023-10-18T12:40:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "500", 2023-10-18T12:40:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "400", 2023-10-18T12:40:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "599", 2023-10-18T12:40:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.954 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.954 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.954 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "url_prefix": { 2023-10-18T12:40:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "url_prefix", 2023-10-18T12:40:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.956 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:13.956 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.956 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.956 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.957 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.957 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.957 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.957 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.958 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:13.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:13.959 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "devicehealth", 2023-10-18T12:40:13.959 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:13.960 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:13.960 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:13.960 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_monitoring": { 2023-10-18T12:40:13.960 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enable_monitoring", 2023-10-18T12:40:13.961 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.961 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.961 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.962 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.962 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.962 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.962 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.963 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "monitor device health metrics", 2023-10-18T12:40:13.963 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.963 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.963 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.964 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:13.964 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:13.964 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.965 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.965 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.965 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:13.965 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.966 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.966 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.966 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.967 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.967 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.967 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.967 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.968 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.968 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.968 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.968 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.969 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.969 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.970 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:13.970 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:13.970 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.970 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.971 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.971 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.971 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.971 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.972 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.972 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.972 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.972 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.973 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.973 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:13.973 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:13.974 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.974 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.974 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.974 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: "mark_out_threshold": { 2023-10-18T12:40:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mark_out_threshold", 2023-10-18T12:40:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "2419200", 2023-10-18T12:40:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-18T12:40:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": { 2023-10-18T12:40:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pool_name", 2023-10-18T12:40:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "device_health_metrics", 2023-10-18T12:40:13.988 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.988 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.988 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.988 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-18T12:40:13.989 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.989 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.989 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.990 INFO:teuthology.orchestra.run.smithi070.stdout: "retention_period": { 2023-10-18T12:40:13.990 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "retention_period", 2023-10-18T12:40:13.990 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.991 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.991 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.991 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15552000", 2023-10-18T12:40:13.992 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.992 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.992 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.992 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long to retain device health metrics", 2023-10-18T12:40:13.993 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.993 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.993 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.994 INFO:teuthology.orchestra.run.smithi070.stdout: "scrape_frequency": { 2023-10-18T12:40:13.994 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "scrape_frequency", 2023-10-18T12:40:13.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:13.995 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.995 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.995 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "86400", 2023-10-18T12:40:13.995 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.996 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:13.996 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:13.996 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-18T12:40:13.996 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:13.997 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:13.997 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:13.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:13.997 INFO:teuthology.orchestra.run.smithi070.stdout: "self_heal": { 2023-10-18T12:40:13.998 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "self_heal", 2023-10-18T12:40:13.998 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:13.998 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:13.998 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:13.999 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:13.999 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:13.999 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-18T12:40:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.001 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.001 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:14.001 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:14.002 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:14.002 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.002 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.003 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:14.003 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.003 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.003 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.003 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to wake up and check device health", 2023-10-18T12:40:14.004 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.004 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.004 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.005 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_threshold": { 2023-10-18T12:40:14.005 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_threshold", 2023-10-18T12:40:14.005 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:14.006 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.006 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "7257600", 2023-10-18T12:40:14.006 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.007 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.007 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.007 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-18T12:40:14.007 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.008 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.008 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.009 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "diskprediction_local", 2023-10-18T12:40:14.009 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.010 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.010 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.010 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.011 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.011 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.011 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.011 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.012 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.012 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.012 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.012 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.013 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.013 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.013 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.013 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.014 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.014 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.014 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.015 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.015 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.015 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.015 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.016 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.016 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.016 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.017 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.017 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.017 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.017 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.018 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.018 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.018 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.018 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.019 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.019 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.019 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.020 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.020 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.020 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.020 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.021 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.021 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.021 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.021 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.022 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.022 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.022 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.023 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.023 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.023 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.023 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.024 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.024 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.024 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.024 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.025 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.025 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.026 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.026 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.026 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.027 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.027 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.027 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.027 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.028 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.028 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.028 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.029 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.029 INFO:teuthology.orchestra.run.smithi070.stdout: "predict_interval": { 2023-10-18T12:40:14.029 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "predict_interval", 2023-10-18T12:40:14.030 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.030 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.030 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.030 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "86400", 2023-10-18T12:40:14.031 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.031 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.031 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.032 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.032 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.032 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.032 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.033 INFO:teuthology.orchestra.run.smithi070.stdout: "predictor_model": { 2023-10-18T12:40:14.033 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "predictor_model", 2023-10-18T12:40:14.033 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.034 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.034 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.034 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "prophetstor", 2023-10-18T12:40:14.035 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.035 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.035 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.035 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.036 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.036 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.036 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.037 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:14.037 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:14.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.038 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.038 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.038 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:14.038 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.039 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.039 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.039 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.039 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.040 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.040 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.041 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "influx", 2023-10-18T12:40:14.042 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": false, 2023-10-18T12:40:14.042 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "influxdb python module not found", 2023-10-18T12:40:14.042 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.042 INFO:teuthology.orchestra.run.smithi070.stdout: "batch_size": { 2023-10-18T12:40:14.043 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "batch_size", 2023-10-18T12:40:14.043 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.043 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.044 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.044 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5000", 2023-10-18T12:40:14.044 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.044 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.045 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.045 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-18T12:40:14.045 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.045 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.046 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.046 INFO:teuthology.orchestra.run.smithi070.stdout: "database": { 2023-10-18T12:40:14.047 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "database", 2023-10-18T12:40:14.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.047 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.047 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.048 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "ceph", 2023-10-18T12:40:14.048 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.048 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.048 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.049 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-18T12:40:14.049 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.049 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.050 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.050 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": { 2023-10-18T12:40:14.050 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "hostname", 2023-10-18T12:40:14.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.051 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.051 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.052 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.052 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.052 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.052 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.053 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB server hostname", 2023-10-18T12:40:14.053 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.053 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.054 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.054 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:14.054 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:14.055 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:14.055 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.055 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.055 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "30", 2023-10-18T12:40:14.056 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "5", 2023-10-18T12:40:14.056 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.056 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.057 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-18T12:40:14.057 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.057 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.057 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.058 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.058 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.059 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.059 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.059 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.060 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.060 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.060 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.060 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.061 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.061 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.061 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.061 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.062 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.062 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.062 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.063 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.063 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.063 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.064 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.064 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.064 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.064 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.065 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.065 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.065 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.066 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.066 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.066 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.066 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.067 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.067 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.068 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.068 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.068 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.069 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.069 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.069 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.070 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.070 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.070 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.070 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.071 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.071 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.071 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.072 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.072 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.072 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.072 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.073 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.073 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.073 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.074 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.074 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.075 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.075 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.075 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.075 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.076 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.076 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.076 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.076 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.077 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.077 INFO:teuthology.orchestra.run.smithi070.stdout: "password": { 2023-10-18T12:40:14.078 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "password", 2023-10-18T12:40:14.078 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.078 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.078 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.079 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.079 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.079 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.080 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.080 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "password of InfluxDB server user", 2023-10-18T12:40:14.080 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.080 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.081 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.081 INFO:teuthology.orchestra.run.smithi070.stdout: "port": { 2023-10-18T12:40:14.082 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "port", 2023-10-18T12:40:14.082 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.082 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.082 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.083 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8086", 2023-10-18T12:40:14.083 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.083 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.084 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.084 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB server port", 2023-10-18T12:40:14.084 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.085 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.085 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl": { 2023-10-18T12:40:14.086 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl", 2023-10-18T12:40:14.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.086 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.086 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.087 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "false", 2023-10-18T12:40:14.087 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.087 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.087 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.088 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T12:40:14.088 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.088 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.089 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.089 INFO:teuthology.orchestra.run.smithi070.stdout: "threads": { 2023-10-18T12:40:14.089 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "threads", 2023-10-18T12:40:14.090 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.090 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.090 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.091 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:14.091 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:14.091 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "32", 2023-10-18T12:40:14.091 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.092 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-18T12:40:14.092 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.092 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.093 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.093 INFO:teuthology.orchestra.run.smithi070.stdout: "username": { 2023-10-18T12:40:14.093 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "username", 2023-10-18T12:40:14.094 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.094 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.094 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.094 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.095 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.095 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.095 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.096 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "username of InfluxDB server user", 2023-10-18T12:40:14.096 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.096 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.096 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.097 INFO:teuthology.orchestra.run.smithi070.stdout: "verify_ssl": { 2023-10-18T12:40:14.097 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "verify_ssl", 2023-10-18T12:40:14.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.098 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.098 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.098 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "true", 2023-10-18T12:40:14.099 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.099 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.099 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.099 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T12:40:14.100 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.100 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.100 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.102 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "insights", 2023-10-18T12:40:14.102 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.102 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.102 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.103 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.103 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.103 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.104 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.104 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.104 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.104 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.105 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.105 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.105 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.105 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.106 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.106 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.106 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.107 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.107 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.107 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.107 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.108 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.108 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.109 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.109 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.109 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.110 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.110 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.110 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.110 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.111 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.111 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.111 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.112 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.112 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.112 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.113 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.113 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.113 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.113 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.114 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.114 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.114 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.115 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.115 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.115 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.115 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.116 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.116 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.116 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.117 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.117 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.117 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.117 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.118 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.118 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.119 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.119 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.119 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.119 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.120 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.120 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.120 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.120 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.121 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.121 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.121 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.122 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.122 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.122 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.123 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "iostat", 2023-10-18T12:40:14.123 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.124 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.124 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.124 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.124 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.125 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.125 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.125 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.125 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.126 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.126 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.126 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.127 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.127 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.127 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.127 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.128 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.128 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.128 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.128 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.129 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.129 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.129 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.130 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.130 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.130 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.131 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.131 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.131 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.132 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.132 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.132 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.132 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.133 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.133 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.133 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.134 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.134 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.134 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.135 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.135 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.135 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.135 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.136 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.136 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.136 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.136 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.137 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.137 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.137 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.138 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.138 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.138 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.138 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.139 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.139 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.140 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.140 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.140 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.140 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.141 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.141 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.141 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.142 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.142 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.142 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.142 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.143 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.143 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.143 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.144 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "k8sevents", 2023-10-18T12:40:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_event_retention_days": { 2023-10-18T12:40:14.146 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ceph_event_retention_days", 2023-10-18T12:40:14.146 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.146 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.147 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.147 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "7", 2023-10-18T12:40:14.147 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.147 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.148 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.148 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-18T12:40:14.148 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.148 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.149 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.149 INFO:teuthology.orchestra.run.smithi070.stdout: "config_check_secs": { 2023-10-18T12:40:14.150 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_check_secs", 2023-10-18T12:40:14.150 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.150 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.151 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.151 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:14.151 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "10", 2023-10-18T12:40:14.151 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.152 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.152 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-18T12:40:14.152 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.153 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.153 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.154 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.154 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.154 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.155 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.155 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.155 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.155 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.156 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.156 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.156 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.157 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.157 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.157 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.157 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.158 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.158 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.158 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.159 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.159 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.159 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.160 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.160 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.160 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.161 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.161 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.161 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.161 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.162 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.162 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.162 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.163 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.163 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.163 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.164 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.164 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.165 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.165 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.165 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.165 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.166 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.166 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.166 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.166 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.167 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.167 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.167 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.168 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.168 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.168 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.168 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.169 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.169 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.169 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.170 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.170 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.170 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.171 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.171 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.171 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.171 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.172 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.172 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.172 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.173 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.173 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.173 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.174 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "localpool", 2023-10-18T12:40:14.174 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.175 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.175 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.175 INFO:teuthology.orchestra.run.smithi070.stdout: "failure_domain": { 2023-10-18T12:40:14.175 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "failure_domain", 2023-10-18T12:40:14.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.176 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.176 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "host", 2023-10-18T12:40:14.177 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.177 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.177 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.177 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "failure domain for any created local pool", 2023-10-18T12:40:14.178 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-18T12:40:14.178 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.178 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.179 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.179 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.180 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.180 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.180 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.180 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.181 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.181 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.181 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.182 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.182 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.182 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.183 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.183 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.183 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.183 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.184 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.184 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.184 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.185 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.185 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.185 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.186 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.186 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.186 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.186 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.187 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.187 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.187 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.188 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.188 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.188 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.188 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.189 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.189 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.190 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.190 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.190 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.190 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.191 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.191 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.191 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.191 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.192 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.192 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.192 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.192 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.193 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.193 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.193 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.194 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.194 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.195 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.195 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.195 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.196 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.196 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.196 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.196 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.197 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.197 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.197 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.197 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.198 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.198 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.199 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": { 2023-10-18T12:40:14.199 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "min_size", 2023-10-18T12:40:14.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.199 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.200 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.200 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.200 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.201 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.201 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default min_size for any created local pool", 2023-10-18T12:40:14.201 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-18T12:40:14.202 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.202 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.203 INFO:teuthology.orchestra.run.smithi070.stdout: "num_rep": { 2023-10-18T12:40:14.203 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "num_rep", 2023-10-18T12:40:14.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.203 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.204 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.204 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3", 2023-10-18T12:40:14.204 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.204 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.205 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.205 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default replica count for any created local pool", 2023-10-18T12:40:14.205 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.206 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.206 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.207 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": { 2023-10-18T12:40:14.207 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pg_num", 2023-10-18T12:40:14.207 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.207 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.208 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.208 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "128", 2023-10-18T12:40:14.208 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.208 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.209 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.209 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default pg_num for any created local pool", 2023-10-18T12:40:14.209 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.210 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.210 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.210 INFO:teuthology.orchestra.run.smithi070.stdout: "prefix": { 2023-10-18T12:40:14.211 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prefix", 2023-10-18T12:40:14.211 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.211 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.212 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.212 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.212 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.212 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.213 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.213 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "name prefix for any created local pool", 2023-10-18T12:40:14.213 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.214 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.214 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.214 INFO:teuthology.orchestra.run.smithi070.stdout: "subtree": { 2023-10-18T12:40:14.215 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "subtree", 2023-10-18T12:40:14.215 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.215 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.215 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.216 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "rack", 2023-10-18T12:40:14.216 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.216 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.217 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.217 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-18T12:40:14.217 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-18T12:40:14.217 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.218 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.219 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mds_autoscaler", 2023-10-18T12:40:14.219 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.220 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.220 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.220 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.220 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.221 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.221 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.221 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.221 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.221 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.222 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.222 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.222 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.223 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.223 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.223 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.223 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.224 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.224 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.224 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.225 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.225 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.225 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.226 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.226 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.227 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.227 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.227 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.227 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.228 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.228 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.228 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.228 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.229 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.229 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.230 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.230 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.230 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.231 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.231 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.231 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.232 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.232 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.232 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.232 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.233 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.233 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.233 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.233 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.234 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.234 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.234 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.235 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.235 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.235 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.236 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.237 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.237 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.237 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.237 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.238 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.238 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.238 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.241 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.242 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.242 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.244 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mirroring", 2023-10-18T12:40:14.244 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.244 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.244 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.245 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.245 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.245 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.246 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.246 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.246 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.247 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.247 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.247 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.247 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.248 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.248 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.249 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.249 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.250 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.250 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.251 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.251 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "nfs", 2023-10-18T12:40:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "orchestrator", 2023-10-18T12:40:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "fail_fs": { 2023-10-18T12:40:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "fail_fs", 2023-10-18T12:40:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-18T12:40:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator": { 2023-10-18T12:40:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "orchestrator", 2023-10-18T12:40:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm", 2023-10-18T12:40:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "rook", 2023-10-18T12:40:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "test_orchestrator" 2023-10-18T12:40:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Orchestrator backend", 2023-10-18T12:40:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "osd_perf_query", 2023-10-18T12:40:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.345 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.345 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.345 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.346 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.346 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.346 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.346 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.347 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.347 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.347 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.348 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.348 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.348 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.348 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.349 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.349 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.349 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.350 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.350 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.350 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.351 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.351 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.351 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.351 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.352 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.352 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.352 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.353 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.354 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "osd_support", 2023-10-18T12:40:14.354 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.355 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.355 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.355 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.355 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.356 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.356 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.357 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.357 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.357 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.357 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.358 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.358 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.358 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.359 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.359 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.359 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.360 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.360 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.360 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.361 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.361 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.362 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.362 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.362 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.363 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.363 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.363 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.363 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.364 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.364 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.364 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.365 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.365 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.365 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.366 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.366 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.366 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.367 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.367 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.367 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.368 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.368 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.368 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.368 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.369 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.369 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.369 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.369 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.370 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.370 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.370 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.371 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.371 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.371 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.372 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.372 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.373 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.373 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.374 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.374 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.374 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.375 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.375 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.375 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.376 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.376 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.377 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.377 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.378 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pg_autoscaler", 2023-10-18T12:40:14.378 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.378 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.379 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.379 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.379 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.380 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.380 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.381 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.381 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.382 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.382 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.383 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.383 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.384 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.384 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.385 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.385 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.386 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.386 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.386 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.387 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.387 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.387 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.388 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.388 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.388 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.389 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.389 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.389 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.389 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.390 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.391 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.391 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.392 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.392 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.393 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.394 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.395 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.395 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.396 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.396 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.397 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.397 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.398 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.398 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.399 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.399 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.400 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.400 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.401 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.401 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.402 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.402 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.403 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.404 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.404 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.405 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.405 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "threshold": { 2023-10-18T12:40:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "threshold", 2023-10-18T12:40:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3.0", 2023-10-18T12:40:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1.0", 2023-10-18T12:40:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "scaling threshold", 2023-10-18T12:40:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-18T12:40:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "progress", 2023-10-18T12:40:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_pg_recovery_event": { 2023-10-18T12:40:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_pg_recovery_event", 2023-10-18T12:40:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-18T12:40:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "enabled": { 2023-10-18T12:40:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enabled", 2023-10-18T12:40:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "max_completed_events": { 2023-10-18T12:40:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_completed_events", 2023-10-18T12:40:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "50", 2023-10-18T12:40:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "number of past completed events to remember", 2023-10-18T12:40:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long the module is going to sleep", 2023-10-18T12:40:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prometheus", 2023-10-18T12:40:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "cache": { 2023-10-18T12:40:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cache", 2023-10-18T12:40:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "exclude_perf_counters": { 2023-10-18T12:40:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "exclude_perf_counters", 2023-10-18T12:40:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-18T12:40:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-18T12:40:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_stats_pools": { 2023-10-18T12:40:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_stats_pools", 2023-10-18T12:40:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-18T12:40:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-18T12:40:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "300", 2023-10-18T12:40:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.508 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.508 INFO:teuthology.orchestra.run.smithi070.stdout: "scrape_interval": { 2023-10-18T12:40:14.509 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "scrape_interval", 2023-10-18T12:40:14.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:14.509 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.510 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15.0", 2023-10-18T12:40:14.510 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.510 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.511 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.511 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.511 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.512 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.512 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.513 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:14.513 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:14.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.513 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.514 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.514 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "::", 2023-10-18T12:40:14.514 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.515 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.515 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.515 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-18T12:40:14.516 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.516 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.516 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.517 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:14.517 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:14.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.518 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.518 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.518 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "9283", 2023-10-18T12:40:14.519 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.519 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.519 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.519 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-18T12:40:14.520 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.520 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.520 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.521 INFO:teuthology.orchestra.run.smithi070.stdout: "stale_cache_strategy": { 2023-10-18T12:40:14.521 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "stale_cache_strategy", 2023-10-18T12:40:14.521 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.522 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.522 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.522 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "log", 2023-10-18T12:40:14.523 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.523 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.523 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.524 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.524 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.524 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.524 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.525 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_behaviour": { 2023-10-18T12:40:14.525 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_behaviour", 2023-10-18T12:40:14.526 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.526 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.526 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.527 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "default", 2023-10-18T12:40:14.527 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.527 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.528 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.528 INFO:teuthology.orchestra.run.smithi070.stdout: "default", 2023-10-18T12:40:14.528 INFO:teuthology.orchestra.run.smithi070.stdout: "error" 2023-10-18T12:40:14.528 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.529 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.529 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.530 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.530 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_error_status_code": { 2023-10-18T12:40:14.530 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_error_status_code", 2023-10-18T12:40:14.531 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.531 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.531 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.532 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "500", 2023-10-18T12:40:14.532 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "400", 2023-10-18T12:40:14.532 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "599", 2023-10-18T12:40:14.533 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.533 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.533 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.534 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.534 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.534 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.534 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.535 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_support", 2023-10-18T12:40:14.536 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.536 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.536 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.537 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.537 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.537 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.537 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.538 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.538 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.538 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.539 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.539 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.539 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.540 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.540 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.540 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.540 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.541 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.541 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.541 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.549 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.549 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.549 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.550 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.550 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.550 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.551 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.551 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.551 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.551 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.552 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.552 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.552 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.553 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.553 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.554 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.554 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.554 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.554 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.555 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.555 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.555 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.556 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.556 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.556 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.557 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.557 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.557 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.558 INFO:teuthology.orchestra.run.smithi070.stdout: "max_concurrent_snap_create": { 2023-10-18T12:40:14.558 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_concurrent_snap_create", 2023-10-18T12:40:14.558 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.559 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.559 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.559 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:14.560 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.560 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.560 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.561 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.561 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.561 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.562 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.562 INFO:teuthology.orchestra.run.smithi070.stdout: "mirror_snapshot_schedule": { 2023-10-18T12:40:14.563 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mirror_snapshot_schedule", 2023-10-18T12:40:14.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.563 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.564 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.564 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.564 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.565 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.565 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.565 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.566 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.566 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.566 INFO:teuthology.orchestra.run.smithi070.stdout: "trash_purge_schedule": { 2023-10-18T12:40:14.567 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "trash_purge_schedule", 2023-10-18T12:40:14.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.567 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.568 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.568 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.568 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.569 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.569 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.569 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.570 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.570 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.570 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.570 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.572 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "restful", 2023-10-18T12:40:14.572 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.572 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.573 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.573 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_auth": { 2023-10-18T12:40:14.573 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enable_auth", 2023-10-18T12:40:14.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.574 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.574 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.574 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.575 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.575 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.575 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.576 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.576 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.576 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.577 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.577 INFO:teuthology.orchestra.run.smithi070.stdout: "key_file": { 2023-10-18T12:40:14.578 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "key_file", 2023-10-18T12:40:14.578 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.578 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.578 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.579 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.579 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.579 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.580 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.580 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.580 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.581 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.581 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.582 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.582 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.582 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.583 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.583 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.583 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.583 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.584 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.584 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.584 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.585 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.585 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.585 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.586 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.586 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.586 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.586 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.587 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.587 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.587 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.588 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.588 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.589 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.589 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.589 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.590 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.590 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.590 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.590 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.591 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.591 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.591 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.592 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.592 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.593 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.593 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.593 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.594 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.594 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.594 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.594 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.595 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.595 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.595 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.596 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.596 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.596 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.597 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.597 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.597 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.598 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.598 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.598 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.599 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.599 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.599 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.600 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.600 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.601 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.601 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.601 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.602 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.602 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.602 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.602 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.603 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:14.603 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:14.604 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.604 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.604 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.604 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.605 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.605 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.605 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.606 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.606 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.606 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.607 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.607 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:14.607 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:14.608 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.608 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.608 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.609 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.609 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.609 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.609 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.610 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.610 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.610 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.611 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.611 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.611 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.612 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rgw", 2023-10-18T12:40:14.612 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.613 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.613 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.613 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.614 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.614 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.614 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.614 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.615 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.615 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.615 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.616 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.616 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.616 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.617 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.617 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.617 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.618 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.618 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.618 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.618 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.619 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.619 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.619 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.620 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.620 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.620 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.621 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.621 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.621 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.621 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.622 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.622 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.622 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.623 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.623 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.623 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.624 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.624 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.624 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.625 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.625 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.625 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.626 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.626 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.626 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.626 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.627 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.627 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.627 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.628 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.628 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.628 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.629 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.629 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.629 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.630 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.630 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.631 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.631 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.631 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.631 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.632 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.632 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.632 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.633 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.633 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.633 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.634 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.634 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.634 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.635 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rook", 2023-10-18T12:40:14.635 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.636 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.636 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.636 INFO:teuthology.orchestra.run.smithi070.stdout: "drive_group_interval": { 2023-10-18T12:40:14.637 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "drive_group_interval", 2023-10-18T12:40:14.637 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:14.637 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.637 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.638 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "300.0", 2023-10-18T12:40:14.638 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.638 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.639 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.639 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-18T12:40:14.639 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.640 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.640 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.640 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.641 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.641 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.641 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.642 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.642 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.642 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.643 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.643 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.643 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.644 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.644 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.644 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.645 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.645 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.645 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.645 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.646 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.646 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.646 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.647 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.647 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.648 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.648 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.648 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.650 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.650 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.650 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.650 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.651 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.654 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.654 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.655 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.655 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.655 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.656 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.656 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.656 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.657 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.657 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.657 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.657 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.658 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.658 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.658 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.659 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.659 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.659 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.660 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.660 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.661 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.661 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.661 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.662 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.662 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.662 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.663 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.663 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.663 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.664 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.664 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.664 INFO:teuthology.orchestra.run.smithi070.stdout: "storage_class": { 2023-10-18T12:40:14.665 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "storage_class", 2023-10-18T12:40:14.665 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.665 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.666 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.666 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "local", 2023-10-18T12:40:14.666 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.666 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.667 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.667 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-18T12:40:14.667 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.668 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.668 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.668 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.669 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.669 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "selftest", 2023-10-18T12:40:14.670 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.670 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.670 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.671 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.671 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.672 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.672 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.672 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.672 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.673 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.673 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.673 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.674 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.674 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.674 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.674 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.675 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.675 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.675 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.676 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.676 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.676 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.677 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.677 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.678 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.678 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.678 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.679 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.679 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.679 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.680 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.680 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.680 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.680 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.681 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.681 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.682 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.682 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.682 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.683 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.683 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.683 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.684 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.684 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.684 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.684 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.685 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.685 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.685 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.686 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.686 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.686 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.687 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.687 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.687 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.688 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.688 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.689 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.689 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.689 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.689 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.690 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.690 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.690 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.691 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.691 INFO:teuthology.orchestra.run.smithi070.stdout: "roption1": { 2023-10-18T12:40:14.692 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "roption1", 2023-10-18T12:40:14.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.692 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.693 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.693 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.693 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.694 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.694 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.694 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.695 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.695 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.695 INFO:teuthology.orchestra.run.smithi070.stdout: "roption2": { 2023-10-18T12:40:14.696 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "roption2", 2023-10-18T12:40:14.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.696 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.697 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.697 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "xyz", 2023-10-18T12:40:14.697 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.698 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.698 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.698 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.698 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.699 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.699 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.700 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption1": { 2023-10-18T12:40:14.700 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption1", 2023-10-18T12:40:14.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.700 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.701 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.701 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.701 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.702 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.702 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.702 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.703 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.703 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.703 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.704 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption2": { 2023-10-18T12:40:14.704 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption2", 2023-10-18T12:40:14.704 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.705 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.705 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.705 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.705 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.706 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.706 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.706 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.707 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.707 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.707 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.708 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption3": { 2023-10-18T12:40:14.708 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption3", 2023-10-18T12:40:14.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:14.709 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.709 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.709 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.710 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.710 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.710 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.710 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.711 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.711 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.711 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.712 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption4": { 2023-10-18T12:40:14.712 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption4", 2023-10-18T12:40:14.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.713 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.713 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.713 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.714 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.714 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.714 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.715 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.715 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.715 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.716 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.716 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption5": { 2023-10-18T12:40:14.716 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption5", 2023-10-18T12:40:14.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.717 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.717 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.718 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.718 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.718 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.718 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.719 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.719 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.719 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.720 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.720 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption6": { 2023-10-18T12:40:14.721 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption6", 2023-10-18T12:40:14.721 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.721 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.721 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.722 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.722 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.723 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.723 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.723 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.723 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.724 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.724 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption7": { 2023-10-18T12:40:14.725 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption7", 2023-10-18T12:40:14.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.725 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.726 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.726 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.726 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:14.726 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "42", 2023-10-18T12:40:14.727 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.727 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.727 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.728 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.728 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.729 INFO:teuthology.orchestra.run.smithi070.stdout: "testkey": { 2023-10-18T12:40:14.729 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testkey", 2023-10-18T12:40:14.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.729 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.730 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.730 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.730 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.731 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.731 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.731 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.732 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.732 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.732 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.733 INFO:teuthology.orchestra.run.smithi070.stdout: "testlkey": { 2023-10-18T12:40:14.733 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testlkey", 2023-10-18T12:40:14.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.734 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.734 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.734 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.735 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.735 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.735 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.736 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.736 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.736 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.736 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.737 INFO:teuthology.orchestra.run.smithi070.stdout: "testnewline": { 2023-10-18T12:40:14.737 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testnewline", 2023-10-18T12:40:14.738 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.738 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.738 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.739 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.739 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.739 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.739 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.740 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.740 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.740 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.741 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.741 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.741 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.742 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snap_schedule", 2023-10-18T12:40:14.742 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.743 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.743 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.743 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_m_granularity": { 2023-10-18T12:40:14.744 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_m_granularity", 2023-10-18T12:40:14.744 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.744 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.745 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.745 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.745 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.745 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.746 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.746 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow minute scheduled snapshots", 2023-10-18T12:40:14.746 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.747 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.747 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.748 INFO:teuthology.orchestra.run.smithi070.stdout: "dump_on_update": { 2023-10-18T12:40:14.748 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "dump_on_update", 2023-10-18T12:40:14.748 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.748 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.749 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.749 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.749 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.750 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.750 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.750 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "dump database to debug log on update", 2023-10-18T12:40:14.751 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.751 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.751 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.752 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.752 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.752 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.753 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.753 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.753 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.754 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.754 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.754 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.755 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.755 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.755 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.756 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.756 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.756 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.756 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.757 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.757 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.757 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.758 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.758 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.759 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.759 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.759 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.760 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.760 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.760 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.761 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.761 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.761 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.762 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.762 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.762 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.763 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.763 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.764 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.764 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.764 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.765 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.765 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.765 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.765 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.766 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.766 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.766 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.767 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.767 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.767 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.767 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.768 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.768 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "stats", 2023-10-18T12:40:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "status", 2023-10-18T12:40:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "telegraf", 2023-10-18T12:40:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.822 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.822 INFO:teuthology.orchestra.run.smithi070.stdout: "address": { 2023-10-18T12:40:14.822 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "address", 2023-10-18T12:40:14.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.823 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.823 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-18T12:40:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15", 2023-10-18T12:40:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "telemetry", 2023-10-18T12:40:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_basic": { 2023-10-18T12:40:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_basic", 2023-10-18T12:40:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-18T12:40:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_crash": { 2023-10-18T12:40:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_crash", 2023-10-18T12:40:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-18T12:40:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_device": { 2023-10-18T12:40:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_device", 2023-10-18T12:40:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.866 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:14.866 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.866 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.867 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.867 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-18T12:40:14.867 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.868 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.868 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.869 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_ident": { 2023-10-18T12:40:14.869 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_ident", 2023-10-18T12:40:14.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.869 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.870 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.870 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.870 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.871 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.871 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.871 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-18T12:40:14.872 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.872 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.873 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_perf": { 2023-10-18T12:40:14.873 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_perf", 2023-10-18T12:40:14.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.874 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.874 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.874 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.875 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.875 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.875 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.875 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-18T12:40:14.876 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.876 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.876 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.877 INFO:teuthology.orchestra.run.smithi070.stdout: "contact": { 2023-10-18T12:40:14.877 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "contact", 2023-10-18T12:40:14.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.878 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.878 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.878 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.879 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.879 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.879 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.880 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.880 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.880 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.881 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.881 INFO:teuthology.orchestra.run.smithi070.stdout: "description": { 2023-10-18T12:40:14.881 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "description", 2023-10-18T12:40:14.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.882 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.882 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.883 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.883 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.883 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.884 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.884 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.884 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.884 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.885 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.885 INFO:teuthology.orchestra.run.smithi070.stdout: "device_url": { 2023-10-18T12:40:14.885 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_url", 2023-10-18T12:40:14.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.886 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.886 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.887 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-18T12:40:14.887 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.887 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.888 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.888 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.888 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.888 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.889 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.889 INFO:teuthology.orchestra.run.smithi070.stdout: "enabled": { 2023-10-18T12:40:14.890 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enabled", 2023-10-18T12:40:14.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.890 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.890 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.891 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.891 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.891 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.892 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.892 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.892 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.893 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.893 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.893 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:14.894 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:14.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.894 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.895 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.895 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "24", 2023-10-18T12:40:14.895 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "8", 2023-10-18T12:40:14.896 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.896 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.896 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.896 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.897 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.897 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.898 INFO:teuthology.orchestra.run.smithi070.stdout: "last_opt_revision": { 2023-10-18T12:40:14.898 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "last_opt_revision", 2023-10-18T12:40:14.898 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.898 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.899 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.899 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1", 2023-10-18T12:40:14.899 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.900 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.900 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.900 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.901 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.901 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.901 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.902 INFO:teuthology.orchestra.run.smithi070.stdout: "leaderboard": { 2023-10-18T12:40:14.902 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "leaderboard", 2023-10-18T12:40:14.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "leaderboard_description": { 2023-10-18T12:40:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "leaderboard_description", 2023-10-18T12:40:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "organization": { 2023-10-18T12:40:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "organization", 2023-10-18T12:40:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "proxy": { 2023-10-18T12:40:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "proxy", 2023-10-18T12:40:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.937 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.937 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.937 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.937 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.938 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.938 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.938 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.939 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.939 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.940 INFO:teuthology.orchestra.run.smithi070.stdout: "url": { 2023-10-18T12:40:14.940 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "url", 2023-10-18T12:40:14.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.940 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.941 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.941 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-18T12:40:14.941 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.942 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.942 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.942 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.943 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.943 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.943 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "test_orchestrator", 2023-10-18T12:40:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.952 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.952 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.953 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.953 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.953 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "volumes", 2023-10-18T12:40:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "max_concurrent_clones": { 2023-10-18T12:40:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_concurrent_clones", 2023-10-18T12:40:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "4", 2023-10-18T12:40:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-18T12:40:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "periodic_async_work": { 2023-10-18T12:40:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "periodic_async_work", 2023-10-18T12:40:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Periodically check for async work", 2023-10-18T12:40:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "snapshot_clone_delay": { 2023-10-18T12:40:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snapshot_clone_delay", 2023-10-18T12:40:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-18T12:40:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix", 2023-10-18T12:40:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "discovery_interval": { 2023-10-18T12:40:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "discovery_interval", 2023-10-18T12:40:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "100", 2023-10-18T12:40:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.008 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.008 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.009 INFO:teuthology.orchestra.run.smithi070.stdout: "identifier": { 2023-10-18T12:40:15.009 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "identifier", 2023-10-18T12:40:15.009 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.009 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.010 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.010 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.010 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.011 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.011 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.011 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.012 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.012 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.012 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.013 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:15.013 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:15.013 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.014 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.014 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.014 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:15.015 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.015 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.015 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.016 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.016 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.016 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.016 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.017 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.017 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.018 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.018 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.018 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.019 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.019 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.019 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.020 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.020 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.020 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.021 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.021 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.021 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.021 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.022 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.022 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.022 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.023 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.023 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.023 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.024 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.024 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.024 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.025 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.025 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.025 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.026 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.026 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.026 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.026 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.027 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.027 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.028 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.028 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.028 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.029 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.029 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.029 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.029 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.030 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.030 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.030 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.031 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.031 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.031 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.032 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.032 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.032 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.032 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.033 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.033 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.034 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.034 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.034 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.034 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.035 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.035 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.035 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.036 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.036 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.036 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.036 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.037 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.037 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.038 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_host": { 2023-10-18T12:40:15.038 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_host", 2023-10-18T12:40:15.038 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.039 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.039 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.039 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.039 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.040 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.040 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.040 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.041 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.041 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.041 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.042 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_port": { 2023-10-18T12:40:15.042 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_port", 2023-10-18T12:40:15.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.043 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.043 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.043 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10051", 2023-10-18T12:40:15.043 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.044 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.044 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.044 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.045 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.045 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.045 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.046 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_sender": { 2023-10-18T12:40:15.046 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_sender", 2023-10-18T12:40:15.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.047 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.047 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.047 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-18T12:40:15.048 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.048 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.048 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.049 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.049 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.049 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.049 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.051 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:15.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.051 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.052 INFO:teuthology.orchestra.run.smithi070.stdout: "modules": [ 2023-10-18T12:40:15.052 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm", 2023-10-18T12:40:15.052 INFO:teuthology.orchestra.run.smithi070.stdout: "dashboard", 2023-10-18T12:40:15.052 INFO:teuthology.orchestra.run.smithi070.stdout: "iostat", 2023-10-18T12:40:15.053 INFO:teuthology.orchestra.run.smithi070.stdout: "nfs", 2023-10-18T12:40:15.053 INFO:teuthology.orchestra.run.smithi070.stdout: "restful", 2023-10-18T12:40:15.053 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_schedule" 2023-10-18T12:40:15.054 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.054 INFO:teuthology.orchestra.run.smithi070.stdout: "available_modules": [ 2023-10-18T12:40:15.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.055 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "alerts", 2023-10-18T12:40:15.055 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.055 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.055 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.056 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:15.056 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:15.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.057 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.057 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.057 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:15.057 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How frequently to reexamine health status", 2023-10-18T12:40:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.083 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.083 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_destination": { 2023-10-18T12:40:15.083 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_destination", 2023-10-18T12:40:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Email address to send alerts to", 2023-10-18T12:40:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_from_name": { 2023-10-18T12:40:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_from_name", 2023-10-18T12:40:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.089 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "Ceph", 2023-10-18T12:40:15.089 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.089 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.090 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.090 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Email From: name", 2023-10-18T12:40:15.090 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.091 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.091 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.091 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_host": { 2023-10-18T12:40:15.092 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_host", 2023-10-18T12:40:15.092 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.092 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.093 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.093 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.093 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.094 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.094 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.094 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP server", 2023-10-18T12:40:15.094 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.095 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.095 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.096 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_password": { 2023-10-18T12:40:15.096 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_password", 2023-10-18T12:40:15.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.096 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.097 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.097 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.097 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.098 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.098 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.098 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Password to authenticate with", 2023-10-18T12:40:15.099 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.099 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.099 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.100 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_port": { 2023-10-18T12:40:15.100 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_port", 2023-10-18T12:40:15.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.101 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.101 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.101 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "465", 2023-10-18T12:40:15.101 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.102 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.102 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.102 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP port", 2023-10-18T12:40:15.103 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.103 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.103 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.104 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_sender": { 2023-10-18T12:40:15.104 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_sender", 2023-10-18T12:40:15.104 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.105 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.105 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.105 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.105 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.106 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.106 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.106 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SMTP envelope sender", 2023-10-18T12:40:15.107 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.107 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.107 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.108 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_ssl": { 2023-10-18T12:40:15.108 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_ssl", 2023-10-18T12:40:15.108 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.109 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.109 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.109 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.109 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.110 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.110 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.110 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-18T12:40:15.111 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.111 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.111 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.112 INFO:teuthology.orchestra.run.smithi070.stdout: "smtp_user": { 2023-10-18T12:40:15.112 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "smtp_user", 2023-10-18T12:40:15.113 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.113 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.113 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.113 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.114 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.114 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.114 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.115 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "User to authenticate as", 2023-10-18T12:40:15.115 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.115 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.116 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.117 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "balancer", 2023-10-18T12:40:15.117 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.118 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.118 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.118 INFO:teuthology.orchestra.run.smithi070.stdout: "active": { 2023-10-18T12:40:15.119 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "active", 2023-10-18T12:40:15.119 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.119 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.119 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.120 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.120 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.120 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.121 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.121 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "automatically balance PGs across cluster", 2023-10-18T12:40:15.121 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.121 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.122 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.122 INFO:teuthology.orchestra.run.smithi070.stdout: "begin_time": { 2023-10-18T12:40:15.123 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "begin_time", 2023-10-18T12:40:15.123 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.123 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.124 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.124 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0000", 2023-10-18T12:40:15.124 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.124 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.125 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.125 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "beginning time of day to automatically balance", 2023-10-18T12:40:15.125 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T12:40:15.126 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.126 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.127 INFO:teuthology.orchestra.run.smithi070.stdout: "begin_weekday": { 2023-10-18T12:40:15.127 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "begin_weekday", 2023-10-18T12:40:15.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:15.127 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.128 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.128 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:15.128 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0", 2023-10-18T12:40:15.129 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "6", 2023-10-18T12:40:15.129 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.129 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-18T12:40:15.129 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T12:40:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.131 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_max_iterations": { 2023-10-18T12:40:15.131 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_max_iterations", 2023-10-18T12:40:15.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:15.131 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.132 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.132 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "25", 2023-10-18T12:40:15.132 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:15.133 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "250", 2023-10-18T12:40:15.133 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.133 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-18T12:40:15.133 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.134 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.134 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.135 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_metrics": { 2023-10-18T12:40:15.135 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_metrics", 2023-10-18T12:40:15.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.136 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.136 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.136 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "pgs,objects,bytes", 2023-10-18T12:40:15.137 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.137 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.137 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.137 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-18T12:40:15.138 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-18T12:40:15.138 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.138 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.139 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_compat_step": { 2023-10-18T12:40:15.139 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crush_compat_step", 2023-10-18T12:40:15.140 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:15.140 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.140 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.141 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0.5", 2023-10-18T12:40:15.141 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0.001", 2023-10-18T12:40:15.141 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "0.999", 2023-10-18T12:40:15.141 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.142 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "aggressiveness of optimization", 2023-10-18T12:40:15.142 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-18T12:40:15.143 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.143 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.143 INFO:teuthology.orchestra.run.smithi070.stdout: "end_time": { 2023-10-18T12:40:15.144 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "end_time", 2023-10-18T12:40:15.144 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.144 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.145 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.145 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "2359", 2023-10-18T12:40:15.145 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.145 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.146 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.146 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "ending time of day to automatically balance", 2023-10-18T12:40:15.146 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T12:40:15.147 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.147 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.147 INFO:teuthology.orchestra.run.smithi070.stdout: "end_weekday": { 2023-10-18T12:40:15.148 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "end_weekday", 2023-10-18T12:40:15.148 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:15.148 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.149 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.149 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:15.149 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "0", 2023-10-18T12:40:15.150 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "6", 2023-10-18T12:40:15.150 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.150 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-18T12:40:15.151 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T12:40:15.151 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.151 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.152 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.152 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.152 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.153 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.153 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.153 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.154 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.154 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.154 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.154 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.155 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.155 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.155 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.156 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.156 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.156 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.156 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.157 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.157 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.157 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.158 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.158 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.159 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.159 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.159 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.159 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.160 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.160 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.160 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.161 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.161 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.161 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.161 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.162 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.162 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.163 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.163 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.163 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.164 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.164 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.164 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.165 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.165 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.165 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.166 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.166 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.166 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.167 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.167 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.167 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.167 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.168 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.168 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.169 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.169 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.169 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.170 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.170 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.170 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.171 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.171 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.171 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.171 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.172 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.172 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.172 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.173 INFO:teuthology.orchestra.run.smithi070.stdout: "min_score": { 2023-10-18T12:40:15.173 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "min_score", 2023-10-18T12:40:15.173 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:15.174 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.174 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.174 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:15.175 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.175 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.175 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.175 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-18T12:40:15.176 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.176 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.176 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.177 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": { 2023-10-18T12:40:15.177 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mode", 2023-10-18T12:40:15.178 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.178 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.178 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.178 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "upmap", 2023-10-18T12:40:15.179 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.179 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.179 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.180 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-compat", 2023-10-18T12:40:15.180 INFO:teuthology.orchestra.run.smithi070.stdout: "none", 2023-10-18T12:40:15.180 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap" 2023-10-18T12:40:15.181 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.181 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Balancer mode", 2023-10-18T12:40:15.181 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.181 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.182 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.182 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_ids": { 2023-10-18T12:40:15.183 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pool_ids", 2023-10-18T12:40:15.183 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.183 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.184 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.184 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.184 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.185 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.185 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.185 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-18T12:40:15.185 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.186 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.187 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:15.187 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:15.187 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.188 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.188 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.188 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:15.188 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.189 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.189 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.189 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-18T12:40:15.190 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.190 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.190 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.191 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap_max_deviation": { 2023-10-18T12:40:15.191 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "upmap_max_deviation", 2023-10-18T12:40:15.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.192 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.192 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.192 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:15.193 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:15.193 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.193 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.194 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-18T12:40:15.194 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-18T12:40:15.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.195 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.195 INFO:teuthology.orchestra.run.smithi070.stdout: "upmap_max_optimizations": { 2023-10-18T12:40:15.196 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "upmap_max_optimizations", 2023-10-18T12:40:15.196 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:15.196 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.196 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.197 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.197 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.197 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.198 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.198 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-18T12:40:15.198 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.198 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.199 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.200 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephadm", 2023-10-18T12:40:15.200 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.201 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.201 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.201 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_down_multiplier": { 2023-10-18T12:40:15.202 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_down_multiplier", 2023-10-18T12:40:15.202 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:15.202 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.203 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.203 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3.0", 2023-10-18T12:40:15.203 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.203 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.204 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.204 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-18T12:40:15.204 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.205 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.205 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.205 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_refresh_rate": { 2023-10-18T12:40:15.206 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_refresh_rate", 2023-10-18T12:40:15.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.206 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.207 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.207 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "20", 2023-10-18T12:40:15.207 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.208 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.208 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.208 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-18T12:40:15.208 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.209 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.209 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.210 INFO:teuthology.orchestra.run.smithi070.stdout: "agent_starting_port": { 2023-10-18T12:40:15.210 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "agent_starting_port", 2023-10-18T12:40:15.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.211 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.211 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.211 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "4721", 2023-10-18T12:40:15.211 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.212 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.212 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.212 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-18T12:40:15.213 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.213 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.213 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.214 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_ptrace": { 2023-10-18T12:40:15.214 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_ptrace", 2023-10-18T12:40:15.215 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.215 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.215 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.216 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.216 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.216 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.217 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.217 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-18T12:40:15.217 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-18T12:40:15.218 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.218 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.218 INFO:teuthology.orchestra.run.smithi070.stdout: "autotune_interval": { 2023-10-18T12:40:15.219 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "autotune_interval", 2023-10-18T12:40:15.219 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.219 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.220 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.220 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:15.220 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.221 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.221 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.221 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-18T12:40:15.221 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.222 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.222 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.223 INFO:teuthology.orchestra.run.smithi070.stdout: "autotune_memory_target_ratio": { 2023-10-18T12:40:15.223 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "autotune_memory_target_ratio", 2023-10-18T12:40:15.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0.7", 2023-10-18T12:40:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-18T12:40:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm_log_destination": { 2023-10-18T12:40:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephadm_log_destination", 2023-10-18T12:40:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.229 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.229 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.229 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "file", 2023-10-18T12:40:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "file,syslog", 2023-10-18T12:40:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "syslog" 2023-10-18T12:40:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-18T12:40:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "cgroups_split": { 2023-10-18T12:40:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cgroups_split", 2023-10-18T12:40:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-18T12:40:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "config_checks_enabled": { 2023-10-18T12:40:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_checks_enabled", 2023-10-18T12:40:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.238 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.238 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.238 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.239 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.239 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.239 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-18T12:40:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "config_dashboard": { 2023-10-18T12:40:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_dashboard", 2023-10-18T12:40:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-18T12:40:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_alertmanager": { 2023-10-18T12:40:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_alertmanager", 2023-10-18T12:40:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-18T12:40:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_base": { 2023-10-18T12:40:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_base", 2023-10-18T12:40:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-18T12:40:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Container image name, without the tag", 2023-10-18T12:40:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_elasticsearch": { 2023-10-18T12:40:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_elasticsearch", 2023-10-18T12:40:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-18T12:40:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "elasticsearch container image", 2023-10-18T12:40:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_grafana": { 2023-10-18T12:40:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_grafana", 2023-10-18T12:40:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-18T12:40:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_haproxy": { 2023-10-18T12:40:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_haproxy", 2023-10-18T12:40:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-18T12:40:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "HAproxy container image", 2023-10-18T12:40:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_agent": { 2023-10-18T12:40:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_agent", 2023-10-18T12:40:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-18T12:40:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger agent container image", 2023-10-18T12:40:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.271 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_collector": { 2023-10-18T12:40:15.271 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_collector", 2023-10-18T12:40:15.271 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.271 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-18T12:40:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger collector container image", 2023-10-18T12:40:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_jaeger_query": { 2023-10-18T12:40:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_jaeger_query", 2023-10-18T12:40:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-18T12:40:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Jaeger query container image", 2023-10-18T12:40:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_keepalived": { 2023-10-18T12:40:15.281 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_keepalived", 2023-10-18T12:40:15.281 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.281 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.281 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-18T12:40:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Keepalived container image", 2023-10-18T12:40:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_loki": { 2023-10-18T12:40:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_loki", 2023-10-18T12:40:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-18T12:40:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Loki container image", 2023-10-18T12:40:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_node_exporter": { 2023-10-18T12:40:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_node_exporter", 2023-10-18T12:40:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-18T12:40:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_nvmeof": { 2023-10-18T12:40:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_nvmeof", 2023-10-18T12:40:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-18T12:40:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Nvme-of container image", 2023-10-18T12:40:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_prometheus": { 2023-10-18T12:40:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_prometheus", 2023-10-18T12:40:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-18T12:40:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Prometheus container image", 2023-10-18T12:40:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_promtail": { 2023-10-18T12:40:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_promtail", 2023-10-18T12:40:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-18T12:40:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Promtail container image", 2023-10-18T12:40:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image_snmp_gateway": { 2023-10-18T12:40:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_image_snmp_gateway", 2023-10-18T12:40:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-18T12:40:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "SNMP Gateway container image", 2023-10-18T12:40:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "container_init": { 2023-10-18T12:40:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "container_init", 2023-10-18T12:40:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Run podman/docker with `--init`", 2023-10-18T12:40:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "daemon_cache_timeout": { 2023-10-18T12:40:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "daemon_cache_timeout", 2023-10-18T12:40:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-18T12:40:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "default_cephadm_command_timeout": { 2023-10-18T12:40:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "default_cephadm_command_timeout", 2023-10-18T12:40:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "900", 2023-10-18T12:40:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-18T12:40:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "default_registry": { 2023-10-18T12:40:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "default_registry", 2023-10-18T12:40:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "docker.io", 2023-10-18T12:40:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-18T12:40:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "device_cache_timeout": { 2023-10-18T12:40:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_cache_timeout", 2023-10-18T12:40:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1800", 2023-10-18T12:40:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache device inventory", 2023-10-18T12:40:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "device_enhanced_scan": { 2023-10-18T12:40:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_enhanced_scan", 2023-10-18T12:40:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-18T12:40:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "facts_cache_timeout": { 2023-10-18T12:40:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "facts_cache_timeout", 2023-10-18T12:40:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "seconds to cache host facts data", 2023-10-18T12:40:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "host_check_interval": { 2023-10-18T12:40:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "host_check_interval", 2023-10-18T12:40:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to perform a host check", 2023-10-18T12:40:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "inventory_list_all": { 2023-10-18T12:40:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "inventory_list_all", 2023-10-18T12:40:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-18T12:40:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "log_refresh_metadata": { 2023-10-18T12:40:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_refresh_metadata", 2023-10-18T12:40:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-18T12:40:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-18T12:40:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-18T12:40:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-18T12:40:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-18T12:40:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-18T12:40:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-18T12:40:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "*", 2023-10-18T12:40:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-18T12:40:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "max_count_per_host": { 2023-10-18T12:40:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_count_per_host", 2023-10-18T12:40:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "max number of daemons per service per host", 2023-10-18T12:40:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd_draining_count": { 2023-10-18T12:40:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_osd_draining_count", 2023-10-18T12:40:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.387 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.387 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.387 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.388 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-18T12:40:15.388 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.388 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.388 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.389 INFO:teuthology.orchestra.run.smithi070.stdout: "migration_current": { 2023-10-18T12:40:15.389 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "migration_current", 2023-10-18T12:40:15.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.390 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.390 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.391 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.391 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.391 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.391 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.392 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "internal - do not modify", 2023-10-18T12:40:15.392 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.392 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.393 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.393 INFO:teuthology.orchestra.run.smithi070.stdout: "mode": { 2023-10-18T12:40:15.393 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mode", 2023-10-18T12:40:15.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.394 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.394 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.395 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "root", 2023-10-18T12:40:15.395 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.395 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.395 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.396 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm-package", 2023-10-18T12:40:15.396 INFO:teuthology.orchestra.run.smithi070.stdout: "root" 2023-10-18T12:40:15.396 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.397 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "mode for remote execution of cephadm", 2023-10-18T12:40:15.397 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.397 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.398 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.398 INFO:teuthology.orchestra.run.smithi070.stdout: "prometheus_alerts_path": { 2023-10-18T12:40:15.398 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prometheus_alerts_path", 2023-10-18T12:40:15.399 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.399 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.399 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.400 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-18T12:40:15.400 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.400 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.400 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.401 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-18T12:40:15.401 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.401 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.402 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.402 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_insecure": { 2023-10-18T12:40:15.402 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_insecure", 2023-10-18T12:40:15.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.403 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.403 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.404 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.404 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.404 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.404 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.405 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-18T12:40:15.405 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.405 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.405 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.406 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_password": { 2023-10-18T12:40:15.406 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_password", 2023-10-18T12:40:15.407 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.407 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.407 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.408 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.408 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.408 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.408 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.409 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-18T12:40:15.409 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.409 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.410 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.410 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_url": { 2023-10-18T12:40:15.410 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_url", 2023-10-18T12:40:15.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.411 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.411 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.412 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.412 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.412 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.412 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.413 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-18T12:40:15.413 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.413 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.414 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.414 INFO:teuthology.orchestra.run.smithi070.stdout: "registry_username": { 2023-10-18T12:40:15.414 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "registry_username", 2023-10-18T12:40:15.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.415 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.415 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.415 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.416 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.416 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.416 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.417 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-18T12:40:15.417 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.417 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.417 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.418 INFO:teuthology.orchestra.run.smithi070.stdout: "secure_monitoring_stack": { 2023-10-18T12:40:15.418 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "secure_monitoring_stack", 2023-10-18T12:40:15.418 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.419 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.419 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.419 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.420 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.420 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.420 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.420 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-18T12:40:15.421 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.421 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.421 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.422 INFO:teuthology.orchestra.run.smithi070.stdout: "service_discovery_port": { 2023-10-18T12:40:15.422 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "service_discovery_port", 2023-10-18T12:40:15.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.423 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.423 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.424 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8765", 2023-10-18T12:40:15.424 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.424 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.425 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.425 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "cephadm service discovery port", 2023-10-18T12:40:15.425 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.425 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.426 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.426 INFO:teuthology.orchestra.run.smithi070.stdout: "ssh_config_file": { 2023-10-18T12:40:15.427 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssh_config_file", 2023-10-18T12:40:15.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.427 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.428 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.428 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.429 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.429 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.429 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-18T12:40:15.430 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.430 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.431 INFO:teuthology.orchestra.run.smithi070.stdout: "use_agent": { 2023-10-18T12:40:15.431 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "use_agent", 2023-10-18T12:40:15.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.432 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.432 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.432 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.433 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.433 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.433 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.433 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-18T12:40:15.434 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.434 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.434 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.435 INFO:teuthology.orchestra.run.smithi070.stdout: "use_repo_digest": { 2023-10-18T12:40:15.435 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "use_repo_digest", 2023-10-18T12:40:15.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.436 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.436 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.437 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.437 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.437 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.438 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-18T12:40:15.438 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.438 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.439 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.439 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_failed_host_check": { 2023-10-18T12:40:15.440 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_failed_host_check", 2023-10-18T12:40:15.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.440 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.441 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.441 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.441 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.442 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.442 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.442 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if the host check fails", 2023-10-18T12:40:15.443 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.443 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.443 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.444 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_stray_daemons": { 2023-10-18T12:40:15.444 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_stray_daemons", 2023-10-18T12:40:15.444 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.445 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.445 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.445 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.445 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.446 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.446 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.446 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-18T12:40:15.447 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.447 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.447 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.448 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_on_stray_hosts": { 2023-10-18T12:40:15.448 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_on_stray_hosts", 2023-10-18T12:40:15.448 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.449 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.449 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.449 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.449 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.450 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.450 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.450 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-18T12:40:15.451 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.451 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.451 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.453 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crash", 2023-10-18T12:40:15.453 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.453 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.454 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.454 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.454 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.455 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.455 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.455 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.456 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.456 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.456 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.457 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.457 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.457 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.458 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.458 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.458 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.459 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.459 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.459 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.459 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.460 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.460 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.461 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.461 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.462 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.462 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.462 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.463 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.463 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.463 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.464 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.464 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.464 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.465 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.465 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.466 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.466 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.467 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.467 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.467 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.468 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.468 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.468 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.468 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.469 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.469 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.469 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.470 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.470 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.470 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.471 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.471 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.471 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.472 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.472 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.472 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.473 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.473 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.473 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.474 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.474 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.474 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.474 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.475 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.475 INFO:teuthology.orchestra.run.smithi070.stdout: "retain_interval": { 2023-10-18T12:40:15.476 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "retain_interval", 2023-10-18T12:40:15.476 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.476 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.477 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.477 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "31536000", 2023-10-18T12:40:15.477 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.478 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.478 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.478 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-18T12:40:15.481 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.481 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.482 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.482 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_recent_interval": { 2023-10-18T12:40:15.483 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_recent_interval", 2023-10-18T12:40:15.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.483 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.484 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.484 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1209600", 2023-10-18T12:40:15.484 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.484 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.485 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.485 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-18T12:40:15.485 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.486 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.486 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.487 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "dashboard", 2023-10-18T12:40:15.488 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.488 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.488 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.489 INFO:teuthology.orchestra.run.smithi070.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-18T12:40:15.489 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-18T12:40:15.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.490 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.490 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.491 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.491 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.491 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.491 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.492 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.492 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.492 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.493 INFO:teuthology.orchestra.run.smithi070.stdout: "ALERTMANAGER_API_HOST": { 2023-10-18T12:40:15.493 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-18T12:40:15.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.494 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.494 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.494 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.495 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.495 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.495 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.496 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.496 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.496 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.496 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.497 INFO:teuthology.orchestra.run.smithi070.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-18T12:40:15.497 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-18T12:40:15.498 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.498 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.498 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.499 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.499 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.499 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.500 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.500 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.500 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.500 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.501 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.501 INFO:teuthology.orchestra.run.smithi070.stdout: "AUDIT_API_ENABLED": { 2023-10-18T12:40:15.501 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "AUDIT_API_ENABLED", 2023-10-18T12:40:15.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.502 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.503 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.503 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.503 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.504 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.504 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.504 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.506 INFO:teuthology.orchestra.run.smithi070.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-18T12:40:15.506 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-18T12:40:15.506 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.507 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.507 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.507 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.507 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.508 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.508 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.508 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.509 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.509 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.509 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.510 INFO:teuthology.orchestra.run.smithi070.stdout: "ENABLE_BROWSABLE_API": { 2023-10-18T12:40:15.510 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-18T12:40:15.510 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.511 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.511 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.511 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.512 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.512 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.512 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.512 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.513 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.513 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.513 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.514 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-18T12:40:15.514 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-18T12:40:15.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.515 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.515 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.515 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.516 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.516 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.516 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.517 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.517 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.517 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.517 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.518 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-18T12:40:15.518 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-18T12:40:15.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.519 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.519 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.519 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-18T12:40:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-18T12:40:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-18T12:40:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-18T12:40:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.530 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.530 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.530 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-18T12:40:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-18T12:40:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-18T12:40:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-18T12:40:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-18T12:40:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-18T12:40:15.540 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.540 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.540 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-18T12:40:15.544 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-18T12:40:15.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.544 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.545 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.545 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.545 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.546 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.546 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.546 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.547 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.547 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.547 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.548 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_PASSWORD": { 2023-10-18T12:40:15.548 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-18T12:40:15.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.549 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.549 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.549 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:15.549 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.550 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.550 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.550 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.551 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.551 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.551 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.552 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-18T12:40:15.552 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-18T12:40:15.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.553 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.553 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.553 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.554 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.554 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.554 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_URL": { 2023-10-18T12:40:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_URL", 2023-10-18T12:40:15.557 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.557 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.557 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.559 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.559 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.559 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.560 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.560 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.560 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_API_USERNAME": { 2023-10-18T12:40:15.561 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-18T12:40:15.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.561 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.562 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.562 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:15.562 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.562 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.563 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.563 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.563 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.564 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.564 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.565 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-18T12:40:15.565 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-18T12:40:15.565 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.565 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.566 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.566 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.566 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.567 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.567 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.567 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.568 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.568 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.568 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.569 INFO:teuthology.orchestra.run.smithi070.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-18T12:40:15.569 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-18T12:40:15.569 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.570 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.570 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.570 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.570 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.571 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.571 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.571 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.572 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.572 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.572 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.573 INFO:teuthology.orchestra.run.smithi070.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-18T12:40:15.573 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-18T12:40:15.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.574 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.574 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.574 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.575 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.575 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.575 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.575 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.576 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.576 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.576 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.577 INFO:teuthology.orchestra.run.smithi070.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-18T12:40:15.577 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-18T12:40:15.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.578 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.578 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.578 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.579 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.579 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.579 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.580 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.580 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.580 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.580 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.581 INFO:teuthology.orchestra.run.smithi070.stdout: "PROMETHEUS_API_HOST": { 2023-10-18T12:40:15.581 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-18T12:40:15.582 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.582 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.582 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.583 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.583 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.583 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.583 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.584 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.584 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.584 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.585 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.585 INFO:teuthology.orchestra.run.smithi070.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-18T12:40:15.585 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-18T12:40:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-18T12:40:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-18T12:40:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-18T12:40:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-18T12:40:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.597 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.597 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.597 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.598 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-18T12:40:15.598 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-18T12:40:15.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.599 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.599 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.599 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.600 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.600 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.600 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.601 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.601 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.601 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.602 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.602 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-18T12:40:15.603 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-18T12:40:15.603 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.603 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.603 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.604 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.604 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.604 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.605 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.605 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.605 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.605 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.606 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.606 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-18T12:40:15.607 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-18T12:40:15.607 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.607 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.608 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.608 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.608 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.608 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.609 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.609 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.609 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.610 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.610 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-18T12:40:15.610 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-18T12:40:15.611 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.611 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.611 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.612 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.612 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.612 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.612 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.613 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.613 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.613 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.614 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.614 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-18T12:40:15.615 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-18T12:40:15.615 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.615 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.615 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.616 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.616 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.616 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.617 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.617 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.617 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.617 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.618 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.618 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_ENABLED": { 2023-10-18T12:40:15.619 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-18T12:40:15.619 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.619 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.620 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.620 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.620 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.620 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.621 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.621 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.621 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.622 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.622 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.622 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-18T12:40:15.623 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-18T12:40:15.623 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.623 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.624 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.624 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-18T12:40:15.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.625 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.625 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.625 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.625 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.626 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.626 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.627 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-18T12:40:15.627 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-18T12:40:15.627 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.628 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.628 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.628 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.628 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.629 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.629 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.629 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.630 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.630 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.630 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.631 INFO:teuthology.orchestra.run.smithi070.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-18T12:40:15.631 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-18T12:40:15.631 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.632 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.632 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.632 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8", 2023-10-18T12:40:15.633 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.633 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.633 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.634 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.634 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.634 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.635 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.635 INFO:teuthology.orchestra.run.smithi070.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-18T12:40:15.636 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-18T12:40:15.636 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.636 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.637 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.637 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "45", 2023-10-18T12:40:15.637 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.637 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.638 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.638 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.638 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.639 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.639 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.640 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_ACCESS_KEY": { 2023-10-18T12:40:15.640 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-18T12:40:15.640 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.640 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.641 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.641 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.641 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.642 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.642 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.642 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.643 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.643 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.643 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.644 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-18T12:40:15.644 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-18T12:40:15.644 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.645 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.645 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.645 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "admin", 2023-10-18T12:40:15.645 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.646 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.646 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.646 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.647 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.647 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.647 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.648 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_SECRET_KEY": { 2023-10-18T12:40:15.648 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-18T12:40:15.649 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.649 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.649 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.649 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.650 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.650 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.650 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.651 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.651 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.651 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.652 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.652 INFO:teuthology.orchestra.run.smithi070.stdout: "RGW_API_SSL_VERIFY": { 2023-10-18T12:40:15.652 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-18T12:40:15.653 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.653 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.653 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.653 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.654 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.654 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.654 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.655 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.655 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.655 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.655 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.656 INFO:teuthology.orchestra.run.smithi070.stdout: "UNSAFE_TLS_v1_2": { 2023-10-18T12:40:15.656 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-18T12:40:15.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.657 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.657 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.658 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.658 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.658 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.658 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.659 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.659 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.659 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.660 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.660 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-18T12:40:15.661 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-18T12:40:15.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.661 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.661 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.662 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:15.662 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.662 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.663 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.663 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.663 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.664 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.664 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.664 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-18T12:40:15.665 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-18T12:40:15.665 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.665 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.666 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.666 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.666 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.667 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.667 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.667 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.667 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.668 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.668 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.669 INFO:teuthology.orchestra.run.smithi070.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-18T12:40:15.669 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-18T12:40:15.669 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.670 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.670 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.670 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:15.670 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.671 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.671 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.671 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.672 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.672 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.672 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.673 INFO:teuthology.orchestra.run.smithi070.stdout: "cross_origin_url": { 2023-10-18T12:40:15.673 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cross_origin_url", 2023-10-18T12:40:15.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.674 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.674 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.674 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.675 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.675 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.675 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.676 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.676 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.676 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.676 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.677 INFO:teuthology.orchestra.run.smithi070.stdout: "crt_file": { 2023-10-18T12:40:15.677 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "crt_file", 2023-10-18T12:40:15.678 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.678 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.678 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.679 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.679 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.679 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.680 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.680 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.680 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.680 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.681 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.681 INFO:teuthology.orchestra.run.smithi070.stdout: "debug": { 2023-10-18T12:40:15.682 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "debug", 2023-10-18T12:40:15.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.682 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.682 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.683 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.683 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.683 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.685 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.685 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Enable/disable debug options", 2023-10-18T12:40:15.686 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.686 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.686 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.687 INFO:teuthology.orchestra.run.smithi070.stdout: "jwt_token_ttl": { 2023-10-18T12:40:15.687 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "jwt_token_ttl", 2023-10-18T12:40:15.687 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.688 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.688 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.688 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "28800", 2023-10-18T12:40:15.689 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.689 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.689 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.690 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.690 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.690 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.690 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.691 INFO:teuthology.orchestra.run.smithi070.stdout: "key_file": { 2023-10-18T12:40:15.691 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "key_file", 2023-10-18T12:40:15.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.692 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.692 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.692 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.693 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.693 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.693 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.694 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.694 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.694 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.695 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.695 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.695 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.696 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.696 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.697 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.697 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.697 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.698 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.698 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.698 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.698 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.699 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.699 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.699 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.700 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.700 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.700 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.701 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.701 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.701 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.702 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.702 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.703 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.703 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.703 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.704 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.704 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.704 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.704 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.705 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.705 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.706 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.706 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.706 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.707 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.707 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.707 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.708 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.708 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.708 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.708 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.709 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.709 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.709 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.710 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.710 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.710 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.711 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.711 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.711 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.712 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.712 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.712 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.713 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.713 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.713 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.713 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.714 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.714 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.714 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.715 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.715 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.715 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.716 INFO:teuthology.orchestra.run.smithi070.stdout: "motd": { 2023-10-18T12:40:15.716 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "motd", 2023-10-18T12:40:15.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.717 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.717 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.717 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.718 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.718 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.718 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.719 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "The message of the day", 2023-10-18T12:40:15.719 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.719 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.719 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.720 INFO:teuthology.orchestra.run.smithi070.stdout: "redirect_resolve_ip_addr": { 2023-10-18T12:40:15.720 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "redirect_resolve_ip_addr", 2023-10-18T12:40:15.721 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.721 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.721 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.721 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.722 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.722 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.723 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.723 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.723 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.724 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.724 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:15.724 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:15.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.725 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.725 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.726 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "::", 2023-10-18T12:40:15.726 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.726 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.727 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.727 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.727 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.727 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.728 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.728 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:15.729 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:15.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.729 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.729 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.730 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8080", 2023-10-18T12:40:15.730 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.730 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.731 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.731 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.731 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.732 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.732 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.732 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl": { 2023-10-18T12:40:15.733 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl", 2023-10-18T12:40:15.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.733 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.734 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.734 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.734 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.734 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.735 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.735 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.735 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.736 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.736 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.737 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl_server_port": { 2023-10-18T12:40:15.737 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl_server_port", 2023-10-18T12:40:15.737 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.737 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.738 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.738 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8443", 2023-10-18T12:40:15.738 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.739 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.739 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.739 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.739 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.740 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.740 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.741 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_behaviour": { 2023-10-18T12:40:15.741 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_behaviour", 2023-10-18T12:40:15.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.742 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.742 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.742 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "redirect", 2023-10-18T12:40:15.742 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.743 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.743 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.743 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.744 INFO:teuthology.orchestra.run.smithi070.stdout: "redirect" 2023-10-18T12:40:15.744 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.744 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.745 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.745 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.745 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.746 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_error_status_code": { 2023-10-18T12:40:15.746 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_error_status_code", 2023-10-18T12:40:15.746 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.747 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.747 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.747 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "500", 2023-10-18T12:40:15.747 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "400", 2023-10-18T12:40:15.748 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "599", 2023-10-18T12:40:15.748 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.748 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.749 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.749 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.749 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.750 INFO:teuthology.orchestra.run.smithi070.stdout: "url_prefix": { 2023-10-18T12:40:15.750 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "url_prefix", 2023-10-18T12:40:15.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.751 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.751 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.751 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.752 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.752 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.752 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.752 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.753 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.753 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.753 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.755 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "devicehealth", 2023-10-18T12:40:15.755 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.755 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.756 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.756 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_monitoring": { 2023-10-18T12:40:15.756 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enable_monitoring", 2023-10-18T12:40:15.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.757 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.757 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.757 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.758 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.758 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.758 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.759 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "monitor device health metrics", 2023-10-18T12:40:15.759 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.759 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.760 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.760 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.760 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.761 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.761 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.761 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.762 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.762 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.762 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.763 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.763 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.763 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.763 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.764 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.764 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.764 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.765 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.765 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.765 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.765 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.766 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.766 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.767 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.767 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.768 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.768 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.768 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.769 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.769 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.769 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.770 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.771 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.771 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.771 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.772 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.772 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.772 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.773 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.773 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.773 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.774 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.774 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.774 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.774 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.775 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.775 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.775 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.776 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.776 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.776 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.777 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.777 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.778 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.778 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.778 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.779 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.779 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.779 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.780 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.780 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.780 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.780 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.781 INFO:teuthology.orchestra.run.smithi070.stdout: "mark_out_threshold": { 2023-10-18T12:40:15.781 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mark_out_threshold", 2023-10-18T12:40:15.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.782 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.782 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.783 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "2419200", 2023-10-18T12:40:15.783 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.783 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.783 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.784 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-18T12:40:15.784 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.784 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.785 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.785 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": { 2023-10-18T12:40:15.785 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pool_name", 2023-10-18T12:40:15.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.786 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.786 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.787 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "device_health_metrics", 2023-10-18T12:40:15.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.787 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.788 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.788 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-18T12:40:15.788 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.788 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.789 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.789 INFO:teuthology.orchestra.run.smithi070.stdout: "retention_period": { 2023-10-18T12:40:15.790 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "retention_period", 2023-10-18T12:40:15.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.790 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.791 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15552000", 2023-10-18T12:40:15.791 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.791 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.792 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.792 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long to retain device health metrics", 2023-10-18T12:40:15.792 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.793 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.793 INFO:teuthology.orchestra.run.smithi070.stdout: "scrape_frequency": { 2023-10-18T12:40:15.794 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "scrape_frequency", 2023-10-18T12:40:15.794 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.794 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.795 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.795 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "86400", 2023-10-18T12:40:15.795 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.796 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.796 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.796 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-18T12:40:15.796 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.797 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.797 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.798 INFO:teuthology.orchestra.run.smithi070.stdout: "self_heal": { 2023-10-18T12:40:15.798 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "self_heal", 2023-10-18T12:40:15.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.799 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.799 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.799 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:15.799 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.800 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.800 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.800 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-18T12:40:15.801 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.801 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.801 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.802 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:15.802 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:15.802 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.803 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.803 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.803 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:15.804 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.804 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.804 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.804 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how frequently to wake up and check device health", 2023-10-18T12:40:15.805 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.805 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.805 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.806 INFO:teuthology.orchestra.run.smithi070.stdout: "warn_threshold": { 2023-10-18T12:40:15.806 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "warn_threshold", 2023-10-18T12:40:15.807 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.807 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.807 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.808 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "7257600", 2023-10-18T12:40:15.808 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.808 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.808 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.809 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-18T12:40:15.809 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.809 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.810 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.811 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "diskprediction_local", 2023-10-18T12:40:15.811 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.812 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.812 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.812 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.813 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.813 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.813 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.813 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.814 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.814 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.814 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.815 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.815 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.815 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.816 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.816 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.816 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.817 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.817 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.817 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.818 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.818 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.819 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.819 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.819 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.820 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.820 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.821 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.821 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.821 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.821 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.822 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.822 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.823 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.823 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.824 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.824 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.824 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.824 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.825 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.825 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.825 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.826 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.826 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.826 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.827 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.827 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.827 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.827 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.828 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.828 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.828 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.829 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.829 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.830 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.830 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.830 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.831 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.831 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.831 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.832 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.832 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.832 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.832 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.833 INFO:teuthology.orchestra.run.smithi070.stdout: "predict_interval": { 2023-10-18T12:40:15.833 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "predict_interval", 2023-10-18T12:40:15.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.834 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.834 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.834 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "86400", 2023-10-18T12:40:15.835 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.835 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.835 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.836 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.836 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.836 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.837 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.837 INFO:teuthology.orchestra.run.smithi070.stdout: "predictor_model": { 2023-10-18T12:40:15.837 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "predictor_model", 2023-10-18T12:40:15.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.838 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.838 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.839 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "prophetstor", 2023-10-18T12:40:15.839 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.839 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.839 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.840 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.840 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.840 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.841 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.841 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:15.842 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:15.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.842 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.843 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.843 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "600", 2023-10-18T12:40:15.843 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.843 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.844 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.844 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.844 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.845 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.845 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.846 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.846 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "influx", 2023-10-18T12:40:15.847 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": false, 2023-10-18T12:40:15.847 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "influxdb python module not found", 2023-10-18T12:40:15.847 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.848 INFO:teuthology.orchestra.run.smithi070.stdout: "batch_size": { 2023-10-18T12:40:15.848 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "batch_size", 2023-10-18T12:40:15.848 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.848 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.849 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.849 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5000", 2023-10-18T12:40:15.849 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.850 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.850 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.850 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-18T12:40:15.851 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.851 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.851 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.852 INFO:teuthology.orchestra.run.smithi070.stdout: "database": { 2023-10-18T12:40:15.852 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "database", 2023-10-18T12:40:15.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.853 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.853 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.853 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "ceph", 2023-10-18T12:40:15.853 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.854 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.854 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.854 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-18T12:40:15.855 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.855 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.855 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.856 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": { 2023-10-18T12:40:15.856 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "hostname", 2023-10-18T12:40:15.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.857 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.857 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.857 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.858 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.858 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.858 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.859 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB server hostname", 2023-10-18T12:40:15.859 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.859 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.859 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.860 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:15.860 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:15.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:15.861 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.861 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.861 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "30", 2023-10-18T12:40:15.862 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "5", 2023-10-18T12:40:15.862 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.862 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.863 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-18T12:40:15.863 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.863 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.864 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.864 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.864 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.865 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.866 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.866 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.866 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.867 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.867 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.867 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.867 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.868 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.868 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.868 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.869 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.869 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.869 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.869 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.870 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.870 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.871 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.871 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.871 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.872 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.872 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.872 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.872 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.873 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.873 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.873 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.874 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.874 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.874 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.875 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.875 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.875 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.876 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.876 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.876 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.876 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.877 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.877 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.877 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.878 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.878 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.878 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.879 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.879 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.879 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.879 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.880 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.880 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.880 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.881 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.881 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.882 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.882 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.882 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.883 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.883 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.883 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.883 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.884 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.884 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.885 INFO:teuthology.orchestra.run.smithi070.stdout: "password": { 2023-10-18T12:40:15.885 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "password", 2023-10-18T12:40:15.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.886 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.886 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.887 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.887 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.887 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.887 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.888 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "password of InfluxDB server user", 2023-10-18T12:40:15.888 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.888 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.889 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.889 INFO:teuthology.orchestra.run.smithi070.stdout: "port": { 2023-10-18T12:40:15.890 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "port", 2023-10-18T12:40:15.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.890 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.891 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.891 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "8086", 2023-10-18T12:40:15.891 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.892 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.892 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.892 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "InfluxDB server port", 2023-10-18T12:40:15.893 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.893 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.893 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ssl": { 2023-10-18T12:40:15.894 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ssl", 2023-10-18T12:40:15.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.895 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.895 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.895 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "false", 2023-10-18T12:40:15.896 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.896 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.896 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.897 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T12:40:15.897 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.897 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.897 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.898 INFO:teuthology.orchestra.run.smithi070.stdout: "threads": { 2023-10-18T12:40:15.898 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "threads", 2023-10-18T12:40:15.899 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.899 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.899 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.900 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:15.900 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:15.900 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "32", 2023-10-18T12:40:15.900 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.901 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-18T12:40:15.901 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.901 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.902 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.902 INFO:teuthology.orchestra.run.smithi070.stdout: "username": { 2023-10-18T12:40:15.902 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "username", 2023-10-18T12:40:15.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.903 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.903 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.904 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.904 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.904 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.905 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.905 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "username of InfluxDB server user", 2023-10-18T12:40:15.905 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.905 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.906 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.906 INFO:teuthology.orchestra.run.smithi070.stdout: "verify_ssl": { 2023-10-18T12:40:15.907 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "verify_ssl", 2023-10-18T12:40:15.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.907 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.908 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.908 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "true", 2023-10-18T12:40:15.908 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.908 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.909 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.909 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T12:40:15.909 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.910 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.910 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.910 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.911 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.911 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "insights", 2023-10-18T12:40:15.912 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.912 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.912 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.913 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.913 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.913 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.913 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.914 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.914 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.914 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.915 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.915 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.915 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.916 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.916 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.916 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.917 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.917 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.917 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.917 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.918 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.918 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.918 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.919 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.919 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.920 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.920 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.920 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.920 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.921 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.921 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.921 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.922 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.922 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.922 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.923 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.923 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.923 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.924 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.924 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.924 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.924 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.925 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.925 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.925 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.926 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.926 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.926 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.927 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.927 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.927 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.927 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.928 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.928 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.928 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.929 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.929 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.930 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.930 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.930 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.930 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.931 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.931 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.931 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.932 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.932 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.932 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.932 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.933 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.934 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "iostat", 2023-10-18T12:40:15.934 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.935 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.935 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.935 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.935 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.936 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.936 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.936 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.937 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.937 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.937 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.937 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.938 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.938 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.938 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.939 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.939 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.939 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.939 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.940 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.940 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.940 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.940 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.941 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.941 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.942 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.942 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.942 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.943 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.943 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.943 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.944 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.944 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.944 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.944 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.945 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.945 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.945 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.946 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.946 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.947 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.947 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.947 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.947 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.948 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.948 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.948 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.949 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.949 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.949 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.949 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.950 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.950 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.950 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.951 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.952 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.952 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.953 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.953 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.953 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.954 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.954 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.954 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.955 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.955 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.955 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.956 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.957 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "k8sevents", 2023-10-18T12:40:15.958 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.958 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.958 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.958 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_event_retention_days": { 2023-10-18T12:40:15.959 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "ceph_event_retention_days", 2023-10-18T12:40:15.959 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.959 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.960 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.960 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "7", 2023-10-18T12:40:15.960 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.960 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.961 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.961 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-18T12:40:15.961 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.962 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.962 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.963 INFO:teuthology.orchestra.run.smithi070.stdout: "config_check_secs": { 2023-10-18T12:40:15.963 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "config_check_secs", 2023-10-18T12:40:15.963 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:15.963 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.964 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:15.964 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:15.964 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "10", 2023-10-18T12:40:15.965 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.965 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.965 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-18T12:40:15.966 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.966 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.966 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.967 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.967 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.967 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.968 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.968 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.968 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.969 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.969 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.969 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.970 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.970 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.970 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.970 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.971 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.971 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.971 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.972 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.972 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.972 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.972 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.973 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.973 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.974 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.974 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.974 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.975 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.975 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.975 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.975 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.976 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.976 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.976 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.976 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.977 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:15.977 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:15.978 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.978 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.978 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.979 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:15.979 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.979 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.979 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.980 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.980 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.980 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.980 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.981 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.981 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.981 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.982 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.982 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.982 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.983 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.983 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:15.983 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:15.984 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:15.984 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.984 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.984 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:15.985 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.985 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.985 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.986 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.986 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.986 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.986 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.987 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.987 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:15.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:15.988 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "localpool", 2023-10-18T12:40:15.988 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:15.988 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:15.989 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:15.989 INFO:teuthology.orchestra.run.smithi070.stdout: "failure_domain": { 2023-10-18T12:40:15.989 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "failure_domain", 2023-10-18T12:40:15.989 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "host", 2023-10-18T12:40:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "failure domain for any created local pool", 2023-10-18T12:40:15.992 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-18T12:40:15.992 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.992 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.993 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:15.993 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:15.993 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:15.993 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:15.994 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:15.994 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:15.994 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:15.995 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:15.995 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:15.995 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:15.995 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:15.996 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:15.996 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:15.996 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:15.997 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:15.997 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:15.997 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:15.997 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:15.998 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:15.998 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:15.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:15.999 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:15.999 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:15.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.000 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.000 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.000 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.000 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.001 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.001 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.001 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.001 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.002 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.002 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.003 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.003 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.003 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.003 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.004 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.004 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.004 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.004 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.005 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.005 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.005 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.005 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.006 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.006 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.006 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.007 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.007 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.007 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.007 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.008 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.008 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.008 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.009 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.009 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.009 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.010 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.010 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.010 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.010 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.011 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.011 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.011 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.012 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.012 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": { 2023-10-18T12:40:16.012 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "min_size", 2023-10-18T12:40:16.013 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.013 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.013 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.013 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.014 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.014 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.014 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.015 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default min_size for any created local pool", 2023-10-18T12:40:16.015 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-18T12:40:16.015 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.015 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.016 INFO:teuthology.orchestra.run.smithi070.stdout: "num_rep": { 2023-10-18T12:40:16.016 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "num_rep", 2023-10-18T12:40:16.016 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.017 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.017 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.017 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3", 2023-10-18T12:40:16.018 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.018 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.018 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.019 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default replica count for any created local pool", 2023-10-18T12:40:16.019 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.019 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.019 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": { 2023-10-18T12:40:16.020 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pg_num", 2023-10-18T12:40:16.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.021 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.021 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.022 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "128", 2023-10-18T12:40:16.022 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.022 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.022 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.023 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "default pg_num for any created local pool", 2023-10-18T12:40:16.023 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.023 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.024 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.024 INFO:teuthology.orchestra.run.smithi070.stdout: "prefix": { 2023-10-18T12:40:16.025 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prefix", 2023-10-18T12:40:16.025 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.025 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.025 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.026 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.026 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.026 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.027 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.027 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "name prefix for any created local pool", 2023-10-18T12:40:16.027 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.028 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.028 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.029 INFO:teuthology.orchestra.run.smithi070.stdout: "subtree": { 2023-10-18T12:40:16.029 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "subtree", 2023-10-18T12:40:16.029 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.029 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.030 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.030 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "rack", 2023-10-18T12:40:16.030 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.031 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.031 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.031 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-18T12:40:16.031 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-18T12:40:16.032 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.032 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mds_autoscaler", 2023-10-18T12:40:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.037 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.037 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.037 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.038 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.038 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.038 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.039 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.039 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.039 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.039 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.040 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.040 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.040 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.041 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.041 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.042 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.042 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.042 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.043 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.043 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.043 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.044 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.044 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.044 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.045 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.045 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.046 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.046 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.046 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.047 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.047 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.047 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.047 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.048 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.048 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.048 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.049 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.049 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.049 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.050 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.050 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.050 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.051 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.051 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.051 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.052 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.052 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.052 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.053 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.053 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.053 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.054 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.054 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.054 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.055 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.055 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.055 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.055 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.057 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mirroring", 2023-10-18T12:40:16.057 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.057 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.058 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.058 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.058 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.059 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.059 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.060 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.060 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.060 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.060 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.061 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.061 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.061 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.062 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.062 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.062 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.063 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.063 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.063 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.063 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.064 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.064 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.065 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.065 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.065 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.065 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.066 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.066 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.066 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.067 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.067 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.067 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.068 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.068 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.069 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.069 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.069 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.070 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.070 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.070 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.071 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.071 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.071 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.072 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.072 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.072 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.072 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.073 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.073 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.073 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.074 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.074 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.074 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.075 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.075 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.075 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.076 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.076 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.076 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.076 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.077 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.077 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.077 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.078 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.078 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.078 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.080 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "nfs", 2023-10-18T12:40:16.080 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.080 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.081 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.081 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.081 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.081 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.082 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.082 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.082 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.083 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.083 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.083 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.084 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.084 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.084 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.084 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.085 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.085 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.085 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.086 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.086 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.086 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.087 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.087 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.087 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.088 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.088 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.089 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.089 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.089 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.089 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.090 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.090 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.090 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.091 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.091 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.092 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.092 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.092 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.092 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.093 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.093 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.093 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.094 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.094 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.094 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.095 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.095 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.095 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.096 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.096 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.096 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.096 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.097 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.097 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.098 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.098 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.099 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.099 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.099 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.100 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.100 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.100 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.100 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.101 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.101 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.101 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.103 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "orchestrator", 2023-10-18T12:40:16.103 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.103 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.104 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.104 INFO:teuthology.orchestra.run.smithi070.stdout: "fail_fs": { 2023-10-18T12:40:16.104 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "fail_fs", 2023-10-18T12:40:16.105 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.105 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.105 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.106 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.106 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.106 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.106 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.107 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-18T12:40:16.107 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.107 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.108 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.108 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.108 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.109 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.109 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.110 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.110 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.110 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.111 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.111 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.111 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.111 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.112 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.112 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.112 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.113 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.113 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.113 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.113 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.114 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.114 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.115 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.115 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.115 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.116 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.116 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.116 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.117 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.117 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.117 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.117 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.118 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.118 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.119 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.119 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.119 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.120 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.120 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.120 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.120 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.121 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.121 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.121 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.122 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.122 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.122 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.123 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.123 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.123 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.123 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.124 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.124 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.124 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.125 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.125 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.125 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.126 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.126 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.126 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.127 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.127 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.127 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.127 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.128 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.128 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.128 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.129 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator": { 2023-10-18T12:40:16.129 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "orchestrator", 2023-10-18T12:40:16.129 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.130 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.130 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.130 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.131 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.131 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.131 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.131 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm", 2023-10-18T12:40:16.132 INFO:teuthology.orchestra.run.smithi070.stdout: "rook", 2023-10-18T12:40:16.132 INFO:teuthology.orchestra.run.smithi070.stdout: "test_orchestrator" 2023-10-18T12:40:16.132 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.132 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Orchestrator backend", 2023-10-18T12:40:16.133 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.133 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.133 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.135 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "osd_perf_query", 2023-10-18T12:40:16.135 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.135 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.136 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.136 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.136 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.137 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.137 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.137 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.137 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.138 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.138 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.138 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.139 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.139 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.139 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.139 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.140 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.140 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.140 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.141 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.141 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.142 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.142 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.143 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.143 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.143 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.143 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.144 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.144 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.144 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.144 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.145 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.145 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.145 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.146 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.146 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.147 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.147 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.147 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.147 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.148 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.148 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.148 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.149 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.149 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.149 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.149 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.150 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.150 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.150 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.151 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.151 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.151 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.152 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.152 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.152 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.153 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.153 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.153 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.153 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.154 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.154 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.154 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.155 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.155 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.155 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.155 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.156 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.157 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "osd_support", 2023-10-18T12:40:16.157 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.158 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.158 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.158 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.159 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.159 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.159 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.160 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.160 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.160 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.160 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.161 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.161 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.161 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.162 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.162 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.162 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.163 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.163 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.163 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.163 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.164 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.164 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.164 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.165 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.165 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.165 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.166 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.166 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.166 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.167 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.167 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.167 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.167 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.168 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.168 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.169 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.169 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.169 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.170 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.170 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.170 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.170 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.171 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.171 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.171 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.172 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.172 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.172 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.173 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.173 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.173 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.174 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.174 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.174 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.175 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.175 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.176 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.176 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.177 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.177 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.177 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.178 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.178 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.178 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.179 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.179 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.179 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.180 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.180 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "pg_autoscaler", 2023-10-18T12:40:16.181 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.181 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.181 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.182 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.182 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.182 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.183 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.183 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.183 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.184 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.184 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.184 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.184 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.185 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.185 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.185 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.186 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.186 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.186 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.187 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.187 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.187 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.188 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.188 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.188 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.189 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.189 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.189 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.190 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.190 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.190 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.191 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.191 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.191 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.192 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.192 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.192 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.193 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.193 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.193 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.194 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.194 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.194 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.195 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.195 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.195 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.196 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.196 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.196 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.196 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.197 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.197 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.197 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.198 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.198 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.199 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.199 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.200 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.200 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.201 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.201 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.201 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.201 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.202 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.202 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.202 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.203 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:16.203 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:16.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:16.204 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.204 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.204 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:16.205 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.205 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.205 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.206 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.206 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.206 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.207 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.207 INFO:teuthology.orchestra.run.smithi070.stdout: "threshold": { 2023-10-18T12:40:16.207 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "threshold", 2023-10-18T12:40:16.208 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:16.208 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.208 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.209 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "3.0", 2023-10-18T12:40:16.209 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1.0", 2023-10-18T12:40:16.209 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.209 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.210 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "scaling threshold", 2023-10-18T12:40:16.210 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-18T12:40:16.210 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.211 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.212 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "progress", 2023-10-18T12:40:16.212 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.213 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.213 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.213 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_pg_recovery_event": { 2023-10-18T12:40:16.213 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_pg_recovery_event", 2023-10-18T12:40:16.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.214 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.214 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.215 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.215 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.215 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.216 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.216 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-18T12:40:16.216 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.216 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.217 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.217 INFO:teuthology.orchestra.run.smithi070.stdout: "enabled": { 2023-10-18T12:40:16.218 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enabled", 2023-10-18T12:40:16.218 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.218 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.219 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.219 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.219 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.219 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.220 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.220 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.220 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.221 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.221 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.221 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.222 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.222 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.222 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.223 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.223 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.223 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.223 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.224 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.224 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.224 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.225 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.225 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.225 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.226 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.226 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.226 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.227 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.227 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.227 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.228 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.228 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.229 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.229 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.229 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.229 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.234 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.234 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.234 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.235 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.235 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.235 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.236 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.236 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.236 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.237 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.237 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.237 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.238 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.238 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.239 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.239 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.240 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.240 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.240 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.241 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.241 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.241 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.241 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.242 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.242 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.242 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.243 INFO:teuthology.orchestra.run.smithi070.stdout: "max_completed_events": { 2023-10-18T12:40:16.243 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_completed_events", 2023-10-18T12:40:16.244 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.244 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.244 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.244 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "50", 2023-10-18T12:40:16.245 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.245 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.245 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.246 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "number of past completed events to remember", 2023-10-18T12:40:16.246 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.246 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.247 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.247 INFO:teuthology.orchestra.run.smithi070.stdout: "sleep_interval": { 2023-10-18T12:40:16.247 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "sleep_interval", 2023-10-18T12:40:16.248 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:16.248 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.248 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.249 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "5", 2023-10-18T12:40:16.249 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.249 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.250 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.250 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "how long the module is going to sleep", 2023-10-18T12:40:16.250 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.250 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.251 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.252 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "prometheus", 2023-10-18T12:40:16.253 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.253 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.253 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.254 INFO:teuthology.orchestra.run.smithi070.stdout: "cache": { 2023-10-18T12:40:16.254 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cache", 2023-10-18T12:40:16.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.254 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.255 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.255 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.256 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.256 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.256 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.256 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.257 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.257 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.258 INFO:teuthology.orchestra.run.smithi070.stdout: "exclude_perf_counters": { 2023-10-18T12:40:16.258 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "exclude_perf_counters", 2023-10-18T12:40:16.258 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.259 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.259 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.260 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.260 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.260 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.260 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-18T12:40:16.261 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-18T12:40:16.261 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.261 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.262 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.262 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.263 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.263 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.263 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.263 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.264 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.264 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.264 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.265 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.265 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.265 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.266 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.266 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.266 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.267 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.267 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.267 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.267 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.268 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.268 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.269 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.269 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.269 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.270 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.270 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.270 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.270 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.271 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.271 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.271 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.272 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.272 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.273 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.273 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.273 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.274 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.274 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.274 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.275 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.275 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.275 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.275 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.276 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.276 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.276 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.277 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.277 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.277 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.278 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.278 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.278 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.279 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.279 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.280 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.280 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.280 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.281 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.281 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.281 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.282 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.282 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.282 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.283 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.283 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_stats_pools": { 2023-10-18T12:40:16.284 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_stats_pools", 2023-10-18T12:40:16.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.284 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.284 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.285 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.285 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.285 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.286 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.286 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.286 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.287 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.287 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.287 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-18T12:40:16.288 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-18T12:40:16.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.288 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.289 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.289 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "300", 2023-10-18T12:40:16.289 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.290 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.290 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.290 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "scrape_interval": { 2023-10-18T12:40:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "scrape_interval", 2023-10-18T12:40:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:16.293 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.293 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.293 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15.0", 2023-10-18T12:40:16.294 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.294 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.294 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.295 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.295 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.295 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.295 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.296 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:16.296 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:16.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.297 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.297 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.297 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "::", 2023-10-18T12:40:16.298 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.298 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.298 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.299 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-18T12:40:16.299 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.299 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.299 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.300 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:16.300 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:16.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.301 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.301 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.302 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "9283", 2023-10-18T12:40:16.302 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.302 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.302 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.303 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-18T12:40:16.303 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.305 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.305 INFO:teuthology.orchestra.run.smithi070.stdout: "stale_cache_strategy": { 2023-10-18T12:40:16.306 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "stale_cache_strategy", 2023-10-18T12:40:16.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.306 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.307 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.307 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "log", 2023-10-18T12:40:16.307 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.307 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.308 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.308 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.308 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.309 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.309 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.309 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_behaviour": { 2023-10-18T12:40:16.310 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_behaviour", 2023-10-18T12:40:16.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.310 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.311 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.311 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "default", 2023-10-18T12:40:16.311 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.312 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.312 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.312 INFO:teuthology.orchestra.run.smithi070.stdout: "default", 2023-10-18T12:40:16.313 INFO:teuthology.orchestra.run.smithi070.stdout: "error" 2023-10-18T12:40:16.313 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.313 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.313 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.314 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.314 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.315 INFO:teuthology.orchestra.run.smithi070.stdout: "standby_error_status_code": { 2023-10-18T12:40:16.315 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "standby_error_status_code", 2023-10-18T12:40:16.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.315 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.316 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.316 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "500", 2023-10-18T12:40:16.316 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "400", 2023-10-18T12:40:16.317 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "599", 2023-10-18T12:40:16.317 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.317 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.318 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.318 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.320 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_support", 2023-10-18T12:40:16.320 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.320 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.321 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.321 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.321 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.322 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.322 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.322 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.323 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.323 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.323 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.324 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.324 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.324 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.325 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.325 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.325 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.325 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.326 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.326 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.326 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.327 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.327 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.328 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.328 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.328 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.329 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.329 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.329 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.330 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.330 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.330 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.331 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.331 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.331 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.334 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.334 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.334 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.335 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.335 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.335 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.336 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.336 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.336 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.337 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.337 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.337 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.338 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.338 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.338 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.339 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.339 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.339 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.340 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.340 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.340 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.340 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.341 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.341 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.342 INFO:teuthology.orchestra.run.smithi070.stdout: "max_concurrent_snap_create": { 2023-10-18T12:40:16.342 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_concurrent_snap_create", 2023-10-18T12:40:16.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.343 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.343 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.343 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10", 2023-10-18T12:40:16.344 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.344 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.344 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.345 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.345 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.345 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.346 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.346 INFO:teuthology.orchestra.run.smithi070.stdout: "mirror_snapshot_schedule": { 2023-10-18T12:40:16.347 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mirror_snapshot_schedule", 2023-10-18T12:40:16.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.347 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.347 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.348 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.348 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.348 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.349 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.349 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.349 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.349 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.350 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.350 INFO:teuthology.orchestra.run.smithi070.stdout: "trash_purge_schedule": { 2023-10-18T12:40:16.351 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "trash_purge_schedule", 2023-10-18T12:40:16.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.351 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.351 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.352 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.352 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.352 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.353 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.353 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.353 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.354 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.354 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.355 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "restful", 2023-10-18T12:40:16.356 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.356 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.356 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.357 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_auth": { 2023-10-18T12:40:16.357 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enable_auth", 2023-10-18T12:40:16.357 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.357 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.358 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.358 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.358 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.359 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.359 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.359 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.360 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.360 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.360 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.361 INFO:teuthology.orchestra.run.smithi070.stdout: "key_file": { 2023-10-18T12:40:16.361 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "key_file", 2023-10-18T12:40:16.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.362 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.362 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.362 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.363 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.363 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.363 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.363 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.364 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.364 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.364 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.365 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.365 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.366 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.366 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.366 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.367 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.367 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.367 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.368 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.368 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.368 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.368 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.369 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.369 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.369 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.370 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.370 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.370 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.370 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.371 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.371 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.372 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.372 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.373 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.373 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.373 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.373 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.374 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.374 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.374 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.375 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.375 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.376 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.376 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.376 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.377 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.377 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.377 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.378 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.378 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.378 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.379 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.379 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.379 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.379 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.380 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.380 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.380 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.381 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.381 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.381 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.382 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.382 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.382 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.383 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.383 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.383 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.384 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.384 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.384 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.384 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.385 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.385 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.385 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.386 INFO:teuthology.orchestra.run.smithi070.stdout: "server_addr": { 2023-10-18T12:40:16.386 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_addr", 2023-10-18T12:40:16.387 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.387 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.387 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.387 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.388 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.388 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.388 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.389 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.389 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.389 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.390 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.390 INFO:teuthology.orchestra.run.smithi070.stdout: "server_port": { 2023-10-18T12:40:16.390 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "server_port", 2023-10-18T12:40:16.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.391 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.391 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.392 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.392 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.392 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.392 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.393 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.393 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.393 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.394 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.395 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rgw", 2023-10-18T12:40:16.396 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.396 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.396 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.396 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.397 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.397 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.397 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.398 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.398 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.398 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.399 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.399 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.399 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.399 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.400 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.400 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.400 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.401 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.401 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.401 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.402 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.402 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.402 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.403 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.403 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.404 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.404 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.404 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.405 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.405 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.405 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.406 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.406 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.406 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.407 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.407 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.407 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.408 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.408 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.408 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.409 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.409 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.409 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.410 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.410 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.410 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.411 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.411 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.411 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.412 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.412 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.412 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.413 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.413 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.413 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.414 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.414 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.414 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.415 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.415 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.415 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.416 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.416 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.416 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.417 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.417 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.417 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.418 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.419 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rook", 2023-10-18T12:40:16.419 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.420 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.420 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.420 INFO:teuthology.orchestra.run.smithi070.stdout: "drive_group_interval": { 2023-10-18T12:40:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "drive_group_interval", 2023-10-18T12:40:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "300.0", 2023-10-18T12:40:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-18T12:40:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.424 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.424 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.424 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.425 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.425 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.426 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.426 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.426 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.427 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.427 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.427 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.428 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.428 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.428 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.428 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.429 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.429 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.429 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.430 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.430 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.431 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.431 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.432 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.432 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.432 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.433 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.433 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.433 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.433 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.434 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.434 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.434 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.436 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.436 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.437 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.437 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.437 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.438 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.438 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.438 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.439 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.439 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.439 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.439 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.440 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.440 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.440 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.441 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.441 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.442 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.442 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.442 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.443 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.443 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.443 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.444 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.444 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.444 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.444 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.445 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.445 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.446 INFO:teuthology.orchestra.run.smithi070.stdout: "storage_class": { 2023-10-18T12:40:16.446 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "storage_class", 2023-10-18T12:40:16.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.446 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.447 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.447 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "local", 2023-10-18T12:40:16.447 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.448 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.448 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.448 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-18T12:40:16.449 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.449 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.449 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.449 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.451 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "selftest", 2023-10-18T12:40:16.451 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.451 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.451 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.452 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.452 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.452 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.453 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.453 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.453 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.455 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.455 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.455 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.456 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.456 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.456 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.458 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.458 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.464 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.464 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.464 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.464 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.465 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.465 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.465 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.466 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.466 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.466 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.467 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.467 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.467 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.467 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.468 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.468 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.469 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.469 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.469 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.470 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.470 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.470 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.470 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.471 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.471 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.471 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.472 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.472 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.472 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.473 INFO:teuthology.orchestra.run.smithi070.stdout: "roption1": { 2023-10-18T12:40:16.473 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "roption1", 2023-10-18T12:40:16.473 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.474 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.474 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.474 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.475 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.475 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.475 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.475 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.476 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.476 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.476 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.477 INFO:teuthology.orchestra.run.smithi070.stdout: "roption2": { 2023-10-18T12:40:16.477 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "roption2", 2023-10-18T12:40:16.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.478 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.478 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.478 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "xyz", 2023-10-18T12:40:16.479 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.479 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.479 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.480 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.480 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.480 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.481 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.481 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption1": { 2023-10-18T12:40:16.481 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption1", 2023-10-18T12:40:16.482 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.482 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.482 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.483 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.483 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.483 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.483 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.484 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.484 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.485 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.485 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption2": { 2023-10-18T12:40:16.486 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption2", 2023-10-18T12:40:16.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.486 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.486 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.487 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.487 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.487 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.488 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.488 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.488 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.489 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.489 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.489 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption3": { 2023-10-18T12:40:16.490 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption3", 2023-10-18T12:40:16.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "float", 2023-10-18T12:40:16.490 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.491 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.491 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.491 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.492 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.492 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.492 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.493 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.493 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.493 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.494 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption4": { 2023-10-18T12:40:16.494 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption4", 2023-10-18T12:40:16.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.495 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.495 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.495 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.496 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.496 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.496 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.496 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.497 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.497 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.497 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.498 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption5": { 2023-10-18T12:40:16.498 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption5", 2023-10-18T12:40:16.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.499 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.499 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.499 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.500 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.500 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.500 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.501 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.501 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.501 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.502 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.502 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption6": { 2023-10-18T12:40:16.502 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption6", 2023-10-18T12:40:16.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.503 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.503 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.504 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.504 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.504 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.505 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.505 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.505 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.506 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.506 INFO:teuthology.orchestra.run.smithi070.stdout: "rwoption7": { 2023-10-18T12:40:16.507 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rwoption7", 2023-10-18T12:40:16.507 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.507 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.508 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.508 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.508 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "1", 2023-10-18T12:40:16.508 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "42", 2023-10-18T12:40:16.509 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.509 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.509 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.510 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.510 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.514 INFO:teuthology.orchestra.run.smithi070.stdout: "testkey": { 2023-10-18T12:40:16.514 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testkey", 2023-10-18T12:40:16.514 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.515 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.515 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.515 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.516 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.516 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.516 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.517 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.517 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.517 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.517 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.518 INFO:teuthology.orchestra.run.smithi070.stdout: "testlkey": { 2023-10-18T12:40:16.518 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testlkey", 2023-10-18T12:40:16.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.519 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.519 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.520 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.520 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.520 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.520 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.521 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.521 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.521 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.522 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.522 INFO:teuthology.orchestra.run.smithi070.stdout: "testnewline": { 2023-10-18T12:40:16.523 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "testnewline", 2023-10-18T12:40:16.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.523 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.523 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.524 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.524 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.524 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.525 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.525 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.525 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.526 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.526 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.526 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.526 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.527 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snap_schedule", 2023-10-18T12:40:16.528 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.528 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.528 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.529 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_m_granularity": { 2023-10-18T12:40:16.529 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "allow_m_granularity", 2023-10-18T12:40:16.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.529 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.530 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.530 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.530 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.531 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.531 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.531 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "allow minute scheduled snapshots", 2023-10-18T12:40:16.532 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.532 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.532 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.533 INFO:teuthology.orchestra.run.smithi070.stdout: "dump_on_update": { 2023-10-18T12:40:16.533 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "dump_on_update", 2023-10-18T12:40:16.533 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.534 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.534 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.534 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.535 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.535 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.535 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.536 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "dump database to debug log on update", 2023-10-18T12:40:16.536 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.536 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.536 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.537 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.537 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.538 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.538 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.538 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.539 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.539 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.539 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.539 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.540 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.540 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.540 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.541 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.541 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.541 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.541 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.542 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.542 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.542 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.543 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.543 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.544 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.544 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.545 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.545 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.545 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.545 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.546 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.546 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.546 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.547 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.547 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.548 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.548 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.549 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.549 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.549 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.549 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.550 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.550 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.550 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.551 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.551 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.551 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.552 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.552 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.552 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.553 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.553 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.553 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.553 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.554 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.554 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.555 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.555 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.555 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.556 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.556 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.556 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.556 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.557 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.557 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.557 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.557 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.558 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.558 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.559 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "stats", 2023-10-18T12:40:16.559 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.560 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.560 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.560 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.560 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.561 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.561 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.561 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.562 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.562 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.562 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.563 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.563 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.563 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.563 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.564 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.564 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.564 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.565 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.565 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.565 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.565 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.566 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.566 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.567 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.567 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.568 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.568 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.568 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.568 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.569 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.569 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.569 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.570 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.570 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.571 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.571 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.571 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.571 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.572 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.572 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.572 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.573 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.573 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.573 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.573 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.574 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.574 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.574 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.575 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.575 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.575 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.575 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.576 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.576 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.577 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.577 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.578 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.578 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.578 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.578 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.579 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.579 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.579 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.580 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.580 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.580 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.581 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.581 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "status", 2023-10-18T12:40:16.582 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.582 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.582 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.583 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.583 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.583 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.584 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.584 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.584 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.584 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.585 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.585 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.585 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.586 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.586 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.586 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.587 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.587 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.587 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.587 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.588 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.588 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.588 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.589 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.589 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.589 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.590 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.590 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.590 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.591 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.591 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.591 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.591 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.592 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.592 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.592 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.593 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.593 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.593 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.594 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.594 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.594 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.595 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.595 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.595 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.596 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.596 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.596 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.596 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.597 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.597 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.597 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.598 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.598 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.598 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.599 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.599 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.599 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.600 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.600 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.601 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.601 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.601 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.602 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.602 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.602 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.603 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.603 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.603 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.604 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "telegraf", 2023-10-18T12:40:16.605 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.605 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.605 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.606 INFO:teuthology.orchestra.run.smithi070.stdout: "address": { 2023-10-18T12:40:16.606 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "address", 2023-10-18T12:40:16.606 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.607 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.607 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-18T12:40:16.607 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.608 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.608 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.608 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.609 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.609 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.609 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.610 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:16.610 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:16.610 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:16.611 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.611 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.611 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "15", 2023-10-18T12:40:16.612 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.612 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.612 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.613 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.613 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.613 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.613 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.614 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.614 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.615 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.615 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.615 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.615 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.616 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.616 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.616 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.617 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.617 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.617 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.617 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.618 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.618 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.618 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.619 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.619 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.619 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.620 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.620 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.621 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.621 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.621 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.622 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.622 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.622 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.622 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.623 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.623 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.623 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.624 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.624 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.625 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.625 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.625 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.626 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.626 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.626 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.627 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.627 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.627 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.627 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.628 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.628 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.628 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.629 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.629 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.629 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.629 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.630 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.630 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.631 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.631 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.631 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.632 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.632 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.632 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.632 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.633 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.633 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.633 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.634 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.634 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.634 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.636 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "telemetry", 2023-10-18T12:40:16.636 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.636 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.637 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.637 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_basic": { 2023-10-18T12:40:16.637 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_basic", 2023-10-18T12:40:16.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.638 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.638 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.639 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.639 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.639 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.639 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.640 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-18T12:40:16.640 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.640 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.641 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.641 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_crash": { 2023-10-18T12:40:16.642 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_crash", 2023-10-18T12:40:16.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.642 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.643 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.643 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.643 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.643 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.644 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.644 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-18T12:40:16.644 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.644 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.645 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.645 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_device": { 2023-10-18T12:40:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_device", 2023-10-18T12:40:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.647 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "True", 2023-10-18T12:40:16.647 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.647 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.648 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.648 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-18T12:40:16.648 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.649 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.649 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.649 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_ident": { 2023-10-18T12:40:16.650 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_ident", 2023-10-18T12:40:16.650 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.650 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.651 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.651 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.651 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.652 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.652 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.652 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-18T12:40:16.653 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.653 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.653 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.654 INFO:teuthology.orchestra.run.smithi070.stdout: "channel_perf": { 2023-10-18T12:40:16.654 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "channel_perf", 2023-10-18T12:40:16.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.655 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.655 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.655 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.656 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.656 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.656 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.656 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-18T12:40:16.657 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.657 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.657 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.658 INFO:teuthology.orchestra.run.smithi070.stdout: "contact": { 2023-10-18T12:40:16.658 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "contact", 2023-10-18T12:40:16.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.659 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.659 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.659 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.660 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.660 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.660 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.661 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.661 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.661 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.661 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.662 INFO:teuthology.orchestra.run.smithi070.stdout: "description": { 2023-10-18T12:40:16.662 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "description", 2023-10-18T12:40:16.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.663 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.663 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.663 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.664 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.664 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.664 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.665 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.665 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.665 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.665 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.666 INFO:teuthology.orchestra.run.smithi070.stdout: "device_url": { 2023-10-18T12:40:16.666 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "device_url", 2023-10-18T12:40:16.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.667 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.667 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.668 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-18T12:40:16.668 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.668 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.669 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.669 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.669 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.670 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.670 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.671 INFO:teuthology.orchestra.run.smithi070.stdout: "enabled": { 2023-10-18T12:40:16.671 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "enabled", 2023-10-18T12:40:16.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.671 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.672 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.672 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.672 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.672 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.673 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.673 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.673 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.674 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.674 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.675 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:16.675 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:16.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.676 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.676 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.676 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "24", 2023-10-18T12:40:16.676 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "8", 2023-10-18T12:40:16.677 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.677 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.677 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.678 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.678 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.678 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.679 INFO:teuthology.orchestra.run.smithi070.stdout: "last_opt_revision": { 2023-10-18T12:40:16.679 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "last_opt_revision", 2023-10-18T12:40:16.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.680 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.680 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.680 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "1", 2023-10-18T12:40:16.681 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.681 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.681 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.681 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.682 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.682 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.682 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.683 INFO:teuthology.orchestra.run.smithi070.stdout: "leaderboard": { 2023-10-18T12:40:16.683 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "leaderboard", 2023-10-18T12:40:16.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.684 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.684 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.685 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.685 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.685 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.685 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.686 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.686 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.686 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.687 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.687 INFO:teuthology.orchestra.run.smithi070.stdout: "leaderboard_description": { 2023-10-18T12:40:16.687 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "leaderboard_description", 2023-10-18T12:40:16.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.688 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.688 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.688 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.689 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.689 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.689 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.690 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.690 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.690 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.691 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.691 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.692 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.692 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.692 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.693 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.693 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.693 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.694 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.694 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.694 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.695 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.695 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.695 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.696 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.696 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.696 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.696 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.697 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.697 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.698 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.698 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.699 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.699 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.699 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.699 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.700 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.700 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.700 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.701 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.701 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.701 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.702 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.702 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.703 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.703 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.703 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.704 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.704 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.704 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.705 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.705 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.705 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.706 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.706 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.706 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.707 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.707 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.707 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.708 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.708 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.708 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.709 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.709 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.709 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.710 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.710 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.710 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.711 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.711 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.711 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.712 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.712 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.713 INFO:teuthology.orchestra.run.smithi070.stdout: "organization": { 2023-10-18T12:40:16.713 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "organization", 2023-10-18T12:40:16.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.713 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.714 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.714 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.714 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.717 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.717 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.717 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.718 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.718 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.718 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.719 INFO:teuthology.orchestra.run.smithi070.stdout: "proxy": { 2023-10-18T12:40:16.719 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "proxy", 2023-10-18T12:40:16.720 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.720 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.720 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.721 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.721 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.721 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.721 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.722 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.722 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.722 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.723 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.723 INFO:teuthology.orchestra.run.smithi070.stdout: "url": { 2023-10-18T12:40:16.724 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "url", 2023-10-18T12:40:16.724 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.724 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.724 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.725 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-18T12:40:16.725 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.725 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.726 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.726 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.726 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.727 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.727 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.727 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.727 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.728 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "test_orchestrator", 2023-10-18T12:40:16.729 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.729 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.729 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.730 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.730 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.730 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.731 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.731 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.731 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.731 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.732 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.732 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.732 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.733 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.733 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.733 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.733 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.734 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.734 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.734 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.735 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.735 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.735 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.736 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.736 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.737 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.737 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.737 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.738 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.738 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.738 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.738 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.739 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.739 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.739 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.740 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.740 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.741 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.741 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.741 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.742 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.742 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.742 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.743 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.743 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.743 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.743 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.744 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.744 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.744 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.745 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.745 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.745 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.746 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.746 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.747 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.747 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.747 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.747 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.748 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.748 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.748 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.749 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.749 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.749 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.750 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.750 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.750 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.751 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.752 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "volumes", 2023-10-18T12:40:16.752 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.752 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.753 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.753 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.753 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.753 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.754 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.754 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.754 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.755 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.755 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.755 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.755 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.756 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.756 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.756 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.757 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.757 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.757 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.758 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.758 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.758 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.759 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.759 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.759 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.760 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.760 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.760 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.761 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.761 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.761 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.762 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.762 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.762 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.763 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.763 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.763 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.764 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.764 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.764 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.765 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.765 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.765 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.766 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.766 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.766 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.767 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.767 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.767 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.767 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.768 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.768 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.768 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.769 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.769 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.769 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.770 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.770 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.771 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.771 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.771 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.772 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.772 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.772 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.773 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.773 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.773 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.773 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.774 INFO:teuthology.orchestra.run.smithi070.stdout: "max_concurrent_clones": { 2023-10-18T12:40:16.774 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "max_concurrent_clones", 2023-10-18T12:40:16.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.775 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.775 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.776 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "4", 2023-10-18T12:40:16.776 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.776 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.776 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.777 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-18T12:40:16.777 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.777 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.777 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.778 INFO:teuthology.orchestra.run.smithi070.stdout: "periodic_async_work": { 2023-10-18T12:40:16.778 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "periodic_async_work", 2023-10-18T12:40:16.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.779 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.779 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.780 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.780 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.781 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.781 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Periodically check for async work", 2023-10-18T12:40:16.781 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.781 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.782 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.782 INFO:teuthology.orchestra.run.smithi070.stdout: "snapshot_clone_delay": { 2023-10-18T12:40:16.783 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snapshot_clone_delay", 2023-10-18T12:40:16.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.783 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.784 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.784 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "0", 2023-10-18T12:40:16.784 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.784 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.785 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.785 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-18T12:40:16.785 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.786 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.786 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.787 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix", 2023-10-18T12:40:16.788 INFO:teuthology.orchestra.run.smithi070.stdout: "can_run": true, 2023-10-18T12:40:16.788 INFO:teuthology.orchestra.run.smithi070.stdout: "error_string": "", 2023-10-18T12:40:16.788 INFO:teuthology.orchestra.run.smithi070.stdout: "module_options": { 2023-10-18T12:40:16.789 INFO:teuthology.orchestra.run.smithi070.stdout: "discovery_interval": { 2023-10-18T12:40:16.789 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "discovery_interval", 2023-10-18T12:40:16.789 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "uint", 2023-10-18T12:40:16.789 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.790 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.790 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "100", 2023-10-18T12:40:16.790 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.791 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.791 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.791 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.792 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.792 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.793 INFO:teuthology.orchestra.run.smithi070.stdout: "identifier": { 2023-10-18T12:40:16.793 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "identifier", 2023-10-18T12:40:16.793 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.794 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.794 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.794 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.795 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.795 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.795 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.795 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.796 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.796 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.796 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.797 INFO:teuthology.orchestra.run.smithi070.stdout: "interval": { 2023-10-18T12:40:16.797 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "interval", 2023-10-18T12:40:16.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "secs", 2023-10-18T12:40:16.798 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.798 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.798 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "60", 2023-10-18T12:40:16.799 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.799 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.799 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.800 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.800 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.800 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.801 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.801 INFO:teuthology.orchestra.run.smithi070.stdout: "log_level": { 2023-10-18T12:40:16.802 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_level", 2023-10-18T12:40:16.802 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.802 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.802 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.803 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.803 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.803 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.804 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.804 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.804 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.805 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.805 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.805 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.805 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.806 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.806 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.806 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.807 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.807 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.808 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster": { 2023-10-18T12:40:16.808 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster", 2023-10-18T12:40:16.808 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.808 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.809 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.809 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.809 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.810 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.810 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.810 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.811 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.811 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.812 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_cluster_level": { 2023-10-18T12:40:16.812 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_cluster_level", 2023-10-18T12:40:16.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.813 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.813 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.813 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "info", 2023-10-18T12:40:16.814 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.814 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.814 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [ 2023-10-18T12:40:16.815 INFO:teuthology.orchestra.run.smithi070.stdout: "", 2023-10-18T12:40:16.815 INFO:teuthology.orchestra.run.smithi070.stdout: "critical", 2023-10-18T12:40:16.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug", 2023-10-18T12:40:16.816 INFO:teuthology.orchestra.run.smithi070.stdout: "error", 2023-10-18T12:40:16.816 INFO:teuthology.orchestra.run.smithi070.stdout: "info", 2023-10-18T12:40:16.816 INFO:teuthology.orchestra.run.smithi070.stdout: "warning" 2023-10-18T12:40:16.817 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.817 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.817 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.818 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.819 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": { 2023-10-18T12:40:16.819 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "log_to_file", 2023-10-18T12:40:16.819 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bool", 2023-10-18T12:40:16.819 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-10-18T12:40:16.820 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "False", 2023-10-18T12:40:16.820 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.820 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.821 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.821 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.821 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.822 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.822 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.822 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_host": { 2023-10-18T12:40:16.823 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_host", 2023-10-18T12:40:16.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.823 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.824 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.824 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "", 2023-10-18T12:40:16.824 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.825 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.825 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.825 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.825 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.826 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.826 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.827 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_port": { 2023-10-18T12:40:16.827 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_port", 2023-10-18T12:40:16.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "int", 2023-10-18T12:40:16.828 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.828 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.828 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "10051", 2023-10-18T12:40:16.828 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.829 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.829 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.829 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.830 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.830 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.830 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.831 INFO:teuthology.orchestra.run.smithi070.stdout: "zabbix_sender": { 2023-10-18T12:40:16.831 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "zabbix_sender", 2023-10-18T12:40:16.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "str", 2023-10-18T12:40:16.832 INFO:teuthology.orchestra.run.smithi070.stdout: "level": "advanced", 2023-10-18T12:40:16.832 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 0, 2023-10-18T12:40:16.832 INFO:teuthology.orchestra.run.smithi070.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-18T12:40:16.833 INFO:teuthology.orchestra.run.smithi070.stdout: "min": "", 2023-10-18T12:40:16.833 INFO:teuthology.orchestra.run.smithi070.stdout: "max": "", 2023-10-18T12:40:16.833 INFO:teuthology.orchestra.run.smithi070.stdout: "enum_allowed": [], 2023-10-18T12:40:16.834 INFO:teuthology.orchestra.run.smithi070.stdout: "desc": "", 2023-10-18T12:40:16.834 INFO:teuthology.orchestra.run.smithi070.stdout: "long_desc": "", 2023-10-18T12:40:16.834 INFO:teuthology.orchestra.run.smithi070.stdout: "tags": [], 2023-10-18T12:40:16.834 INFO:teuthology.orchestra.run.smithi070.stdout: "see_also": [] 2023-10-18T12:40:16.835 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.835 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.835 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.836 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.836 INFO:teuthology.orchestra.run.smithi070.stdout: "services": { 2023-10-18T12:40:16.836 INFO:teuthology.orchestra.run.smithi070.stdout: "dashboard": "https://172.21.15.70:8443/" 2023-10-18T12:40:16.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.837 INFO:teuthology.orchestra.run.smithi070.stdout: "always_on_modules": { 2023-10-18T12:40:16.837 INFO:teuthology.orchestra.run.smithi070.stdout: "octopus": [ 2023-10-18T12:40:16.838 INFO:teuthology.orchestra.run.smithi070.stdout: "balancer", 2023-10-18T12:40:16.838 INFO:teuthology.orchestra.run.smithi070.stdout: "crash", 2023-10-18T12:40:16.838 INFO:teuthology.orchestra.run.smithi070.stdout: "devicehealth", 2023-10-18T12:40:16.838 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator", 2023-10-18T12:40:16.839 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscaler", 2023-10-18T12:40:16.839 INFO:teuthology.orchestra.run.smithi070.stdout: "progress", 2023-10-18T12:40:16.839 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_support", 2023-10-18T12:40:16.840 INFO:teuthology.orchestra.run.smithi070.stdout: "status", 2023-10-18T12:40:16.840 INFO:teuthology.orchestra.run.smithi070.stdout: "telemetry", 2023-10-18T12:40:16.840 INFO:teuthology.orchestra.run.smithi070.stdout: "volumes" 2023-10-18T12:40:16.841 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.841 INFO:teuthology.orchestra.run.smithi070.stdout: "pacific": [ 2023-10-18T12:40:16.841 INFO:teuthology.orchestra.run.smithi070.stdout: "balancer", 2023-10-18T12:40:16.841 INFO:teuthology.orchestra.run.smithi070.stdout: "crash", 2023-10-18T12:40:16.842 INFO:teuthology.orchestra.run.smithi070.stdout: "devicehealth", 2023-10-18T12:40:16.842 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator", 2023-10-18T12:40:16.842 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscaler", 2023-10-18T12:40:16.843 INFO:teuthology.orchestra.run.smithi070.stdout: "progress", 2023-10-18T12:40:16.843 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_support", 2023-10-18T12:40:16.843 INFO:teuthology.orchestra.run.smithi070.stdout: "status", 2023-10-18T12:40:16.844 INFO:teuthology.orchestra.run.smithi070.stdout: "telemetry", 2023-10-18T12:40:16.844 INFO:teuthology.orchestra.run.smithi070.stdout: "volumes" 2023-10-18T12:40:16.844 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.844 INFO:teuthology.orchestra.run.smithi070.stdout: "quincy": [ 2023-10-18T12:40:16.845 INFO:teuthology.orchestra.run.smithi070.stdout: "balancer", 2023-10-18T12:40:16.845 INFO:teuthology.orchestra.run.smithi070.stdout: "crash", 2023-10-18T12:40:16.845 INFO:teuthology.orchestra.run.smithi070.stdout: "devicehealth", 2023-10-18T12:40:16.846 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator", 2023-10-18T12:40:16.846 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscaler", 2023-10-18T12:40:16.846 INFO:teuthology.orchestra.run.smithi070.stdout: "progress", 2023-10-18T12:40:16.847 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_support", 2023-10-18T12:40:16.847 INFO:teuthology.orchestra.run.smithi070.stdout: "status", 2023-10-18T12:40:16.847 INFO:teuthology.orchestra.run.smithi070.stdout: "telemetry", 2023-10-18T12:40:16.847 INFO:teuthology.orchestra.run.smithi070.stdout: "volumes" 2023-10-18T12:40:16.848 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-10-18T12:40:16.848 INFO:teuthology.orchestra.run.smithi070.stdout: "reef": [ 2023-10-18T12:40:16.848 INFO:teuthology.orchestra.run.smithi070.stdout: "balancer", 2023-10-18T12:40:16.849 INFO:teuthology.orchestra.run.smithi070.stdout: "crash", 2023-10-18T12:40:16.849 INFO:teuthology.orchestra.run.smithi070.stdout: "devicehealth", 2023-10-18T12:40:16.849 INFO:teuthology.orchestra.run.smithi070.stdout: "orchestrator", 2023-10-18T12:40:16.850 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscaler", 2023-10-18T12:40:16.850 INFO:teuthology.orchestra.run.smithi070.stdout: "progress", 2023-10-18T12:40:16.850 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_support", 2023-10-18T12:40:16.851 INFO:teuthology.orchestra.run.smithi070.stdout: "status", 2023-10-18T12:40:16.851 INFO:teuthology.orchestra.run.smithi070.stdout: "telemetry", 2023-10-18T12:40:16.851 INFO:teuthology.orchestra.run.smithi070.stdout: "volumes" 2023-10-18T12:40:16.851 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.852 INFO:teuthology.orchestra.run.smithi070.stdout: "last_failure_osd_epoch": 5, 2023-10-18T12:40:16.852 INFO:teuthology.orchestra.run.smithi070.stdout: "active_clients": [ 2023-10-18T12:40:16.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.853 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "devicehealth", 2023-10-18T12:40:16.853 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:16.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:16.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-18T12:40:16.854 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2279769141 2023-10-18T12:40:16.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.856 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "libcephsqlite", 2023-10-18T12:40:16.856 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:16.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:16.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-18T12:40:16.857 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 808797426 2023-10-18T12:40:16.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.858 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.859 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd_support", 2023-10-18T12:40:16.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:16.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:16.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-18T12:40:16.861 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2387190081 2023-10-18T12:40:16.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.861 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.862 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snap_schedule", 2023-10-18T12:40:16.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:16.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:16.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-18T12:40:16.864 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1286737455 2023-10-18T12:40:16.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.864 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.865 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "snap_schedule", 2023-10-18T12:40:16.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:16.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:16.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-18T12:40:16.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3495928258 2023-10-18T12:40:16.867 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.867 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:16.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.868 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "volumes", 2023-10-18T12:40:16.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-10-18T12:40:16.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-18T12:40:16.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-10-18T12:40:16.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-18T12:40:16.870 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3565418172 2023-10-18T12:40:16.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.870 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.871 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:16.871 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-18T12:40:16.871 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:40:16.874 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:11 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/723736819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:40:16.874 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:11 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/488299454' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-18T12:40:16.874 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:12 smithi152 ceph-mon[134767]: pgmap v755: 129 pgs: 129 active+clean; 765 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 305 op/s 2023-10-18T12:40:16.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:14 smithi152 ceph-mon[134767]: pgmap v756: 129 pgs: 129 active+clean; 766 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 3.1 MiB/s wr, 165 op/s 2023-10-18T12:40:16.875 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:14 smithi152 ceph-mon[134767]: osdmap e109: 12 total, 12 up, 12 in 2023-10-18T12:40:16.875 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:11 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/723736819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:40:16.876 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:11 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/488299454' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-18T12:40:16.876 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:12 smithi186 ceph-mon[135253]: pgmap v755: 129 pgs: 129 active+clean; 765 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 305 op/s 2023-10-18T12:40:16.876 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:14 smithi186 ceph-mon[135253]: pgmap v756: 129 pgs: 129 active+clean; 766 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 3.1 MiB/s wr, 165 op/s 2023-10-18T12:40:16.876 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:14 smithi186 ceph-mon[135253]: osdmap e109: 12 total, 12 up, 12 in 2023-10-18T12:40:16.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbb040a1670 msgr2=0x7fbb040a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:16.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbb040a1670 0x7fbb040a3b30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb0800b260 comp rx=0 tx=0).stop 2023-10-18T12:40:16.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 msgr2=0x7fbb1819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:16.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb1819c540 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000ef30 tx=0x7fbb1000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:40:16.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 shutdown_connections 2023-10-18T12:40:16.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb181066b0 0x7fbb1819c540 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:16.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fbb040a1670 0x7fbb040a3b30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:16.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb181006a0 0x7fbb18198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:16.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 --2- 172.21.15.70:0/488299454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb18101060 0x7fbb18199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:16.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 >> 172.21.15.70:0/488299454 conn(0x7fbb180fc1d0 msgr2=0x7fbb1810be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:16.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 shutdown_connections 2023-10-18T12:40:16.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:11.616+0000 7fbb1e7ff700 1 -- 172.21.15.70:0/488299454 wait complete. 2023-10-18T12:40:16.887 DEBUG:tasks.check_counter:Getting stats from a 2023-10-18T12:40:16.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-18T12:40:16.889 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:11 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/723736819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:40:16.889 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:11 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/488299454' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-18T12:40:16.889 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:12 smithi070 ceph-mon[98639]: pgmap v755: 129 pgs: 129 active+clean; 765 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 305 op/s 2023-10-18T12:40:16.889 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:14 smithi070 ceph-mon[98639]: pgmap v756: 129 pgs: 129 active+clean; 766 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 3.1 MiB/s wr, 165 op/s 2023-10-18T12:40:16.890 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:14 smithi070 ceph-mon[98639]: osdmap e109: 12 total, 12 up, 12 in 2023-10-18T12:40:17.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:16 smithi186 ceph-mon[135253]: pgmap v758: 129 pgs: 129 active+clean; 767 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 652 KiB/s wr, 108 op/s 2023-10-18T12:40:17.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:16 smithi070 ceph-mon[98639]: pgmap v758: 129 pgs: 129 active+clean; 767 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 652 KiB/s wr, 108 op/s 2023-10-18T12:40:17.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:16 smithi152 ceph-mon[134767]: pgmap v758: 129 pgs: 129 active+clean; 767 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 652 KiB/s wr, 108 op/s 2023-10-18T12:40:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:40:19.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:18 smithi070 ceph-mon[98639]: pgmap v759: 129 pgs: 129 active+clean; 767 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 269 KiB/s wr, 26 op/s 2023-10-18T12:40:19.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:18 smithi186 ceph-mon[135253]: pgmap v759: 129 pgs: 129 active+clean; 767 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 269 KiB/s wr, 26 op/s 2023-10-18T12:40:19.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:18 smithi152 ceph-mon[134767]: pgmap v759: 129 pgs: 129 active+clean; 767 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 269 KiB/s wr, 26 op/s 2023-10-18T12:40:19.494 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:40:19.495 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:19.495 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 270, 2023-10-18T12:40:19.495 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 263, 2023-10-18T12:40:19.496 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 761008, 2023-10-18T12:40:19.496 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 157552, 2023-10-18T12:40:19.496 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:19.496 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:19.497 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.054496096, 2023-10-18T12:40:19.497 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.021229288, 2023-10-18T12:40:19.497 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.023169200, 2023-10-18T12:40:19.498 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.008283200, 2023-10-18T12:40:19.498 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:19.498 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 263, 2023-10-18T12:40:19.498 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.019158635, 2023-10-18T12:40:19.499 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000072846 2023-10-18T12:40:19.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.499 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:19.499 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:19.500 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:19.500 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:19.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.501 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 761008, 2023-10-18T12:40:19.501 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 157552 2023-10-18T12:40:19.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.501 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:19.502 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 2, 2023-10-18T12:40:19.502 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 200, 2023-10-18T12:40:19.502 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 320, 2023-10-18T12:40:19.503 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 188512, 2023-10-18T12:40:19.503 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:19.503 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:19.503 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.018533526, 2023-10-18T12:40:19.504 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.013798088, 2023-10-18T12:40:19.504 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.002293187, 2023-10-18T12:40:19.504 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:19.505 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:19.505 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 198, 2023-10-18T12:40:19.505 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.012412158, 2023-10-18T12:40:19.505 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000062687 2023-10-18T12:40:19.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.506 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:19.506 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:19.507 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:19.507 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:19.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.507 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T12:40:19.508 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 188512 2023-10-18T12:40:19.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.508 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:19.509 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:19.509 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:19.509 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:19.509 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:19.510 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:19.510 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:19.510 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.001106988, 2023-10-18T12:40:19.511 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:19.511 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.000973184, 2023-10-18T12:40:19.511 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:19.512 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:19.512 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:19.512 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:19.512 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:19.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.513 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:19.513 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:19.514 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:19.514 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:19.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.515 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:19.515 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:19.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.516 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-10-18T12:40:19.516 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 0, 2023-10-18T12:40:19.517 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-10-18T12:40:19.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.518 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-10-18T12:40:19.518 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:19.519 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:19.519 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:19.520 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:19.520 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:19.521 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:19.521 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:19.522 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:19.522 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:19.523 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:19.523 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:19.524 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:19.524 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:19.525 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:19.525 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:19.526 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:19.526 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:19.526 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:19.527 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:19.527 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:19.527 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:19.527 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:19.528 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:19.528 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:19.528 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:19.529 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:19.529 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:19.529 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:19.529 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:19.530 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:19.530 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:19.530 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-10-18T12:40:19.531 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:19.531 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:19.531 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:19.532 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:19.532 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 147920, 2023-10-18T12:40:19.532 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 27, 2023-10-18T12:40:19.533 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:19.533 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:19.533 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-10-18T12:40:19.533 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-10-18T12:40:19.534 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:19.534 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:19.534 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:19.535 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:19.535 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:19.535 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 0, 2023-10-18T12:40:19.535 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:19.536 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:19.536 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:19.536 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-10-18T12:40:19.537 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:19.537 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 2, 2023-10-18T12:40:19.537 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:19.538 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-10-18T12:40:19.538 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:19.538 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-10-18T12:40:19.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:19.539 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:19.539 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:19.539 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-10-18T12:40:19.540 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:19.540 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 268, 2023-10-18T12:40:19.540 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 720485, 2023-10-18T12:40:19.541 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:19.541 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 268, 2023-10-18T12:40:19.541 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-10-18T12:40:19.542 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-10-18T12:40:19.542 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 268, 2023-10-18T12:40:19.542 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 720485, 2023-10-18T12:40:19.542 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:19.543 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:19.543 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:19.543 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:19.544 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:19.544 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:19.544 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:40:19.933 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-10-18T12:40:19.934 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-10-18T12:40:19.934 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-10-18T12:40:19.935 DEBUG:tasks.check_counter:Getting stats from d 2023-10-18T12:40:19.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-18T12:40:20.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:20 smithi152 ceph-mon[134767]: pgmap v760: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-10-18T12:40:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:40:20.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:20 smithi186 ceph-mon[135253]: pgmap v760: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-10-18T12:40:20.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:20 smithi070 ceph-mon[98639]: pgmap v760: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-10-18T12:40:22.336 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:40:22.337 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:22.337 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 272, 2023-10-18T12:40:22.338 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 267, 2023-10-18T12:40:22.339 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 759792, 2023-10-18T12:40:22.339 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 160528, 2023-10-18T12:40:22.339 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:22.340 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:22.340 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.048344226, 2023-10-18T12:40:22.341 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.019352896, 2023-10-18T12:40:22.341 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.023718929, 2023-10-18T12:40:22.341 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.005332677, 2023-10-18T12:40:22.342 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:22.342 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 266, 2023-10-18T12:40:22.343 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.013395204, 2023-10-18T12:40:22.343 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000050357 2023-10-18T12:40:22.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.344 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:22.344 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:22.344 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:22.344 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:22.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.345 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 759792, 2023-10-18T12:40:22.345 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 160528 2023-10-18T12:40:22.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.346 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:22.346 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1, 2023-10-18T12:40:22.346 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 203, 2023-10-18T12:40:22.347 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 160, 2023-10-18T12:40:22.347 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 112208, 2023-10-18T12:40:22.347 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 3, 2023-10-18T12:40:22.347 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:22.348 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.019050242, 2023-10-18T12:40:22.348 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.015117286, 2023-10-18T12:40:22.348 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.001556200, 2023-10-18T12:40:22.349 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:22.349 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:22.349 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 201, 2023-10-18T12:40:22.350 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.020637712, 2023-10-18T12:40:22.350 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000102675 2023-10-18T12:40:22.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.350 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:22.351 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:22.351 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:22.351 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:22.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.352 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-18T12:40:22.352 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 112208 2023-10-18T12:40:22.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.353 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:22.353 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 4, 2023-10-18T12:40:22.353 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 4, 2023-10-18T12:40:22.354 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 2944, 2023-10-18T12:40:22.354 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 448, 2023-10-18T12:40:22.354 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:22.355 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:22.355 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.001186894, 2023-10-18T12:40:22.355 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.000071326, 2023-10-18T12:40:22.355 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.001053716, 2023-10-18T12:40:22.356 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:22.356 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:22.356 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3, 2023-10-18T12:40:22.357 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.001133424, 2023-10-18T12:40:22.357 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000377808 2023-10-18T12:40:22.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.357 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:22.358 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:22.358 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:22.358 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:22.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.359 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-18T12:40:22.359 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T12:40:22.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.360 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-10-18T12:40:22.360 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 0, 2023-10-18T12:40:22.360 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-10-18T12:40:22.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.361 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-10-18T12:40:22.361 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:22.361 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:22.361 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:22.362 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:22.362 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:22.362 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:22.363 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:22.363 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:22.363 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:22.364 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:22.364 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:22.364 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:22.364 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:22.365 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:22.365 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:22.365 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:22.366 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:22.366 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:22.366 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:22.366 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:22.367 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:22.367 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:22.367 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:22.368 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:22.368 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:22.368 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:22.368 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:22.369 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:22.369 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:22.369 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:22.370 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:22.370 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-10-18T12:40:22.370 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:22.370 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:22.371 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:22.371 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:22.371 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 149007, 2023-10-18T12:40:22.372 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 30, 2023-10-18T12:40:22.372 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:22.372 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:22.372 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-10-18T12:40:22.373 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-10-18T12:40:22.373 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:22.373 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:22.373 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:22.374 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:22.374 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:22.374 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 0, 2023-10-18T12:40:22.375 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:22.375 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:22.375 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:22.376 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-10-18T12:40:22.376 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:22.376 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 2, 2023-10-18T12:40:22.376 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:22.377 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-10-18T12:40:22.377 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:22.377 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-10-18T12:40:22.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:22.378 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:22.378 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:22.378 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-10-18T12:40:22.379 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:22.379 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 273, 2023-10-18T12:40:22.379 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 721341, 2023-10-18T12:40:22.380 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:22.380 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 273, 2023-10-18T12:40:22.380 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-10-18T12:40:22.380 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-10-18T12:40:22.381 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 273, 2023-10-18T12:40:22.381 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 721341, 2023-10-18T12:40:22.381 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:22.381 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:22.382 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:22.382 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:22.382 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:22.383 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:22.383 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:40:22.765 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-10-18T12:40:22.765 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-10-18T12:40:22.766 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-10-18T12:40:22.766 DEBUG:tasks.check_counter:Getting stats from g 2023-10-18T12:40:22.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-18T12:40:23.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:22 smithi070 ceph-mon[98639]: pgmap v761: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 326 KiB/s wr, 14 op/s 2023-10-18T12:40:23.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:22 smithi186 ceph-mon[135253]: pgmap v761: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 326 KiB/s wr, 14 op/s 2023-10-18T12:40:23.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:22 smithi152 ceph-mon[134767]: pgmap v761: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 326 KiB/s wr, 14 op/s 2023-10-18T12:40:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:40:25.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:24 smithi186 ceph-mon[135253]: pgmap v762: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 286 KiB/s wr, 10 op/s 2023-10-18T12:40:25.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:24 smithi070 ceph-mon[98639]: pgmap v762: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 286 KiB/s wr, 10 op/s 2023-10-18T12:40:25.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:24 smithi152 ceph-mon[134767]: pgmap v762: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 286 KiB/s wr, 10 op/s 2023-10-18T12:40:27.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:26 smithi070 ceph-mon[98639]: pgmap v763: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 243 KiB/s wr, 9 op/s 2023-10-18T12:40:27.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:26 smithi186 ceph-mon[135253]: pgmap v763: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 243 KiB/s wr, 9 op/s 2023-10-18T12:40:27.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:26 smithi152 ceph-mon[134767]: pgmap v763: 129 pgs: 129 active+clean; 768 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 243 KiB/s wr, 9 op/s 2023-10-18T12:40:28.051 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:40:28.051 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:28.052 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 2791, 2023-10-18T12:40:28.052 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 2793, 2023-10-18T12:40:28.052 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 567029877, 2023-10-18T12:40:28.053 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 868011, 2023-10-18T12:40:28.053 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 8, 2023-10-18T12:40:28.053 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 4, 2023-10-18T12:40:28.053 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 1.033656912, 2023-10-18T12:40:28.054 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.086664686, 2023-10-18T12:40:28.054 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.840514266, 2023-10-18T12:40:28.054 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.076676469, 2023-10-18T12:40:28.055 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:28.055 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2791, 2023-10-18T12:40:28.055 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.122336481, 2023-10-18T12:40:28.055 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000043832 2023-10-18T12:40:28.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.056 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:28.056 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.057 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.057 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.057 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1609, 2023-10-18T12:40:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 1807, 2023-10-18T12:40:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 512126713, 2023-10-18T12:40:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 921166, 2023-10-18T12:40:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 7, 2023-10-18T12:40:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 6, 2023-10-18T12:40:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.845657447, 2023-10-18T12:40:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.056015749, 2023-10-18T12:40:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.733908917, 2023-10-18T12:40:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.039088586, 2023-10-18T12:40:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:28.062 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1806, 2023-10-18T12:40:28.062 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.082792393, 2023-10-18T12:40:28.062 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000045842 2023-10-18T12:40:28.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.063 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:28.063 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.063 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.064 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.064 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-18T12:40:28.065 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 422256 2023-10-18T12:40:28.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.065 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:28.065 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1394, 2023-10-18T12:40:28.066 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 1388, 2023-10-18T12:40:28.066 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 324055296, 2023-10-18T12:40:28.066 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 504960, 2023-10-18T12:40:28.067 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 5, 2023-10-18T12:40:28.067 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 4, 2023-10-18T12:40:28.067 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.603900487, 2023-10-18T12:40:28.067 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.044709065, 2023-10-18T12:40:28.068 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.516958224, 2023-10-18T12:40:28.068 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.034646535, 2023-10-18T12:40:28.068 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:28.069 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1387, 2023-10-18T12:40:28.069 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.060693771, 2023-10-18T12:40:28.069 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000043759 2023-10-18T12:40:28.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.070 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:28.070 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.070 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.071 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.071 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 795840, 2023-10-18T12:40:28.071 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 163792 2023-10-18T12:40:28.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.072 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-10-18T12:40:28.072 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2023-10-18T12:40:28.073 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-10-18T12:40:28.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.073 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2023-10-18T12:40:28.073 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-10-18T12:40:28.074 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-10-18T12:40:28.074 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 8024, 2023-10-18T12:40:28.074 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.418348429, 2023-10-18T12:40:28.075 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000052137 2023-10-18T12:40:28.075 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.075 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2023-10-18T12:40:28.076 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-10-18T12:40:28.076 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-10-18T12:40:28.076 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.076 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.077 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.077 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.078 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2023-10-18T12:40:28.078 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 0, 2023-10-18T12:40:28.078 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 0, 2023-10-18T12:40:28.078 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2023-10-18T12:40:28.079 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.079 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.079 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.080 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2023-10-18T12:40:28.080 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 0, 2023-10-18T12:40:28.080 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2023-10-18T12:40:28.081 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2023-10-18T12:40:28.081 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 0, 2023-10-18T12:40:28.081 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2023-10-18T12:40:28.081 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2023-10-18T12:40:28.082 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 46941, 2023-10-18T12:40:28.082 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 20547, 2023-10-18T12:40:28.082 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 26228, 2023-10-18T12:40:28.083 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 166, 2023-10-18T12:40:28.083 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 17589, 2023-10-18T12:40:28.083 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 11877, 2023-10-18T12:40:28.083 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 0, 2023-10-18T12:40:28.084 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 0, 2023-10-18T12:40:28.084 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 2, 2023-10-18T12:40:28.084 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 0, 2023-10-18T12:40:28.084 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 0, 2023-10-18T12:40:28.085 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 0, 2023-10-18T12:40:28.085 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 0, 2023-10-18T12:40:28.085 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2023-10-18T12:40:28.086 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2023-10-18T12:40:28.086 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2023-10-18T12:40:28.086 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 0, 2023-10-18T12:40:28.086 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2023-10-18T12:40:28.087 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 0, 2023-10-18T12:40:28.087 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 0, 2023-10-18T12:40:28.087 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 0, 2023-10-18T12:40:28.088 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 0, 2023-10-18T12:40:28.088 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2023-10-18T12:40:28.088 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 0, 2023-10-18T12:40:28.088 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2023-10-18T12:40:28.089 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 611, 2023-10-18T12:40:28.089 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 211189904, 2023-10-18T12:40:28.089 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 6, 2023-10-18T12:40:28.090 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 0, 2023-10-18T12:40:28.090 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2023-10-18T12:40:28.090 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T12:40:28.090 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2023-10-18T12:40:28.091 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 0, 2023-10-18T12:40:28.091 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T12:40:28.091 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T12:40:28.092 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 0, 2023-10-18T12:40:28.092 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 0, 2023-10-18T12:40:28.092 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 0, 2023-10-18T12:40:28.092 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 0, 2023-10-18T12:40:28.093 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T12:40:28.093 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T12:40:28.093 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T12:40:28.094 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 0, 2023-10-18T12:40:28.094 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 0, 2023-10-18T12:40:28.094 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 0, 2023-10-18T12:40:28.094 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 0 2023-10-18T12:40:28.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.095 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2023-10-18T12:40:28.095 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 0, 2023-10-18T12:40:28.096 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 0, 2023-10-18T12:40:28.096 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 0, 2023-10-18T12:40:28.096 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 0, 2023-10-18T12:40:28.096 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 0, 2023-10-18T12:40:28.097 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2023-10-18T12:40:28.097 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2023-10-18T12:40:28.097 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2023-10-18T12:40:28.097 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 0, 2023-10-18T12:40:28.098 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 0, 2023-10-18T12:40:28.098 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2023-10-18T12:40:28.098 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2023-10-18T12:40:28.099 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2023-10-18T12:40:28.099 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2023-10-18T12:40:28.099 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2023-10-18T12:40:28.100 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2023-10-18T12:40:28.100 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2023-10-18T12:40:28.100 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T12:40:28.100 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 0, 2023-10-18T12:40:28.101 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2023-10-18T12:40:28.101 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2023-10-18T12:40:28.101 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2023-10-18T12:40:28.102 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2023-10-18T12:40:28.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.102 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2023-10-18T12:40:28.102 INFO:teuthology.orchestra.run.smithi070.stdout: "evlrg": 0, 2023-10-18T12:40:28.103 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 0, 2023-10-18T12:40:28.103 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 0, 2023-10-18T12:40:28.103 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 0, 2023-10-18T12:40:28.103 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 254009, 2023-10-18T12:40:28.104 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2023-10-18T12:40:28.104 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 0, 2023-10-18T12:40:28.104 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 0, 2023-10-18T12:40:28.105 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 0, 2023-10-18T12:40:28.105 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 0, 2023-10-18T12:40:28.105 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 135, 2023-10-18T12:40:28.105 INFO:teuthology.orchestra.run.smithi070.stdout: "segmjr": 22, 2023-10-18T12:40:28.106 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2023-10-18T12:40:28.106 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 0, 2023-10-18T12:40:28.106 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 873557267, 2023-10-18T12:40:28.107 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 1405088014, 2023-10-18T12:40:28.107 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 1405079292, 2023-10-18T12:40:28.107 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2023-10-18T12:40:28.107 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.108 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.108 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.108 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 254009 2023-10-18T12:40:28.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.109 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2023-10-18T12:40:28.109 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 1683, 2023-10-18T12:40:28.110 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 58065, 2023-10-18T12:40:28.110 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 56382, 2023-10-18T12:40:28.110 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 175, 2023-10-18T12:40:28.110 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 345, 2023-10-18T12:40:28.111 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 170, 2023-10-18T12:40:28.111 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 46941, 2023-10-18T12:40:28.111 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 58818, 2023-10-18T12:40:28.112 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 11877, 2023-10-18T12:40:28.112 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 0, 2023-10-18T12:40:28.112 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 0, 2023-10-18T12:40:28.112 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 0, 2023-10-18T12:40:28.113 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 102424, 2023-10-18T12:40:28.113 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 207152 2023-10-18T12:40:28.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.113 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2023-10-18T12:40:28.114 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 0, 2023-10-18T12:40:28.114 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 0, 2023-10-18T12:40:28.114 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 0, 2023-10-18T12:40:28.115 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 0, 2023-10-18T12:40:28.115 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 0, 2023-10-18T12:40:28.115 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2023-10-18T12:40:28.115 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.116 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.116 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.117 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2023-10-18T12:40:28.117 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.117 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.117 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.118 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2023-10-18T12:40:28.118 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.119 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.119 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.119 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2023-10-18T12:40:28.120 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.120 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.120 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.121 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2023-10-18T12:40:28.121 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.121 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.122 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.122 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2023-10-18T12:40:28.123 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.123 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.123 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.124 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2023-10-18T12:40:28.124 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.124 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.124 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.125 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2023-10-18T12:40:28.125 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.126 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.126 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.126 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2023-10-18T12:40:28.127 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.127 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.127 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.128 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2023-10-18T12:40:28.128 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.128 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.129 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.129 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2023-10-18T12:40:28.130 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.130 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.130 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.131 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2023-10-18T12:40:28.131 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.131 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.132 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.132 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2023-10-18T12:40:28.132 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.133 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.133 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.134 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2023-10-18T12:40:28.134 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.134 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.134 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.135 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2023-10-18T12:40:28.135 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.136 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.136 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.136 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2023-10-18T12:40:28.137 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.137 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.137 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.138 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2023-10-18T12:40:28.138 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.138 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.139 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.139 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2023-10-18T12:40:28.139 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.140 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.140 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.141 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2023-10-18T12:40:28.141 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.141 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.141 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.142 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2023-10-18T12:40:28.142 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.143 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.143 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.143 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2023-10-18T12:40:28.144 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.144 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.144 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.145 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2023-10-18T12:40:28.145 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.146 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.146 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.146 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2023-10-18T12:40:28.147 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.147 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.147 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.148 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2023-10-18T12:40:28.148 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.148 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.149 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.149 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2023-10-18T12:40:28.149 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.150 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.150 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.151 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2023-10-18T12:40:28.151 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.151 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.151 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.152 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2023-10-18T12:40:28.152 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.153 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.153 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.153 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2023-10-18T12:40:28.154 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.154 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.154 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.155 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2023-10-18T12:40:28.155 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.155 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.156 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.156 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2023-10-18T12:40:28.157 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2023-10-18T12:40:28.157 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2023-10-18T12:40:28.157 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.157 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.158 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.158 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.159 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2023-10-18T12:40:28.159 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 2, 2023-10-18T12:40:28.159 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 4, 2023-10-18T12:40:28.159 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 2, 2023-10-18T12:40:28.160 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 2, 2023-10-18T12:40:28.160 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2023-10-18T12:40:28.160 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 0, 2023-10-18T12:40:28.161 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 0, 2023-10-18T12:40:28.161 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 2512, 2023-10-18T12:40:28.161 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2023-10-18T12:40:28.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.162 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-10-18T12:40:28.162 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:28.162 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:28.163 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:28.163 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:28.163 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:28.163 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:28.164 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:28.164 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:28.164 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:28.165 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:28.165 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:28.165 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:28.165 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:28.166 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:28.166 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:28.166 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:28.167 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:28.167 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:28.167 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:28.167 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:28.168 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:28.168 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:28.168 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:28.168 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:28.169 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:28.169 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:28.169 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:28.170 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:28.170 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:28.170 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:28.171 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:28.171 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-10-18T12:40:28.171 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:28.171 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:28.172 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:28.172 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:28.172 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 305094, 2023-10-18T12:40:28.173 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 2001, 2023-10-18T12:40:28.173 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:28.173 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:28.173 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-10-18T12:40:28.174 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-10-18T12:40:28.174 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:28.174 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:28.175 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:28.175 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:28.175 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 9448, 2023-10-18T12:40:28.175 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 181, 2023-10-18T12:40:28.176 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:28.176 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:28.176 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:28.176 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-10-18T12:40:28.177 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 37546142, 2023-10-18T12:40:28.177 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 250551, 2023-10-18T12:40:28.177 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:28.177 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-10-18T12:40:28.178 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:28.178 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-10-18T12:40:28.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.179 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2023-10-18T12:40:28.179 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2023-10-18T12:40:28.179 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2023-10-18T12:40:28.179 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 5492, 2023-10-18T12:40:28.180 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 26, 2023-10-18T12:40:28.180 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 4, 2023-10-18T12:40:28.180 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 5492, 2023-10-18T12:40:28.181 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2023-10-18T12:40:28.181 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 5492, 2023-10-18T12:40:28.181 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 8.365754859, 2023-10-18T12:40:28.181 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001523261 2023-10-18T12:40:28.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.182 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2023-10-18T12:40:28.182 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2023-10-18T12:40:28.183 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 5492, 2023-10-18T12:40:28.183 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 5493 2023-10-18T12:40:28.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.183 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 5492, 2023-10-18T12:40:28.184 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 5492, 2023-10-18T12:40:28.184 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 0, 2023-10-18T12:40:28.184 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2023-10-18T12:40:28.185 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2023-10-18T12:40:28.185 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 3503, 2023-10-18T12:40:28.185 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 0, 2023-10-18T12:40:28.185 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 1986, 2023-10-18T12:40:28.186 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 0, 2023-10-18T12:40:28.186 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 0, 2023-10-18T12:40:28.186 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2023-10-18T12:40:28.186 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2023-10-18T12:40:28.187 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2023-10-18T12:40:28.187 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2023-10-18T12:40:28.187 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 0, 2023-10-18T12:40:28.188 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2023-10-18T12:40:28.188 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2023-10-18T12:40:28.188 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2023-10-18T12:40:28.189 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 2, 2023-10-18T12:40:28.189 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 0, 2023-10-18T12:40:28.189 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2023-10-18T12:40:28.189 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2023-10-18T12:40:28.190 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2023-10-18T12:40:28.190 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2023-10-18T12:40:28.190 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2023-10-18T12:40:28.191 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2023-10-18T12:40:28.191 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T12:40:28.191 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2023-10-18T12:40:28.191 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2023-10-18T12:40:28.192 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2023-10-18T12:40:28.192 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2023-10-18T12:40:28.192 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2023-10-18T12:40:28.193 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2023-10-18T12:40:28.193 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2023-10-18T12:40:28.193 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2023-10-18T12:40:28.193 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2023-10-18T12:40:28.194 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2023-10-18T12:40:28.194 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2023-10-18T12:40:28.194 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2023-10-18T12:40:28.194 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2023-10-18T12:40:28.195 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2023-10-18T12:40:28.195 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2023-10-18T12:40:28.195 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2023-10-18T12:40:28.196 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2023-10-18T12:40:28.196 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2023-10-18T12:40:28.196 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2023-10-18T12:40:28.196 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 109, 2023-10-18T12:40:28.197 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2023-10-18T12:40:28.197 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 20, 2023-10-18T12:40:28.197 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 11, 2023-10-18T12:40:28.198 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 11, 2023-10-18T12:40:28.198 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2023-10-18T12:40:28.198 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2023-10-18T12:40:28.198 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 0, 2023-10-18T12:40:28.199 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 2, 2023-10-18T12:40:28.199 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 0 2023-10-18T12:40:28.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.199 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2023-10-18T12:40:28.200 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 0, 2023-10-18T12:40:28.200 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2023-10-18T12:40:28.200 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2023-10-18T12:40:28.201 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2023-10-18T12:40:28.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.201 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2023-10-18T12:40:28.202 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2023-10-18T12:40:28.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.202 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2023-10-18T12:40:28.202 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2023-10-18T12:40:28.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 0, 2023-10-18T12:40:28.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2023-10-18T12:40:28.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 0, 2023-10-18T12:40:28.204 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 0, 2023-10-18T12:40:28.204 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2023-10-18T12:40:28.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.204 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:28.205 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:28.205 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-10-18T12:40:28.205 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:28.206 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 5790, 2023-10-18T12:40:28.206 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1402672682, 2023-10-18T12:40:28.206 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:28.207 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 5790, 2023-10-18T12:40:28.207 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-10-18T12:40:28.207 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-10-18T12:40:28.207 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 5790, 2023-10-18T12:40:28.208 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1402672682, 2023-10-18T12:40:28.208 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:28.208 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.208 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.209 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.210 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2023-10-18T12:40:28.210 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:28.210 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-10-18T12:40:28.211 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:28.211 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-10-18T12:40:28.211 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-10-18T12:40:28.211 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:28.212 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-10-18T12:40:28.212 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 5492, 2023-10-18T12:40:28.212 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 1400926118, 2023-10-18T12:40:28.213 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 5492, 2023-10-18T12:40:28.213 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1400926118, 2023-10-18T12:40:28.213 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:28.213 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.214 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.214 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.215 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2023-10-18T12:40:28.215 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:28.215 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2023-10-18T12:40:28.216 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:28.216 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-10-18T12:40:28.216 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-10-18T12:40:28.216 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:28.217 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-10-18T12:40:28.217 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 5492, 2023-10-18T12:40:28.217 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 5492, 2023-10-18T12:40:28.218 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 5492, 2023-10-18T12:40:28.218 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 5492, 2023-10-18T12:40:28.218 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:28.218 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.219 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.219 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.219 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.220 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2023-10-18T12:40:28.220 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:28.220 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-10-18T12:40:28.220 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:28.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-10-18T12:40:28.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-10-18T12:40:28.221 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:28.222 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-10-18T12:40:28.222 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-10-18T12:40:28.222 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-10-18T12:40:28.222 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2023-10-18T12:40:28.223 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2023-10-18T12:40:28.223 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:28.223 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.223 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.224 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.224 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:28.225 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x5580d4a8ca00": { 2023-10-18T12:40:28.225 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:28.225 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-10-18T12:40:28.225 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:28.226 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-10-18T12:40:28.226 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-10-18T12:40:28.226 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:28.226 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-10-18T12:40:28.227 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-10-18T12:40:28.227 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-10-18T12:40:28.227 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2023-10-18T12:40:28.228 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2023-10-18T12:40:28.228 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:28.228 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:28.228 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:28.229 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:28.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:28.229 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:40:28.506 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2023-10-18T12:40:28.507 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-10-18T12:40:28.507 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-10-18T12:40:28.507 DEBUG:tasks.check_counter:Getting stats from j 2023-10-18T12:40:28.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-18T12:40:29.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:28 smithi186 ceph-mon[135253]: pgmap v764: 129 pgs: 129 active+clean; 772 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 613 KiB/s wr, 15 op/s 2023-10-18T12:40:29.247 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T12:40:29.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:28 smithi070 ceph-mon[98639]: pgmap v764: 129 pgs: 129 active+clean; 772 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 613 KiB/s wr, 15 op/s 2023-10-18T12:40:29.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:28 smithi152 ceph-mon[134767]: pgmap v764: 129 pgs: 129 active+clean; 772 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 613 KiB/s wr, 15 op/s 2023-10-18T12:40:30.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:30 smithi152 ceph-mon[134767]: pgmap v765: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 29 op/s 2023-10-18T12:40:30.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:30 smithi186 ceph-mon[135253]: pgmap v765: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 29 op/s 2023-10-18T12:40:30.802 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-18T12:40:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 275, 2023-10-18T12:40:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 270, 2023-10-18T12:40:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 760608, 2023-10-18T12:40:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 162400, 2023-10-18T12:40:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.056169900, 2023-10-18T12:40:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.024298065, 2023-10-18T12:40:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.026131846, 2023-10-18T12:40:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.005384128, 2023-10-18T12:40:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 269, 2023-10-18T12:40:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.016983652, 2023-10-18T12:40:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000063136 2023-10-18T12:40:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 760608, 2023-10-18T12:40:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 162400 2023-10-18T12:40:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 2, 2023-10-18T12:40:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 206, 2023-10-18T12:40:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 320, 2023-10-18T12:40:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 189568, 2023-10-18T12:40:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.019762293, 2023-10-18T12:40:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.014997253, 2023-10-18T12:40:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.002195830, 2023-10-18T12:40:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 204, 2023-10-18T12:40:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.013724533, 2023-10-18T12:40:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000067277 2023-10-18T12:40:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T12:40:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 189568 2023-10-18T12:40:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 4, 2023-10-18T12:40:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 4, 2023-10-18T12:40:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 2944, 2023-10-18T12:40:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 448, 2023-10-18T12:40:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.001288735, 2023-10-18T12:40:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.000081530, 2023-10-18T12:40:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.001161376, 2023-10-18T12:40:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3, 2023-10-18T12:40:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.001144049, 2023-10-18T12:40:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000381349 2023-10-18T12:40:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-18T12:40:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T12:40:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-10-18T12:40:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 0, 2023-10-18T12:40:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-10-18T12:40:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-10-18T12:40:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:30.831 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:30.831 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:30.831 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-10-18T12:40:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 147920, 2023-10-18T12:40:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 27, 2023-10-18T12:40:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-10-18T12:40:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-10-18T12:40:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 0, 2023-10-18T12:40:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-10-18T12:40:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 2, 2023-10-18T12:40:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-10-18T12:40:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-10-18T12:40:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-18T12:40:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-10-18T12:40:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-10-18T12:40:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-10-18T12:40:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 277, 2023-10-18T12:40:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 721758, 2023-10-18T12:40:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 277, 2023-10-18T12:40:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-10-18T12:40:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-10-18T12:40:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 277, 2023-10-18T12:40:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 721758, 2023-10-18T12:40:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-10-18T12:40:30.847 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-10-18T12:40:30.847 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-10-18T12:40:30.847 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-10-18T12:40:30.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:30.848 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-18T12:40:30.848 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-18T12:40:30.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:30 smithi070 ceph-mon[98639]: pgmap v765: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 29 op/s 2023-10-18T12:40:31.241 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-10-18T12:40:31.242 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-10-18T12:40:31.242 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-10-18T12:40:31.242 DEBUG:tasks.check_counter:Getting stats from b 2023-10-18T12:40:31.243 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-18T12:40:31.908 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:40:33.049 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:32 smithi152 ceph-mon[134767]: pgmap v766: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.7 MiB/s wr, 34 op/s 2023-10-18T12:40:33.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:32 smithi186 ceph-mon[135253]: pgmap v766: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.7 MiB/s wr, 34 op/s 2023-10-18T12:40:33.284 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-10-18T12:40:33.285 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:33.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 90938, 2023-10-18T12:40:33.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 90940, 2023-10-18T12:40:33.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 24773465, 2023-10-18T12:40:33.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 686607860, 2023-10-18T12:40:33.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 11, 2023-10-18T12:40:33.287 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2023-10-18T12:40:33.287 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 5.981350317, 2023-10-18T12:40:33.287 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 2.092125191, 2023-10-18T12:40:33.288 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 1.170438799, 2023-10-18T12:40:33.288 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 2.341506538, 2023-10-18T12:40:33.288 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:33.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 90938, 2023-10-18T12:40:33.289 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 17.813056550, 2023-10-18T12:40:33.289 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000195881 2023-10-18T12:40:33.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.290 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:33.290 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2023-10-18T12:40:33.290 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000234, 2023-10-18T12:40:33.291 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000117 2023-10-18T12:40:33.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.291 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:33.292 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:33.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.292 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:33.292 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 76389, 2023-10-18T12:40:33.293 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 75820, 2023-10-18T12:40:33.293 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 23536705, 2023-10-18T12:40:33.293 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 576592971, 2023-10-18T12:40:33.294 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 21, 2023-10-18T12:40:33.294 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 6, 2023-10-18T12:40:33.294 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 5.050782861, 2023-10-18T12:40:33.294 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 1.757075335, 2023-10-18T12:40:33.295 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 1.012558529, 2023-10-18T12:40:33.295 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 1.971527086, 2023-10-18T12:40:33.295 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:33.296 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 75818, 2023-10-18T12:40:33.296 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 15.315795508, 2023-10-18T12:40:33.296 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000202007 2023-10-18T12:40:33.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.297 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:33.297 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1212, 2023-10-18T12:40:33.297 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000202330, 2023-10-18T12:40:33.298 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000166 2023-10-18T12:40:33.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.298 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 431920, 2023-10-18T12:40:33.299 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 655504 2023-10-18T12:40:33.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.299 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:33.299 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 533986, 2023-10-18T12:40:33.300 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 883576, 2023-10-18T12:40:33.300 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 167414000, 2023-10-18T12:40:33.300 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 840507870, 2023-10-18T12:40:33.301 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 212, 2023-10-18T12:40:33.301 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 7, 2023-10-18T12:40:33.301 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 25.747734950, 2023-10-18T12:40:33.301 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 12.579748095, 2023-10-18T12:40:33.302 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 15.039352395, 2023-10-18T12:40:33.302 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 1.230243757, 2023-10-18T12:40:33.302 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:33.303 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 883576, 2023-10-18T12:40:33.303 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 40.449184453, 2023-10-18T12:40:33.303 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000045778 2023-10-18T12:40:33.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.304 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:33.304 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 701208, 2023-10-18T12:40:33.304 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.140017017, 2023-10-18T12:40:33.305 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000199 2023-10-18T12:40:33.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.305 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 870080, 2023-10-18T12:40:33.306 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 232880 2023-10-18T12:40:33.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.306 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2023-10-18T12:40:33.307 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2023-10-18T12:40:33.307 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2023-10-18T12:40:33.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.308 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2023-10-18T12:40:33.308 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2023-10-18T12:40:33.308 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2023-10-18T12:40:33.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 177428, 2023-10-18T12:40:33.309 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 141.928130053, 2023-10-18T12:40:33.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000799919 2023-10-18T12:40:33.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.310 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2023-10-18T12:40:33.310 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2023-10-18T12:40:33.311 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2023-10-18T12:40:33.311 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 69809, 2023-10-18T12:40:33.311 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 14.297746929, 2023-10-18T12:40:33.312 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000204812 2023-10-18T12:40:33.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2023-10-18T12:40:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 340082, 2023-10-18T12:40:33.313 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 340082, 2023-10-18T12:40:33.314 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2023-10-18T12:40:33.314 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 340082, 2023-10-18T12:40:33.314 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 738.479972006, 2023-10-18T12:40:33.315 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002171476 2023-10-18T12:40:33.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.315 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2023-10-18T12:40:33.316 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2023-10-18T12:40:33.316 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2023-10-18T12:40:33.316 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2023-10-18T12:40:33.316 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 352, 2023-10-18T12:40:33.317 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 22, 2023-10-18T12:40:33.317 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 38, 2023-10-18T12:40:33.317 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 5352, 2023-10-18T12:40:33.318 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 2068, 2023-10-18T12:40:33.318 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 3284, 2023-10-18T12:40:33.318 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2023-10-18T12:40:33.319 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 2398, 2023-10-18T12:40:33.319 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2023-10-18T12:40:33.319 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 714, 2023-10-18T12:40:33.320 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 715, 2023-10-18T12:40:33.320 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 2, 2023-10-18T12:40:33.320 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 427340, 2023-10-18T12:40:33.321 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 293771, 2023-10-18T12:40:33.321 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2023-10-18T12:40:33.321 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2023-10-18T12:40:33.321 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2023-10-18T12:40:33.322 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2023-10-18T12:40:33.322 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 1992, 2023-10-18T12:40:33.322 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 68858, 2023-10-18T12:40:33.323 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2023-10-18T12:40:33.323 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2023-10-18T12:40:33.323 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2023-10-18T12:40:33.324 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2023-10-18T12:40:33.324 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2023-10-18T12:40:33.324 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2023-10-18T12:40:33.325 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 3028, 2023-10-18T12:40:33.325 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2023-10-18T12:40:33.325 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 611, 2023-10-18T12:40:33.325 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 211189904, 2023-10-18T12:40:33.326 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 6, 2023-10-18T12:40:33.326 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 3028, 2023-10-18T12:40:33.326 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2023-10-18T12:40:33.327 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T12:40:33.327 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2023-10-18T12:40:33.327 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 626, 2023-10-18T12:40:33.328 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 27, 2023-10-18T12:40:33.328 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 599, 2023-10-18T12:40:33.328 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 2402, 2023-10-18T12:40:33.328 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2023-10-18T12:40:33.329 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 48688, 2023-10-18T12:40:33.329 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 277988, 2023-10-18T12:40:33.329 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T12:40:33.330 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T12:40:33.330 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T12:40:33.330 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 119175, 2023-10-18T12:40:33.331 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 45103, 2023-10-18T12:40:33.331 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 20147, 2023-10-18T12:40:33.331 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 226470 2023-10-18T12:40:33.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.332 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2023-10-18T12:40:33.332 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2023-10-18T12:40:33.332 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2023-10-18T12:40:33.333 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2023-10-18T12:40:33.333 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2023-10-18T12:40:33.333 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2023-10-18T12:40:33.333 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2023-10-18T12:40:33.334 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2023-10-18T12:40:33.334 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2023-10-18T12:40:33.334 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 56525, 2023-10-18T12:40:33.335 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 56525, 2023-10-18T12:40:33.335 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2023-10-18T12:40:33.335 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2023-10-18T12:40:33.336 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2023-10-18T12:40:33.336 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2023-10-18T12:40:33.336 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2023-10-18T12:40:33.336 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2023-10-18T12:40:33.337 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2023-10-18T12:40:33.337 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 27, 2023-10-18T12:40:33.337 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2023-10-18T12:40:33.338 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2023-10-18T12:40:33.338 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 60, 2023-10-18T12:40:33.338 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2023-10-18T12:40:33.339 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2023-10-18T12:40:33.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.339 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2023-10-18T12:40:33.340 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2023-10-18T12:40:33.340 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 254254, 2023-10-18T12:40:33.340 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 188427, 2023-10-18T12:40:33.340 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 184351, 2023-10-18T12:40:33.341 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 69903, 2023-10-18T12:40:33.341 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2023-10-18T12:40:33.341 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 4076, 2023-10-18T12:40:33.342 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 414, 2023-10-18T12:40:33.342 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 286, 2023-10-18T12:40:33.342 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 278, 2023-10-18T12:40:33.343 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 136, 2023-10-18T12:40:33.343 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 6, 2023-10-18T12:40:33.343 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2023-10-18T12:40:33.343 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 8, 2023-10-18T12:40:33.344 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 873557267, 2023-10-18T12:40:33.344 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 1405811018, 2023-10-18T12:40:33.344 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2023-10-18T12:40:33.345 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2023-10-18T12:40:33.345 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 90983, 2023-10-18T12:40:33.345 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 272.974215710, 2023-10-18T12:40:33.346 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.003000277 2023-10-18T12:40:33.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.346 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2023-10-18T12:40:33.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.347 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2023-10-18T12:40:33.347 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 1609, 2023-10-18T12:40:33.347 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 58123, 2023-10-18T12:40:33.348 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 56514, 2023-10-18T12:40:33.348 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 175, 2023-10-18T12:40:33.348 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 345, 2023-10-18T12:40:33.349 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 170, 2023-10-18T12:40:33.349 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 5247, 2023-10-18T12:40:33.349 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 111116, 2023-10-18T12:40:33.349 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 105869, 2023-10-18T12:40:33.350 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 712, 2023-10-18T12:40:33.350 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 57997, 2023-10-18T12:40:33.350 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 57285, 2023-10-18T12:40:33.351 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 86888, 2023-10-18T12:40:33.351 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 207152 2023-10-18T12:40:33.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.352 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2023-10-18T12:40:33.352 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 462955, 2023-10-18T12:40:33.352 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2023-10-18T12:40:33.352 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 340082, 2023-10-18T12:40:33.353 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 7791, 2023-10-18T12:40:33.353 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2023-10-18T12:40:33.353 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2023-10-18T12:40:33.354 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 57136, 2023-10-18T12:40:33.354 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 101.494180192, 2023-10-18T12:40:33.354 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001776361 2023-10-18T12:40:33.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.355 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2023-10-18T12:40:33.355 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 265, 2023-10-18T12:40:33.356 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.227133219, 2023-10-18T12:40:33.356 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000857106 2023-10-18T12:40:33.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.356 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2023-10-18T12:40:33.357 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.357 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.357 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.358 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2023-10-18T12:40:33.358 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.359 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.359 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.360 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2023-10-18T12:40:33.360 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 73167, 2023-10-18T12:40:33.360 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 73.981245595, 2023-10-18T12:40:33.360 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001011128 2023-10-18T12:40:33.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.361 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2023-10-18T12:40:33.361 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.362 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.362 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.363 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2023-10-18T12:40:33.363 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2023-10-18T12:40:33.363 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000285727, 2023-10-18T12:40:33.364 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000142863 2023-10-18T12:40:33.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.364 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2023-10-18T12:40:33.364 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.365 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.365 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.366 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2023-10-18T12:40:33.366 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.366 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.367 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.367 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2023-10-18T12:40:33.368 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.368 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.368 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.369 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2023-10-18T12:40:33.369 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 14, 2023-10-18T12:40:33.369 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.007560712, 2023-10-18T12:40:33.370 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000540050 2023-10-18T12:40:33.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.370 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2023-10-18T12:40:33.371 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 119, 2023-10-18T12:40:33.371 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.734559315, 2023-10-18T12:40:33.371 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.006172767 2023-10-18T12:40:33.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.372 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2023-10-18T12:40:33.372 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.372 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.373 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.373 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2023-10-18T12:40:33.373 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 14, 2023-10-18T12:40:33.374 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.016940306, 2023-10-18T12:40:33.374 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.072638593 2023-10-18T12:40:33.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.375 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2023-10-18T12:40:33.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 9, 2023-10-18T12:40:33.375 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.014373667, 2023-10-18T12:40:33.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001597074 2023-10-18T12:40:33.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.376 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2023-10-18T12:40:33.376 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 139153, 2023-10-18T12:40:33.377 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 409.675820420, 2023-10-18T12:40:33.377 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002944067 2023-10-18T12:40:33.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.377 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2023-10-18T12:40:33.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 11836, 2023-10-18T12:40:33.378 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 21.855225662, 2023-10-18T12:40:33.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001846504 2023-10-18T12:40:33.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.379 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2023-10-18T12:40:33.379 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.380 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.380 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.380 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2023-10-18T12:40:33.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.381 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.382 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2023-10-18T12:40:33.382 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2023-10-18T12:40:33.383 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000676412, 2023-10-18T12:40:33.383 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000676412 2023-10-18T12:40:33.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.383 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2023-10-18T12:40:33.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 8, 2023-10-18T12:40:33.384 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.110041557, 2023-10-18T12:40:33.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.013755194 2023-10-18T12:40:33.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.385 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2023-10-18T12:40:33.385 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.386 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.386 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.386 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2023-10-18T12:40:33.387 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 10, 2023-10-18T12:40:33.387 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.170526440, 2023-10-18T12:40:33.387 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.017052644 2023-10-18T12:40:33.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.388 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2023-10-18T12:40:33.388 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.389 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.389 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.390 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2023-10-18T12:40:33.390 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1816, 2023-10-18T12:40:33.390 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.427036750, 2023-10-18T12:40:33.390 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000785813 2023-10-18T12:40:33.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.391 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2023-10-18T12:40:33.391 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.392 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.392 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.392 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2023-10-18T12:40:33.393 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 7, 2023-10-18T12:40:33.393 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.020314193, 2023-10-18T12:40:33.393 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002902027 2023-10-18T12:40:33.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.394 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2023-10-18T12:40:33.394 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.394 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.395 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.395 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2023-10-18T12:40:33.396 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 56525, 2023-10-18T12:40:33.396 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 127.744051839, 2023-10-18T12:40:33.396 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002259956 2023-10-18T12:40:33.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.397 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2023-10-18T12:40:33.397 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2023-10-18T12:40:33.398 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2023-10-18T12:40:33.398 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.398 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.398 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.399 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.399 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2023-10-18T12:40:33.400 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 2, 2023-10-18T12:40:33.400 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 4, 2023-10-18T12:40:33.400 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2023-10-18T12:40:33.401 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 2, 2023-10-18T12:40:33.401 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2023-10-18T12:40:33.401 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 43343, 2023-10-18T12:40:33.401 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 21671, 2023-10-18T12:40:33.402 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 832, 2023-10-18T12:40:33.402 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2023-10-18T12:40:33.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.403 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2023-10-18T12:40:33.403 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:33.403 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:33.403 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:33.404 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:33.404 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:33.404 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:33.405 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:33.405 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:33.405 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:33.406 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:33.406 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:33.406 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:33.406 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:33.407 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:33.407 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:33.407 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:33.408 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:33.408 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:33.408 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:33.408 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:33.409 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:33.409 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:33.409 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:33.410 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:33.410 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:33.410 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:33.412 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:33.412 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2023-10-18T12:40:33.412 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:33.413 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:33.413 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:33.413 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:33.414 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 654100, 2023-10-18T12:40:33.414 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 1434, 2023-10-18T12:40:33.414 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 352, 2023-10-18T12:40:33.415 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 4, 2023-10-18T12:40:33.415 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2023-10-18T12:40:33.415 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2023-10-18T12:40:33.415 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:33.416 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:33.416 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:33.416 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:33.416 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9448, 2023-10-18T12:40:33.417 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 181, 2023-10-18T12:40:33.417 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:33.417 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:33.418 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:33.418 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2023-10-18T12:40:33.418 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 8488091, 2023-10-18T12:40:33.419 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 98222, 2023-10-18T12:40:33.419 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:33.419 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2023-10-18T12:40:33.419 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:33.420 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2023-10-18T12:40:33.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.420 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2023-10-18T12:40:33.420 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2023-10-18T12:40:33.421 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2023-10-18T12:40:33.421 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 211917, 2023-10-18T12:40:33.421 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 1414610809, 2023-10-18T12:40:33.422 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 31, 2023-10-18T12:40:33.422 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 211917, 2023-10-18T12:40:33.422 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2023-10-18T12:40:33.423 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 211917, 2023-10-18T12:40:33.423 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1712.010428864, 2023-10-18T12:40:33.423 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.008078683 2023-10-18T12:40:33.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.424 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2023-10-18T12:40:33.424 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2023-10-18T12:40:33.424 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 211917, 2023-10-18T12:40:33.424 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 213065 2023-10-18T12:40:33.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.425 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 211917, 2023-10-18T12:40:33.425 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 4272, 2023-10-18T12:40:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 207645, 2023-10-18T12:40:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2023-10-18T12:40:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2023-10-18T12:40:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 106, 2023-10-18T12:40:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 163, 2023-10-18T12:40:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 1217, 2023-10-18T12:40:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 92173, 2023-10-18T12:40:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 509, 2023-10-18T12:40:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2023-10-18T12:40:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2023-10-18T12:40:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2023-10-18T12:40:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 22, 2023-10-18T12:40:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 114253, 2023-10-18T12:40:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2023-10-18T12:40:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2023-10-18T12:40:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2023-10-18T12:40:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 3055, 2023-10-18T12:40:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 266, 2023-10-18T12:40:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2023-10-18T12:40:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2023-10-18T12:40:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2023-10-18T12:40:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2023-10-18T12:40:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2023-10-18T12:40:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2023-10-18T12:40:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T12:40:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2023-10-18T12:40:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2023-10-18T12:40:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2023-10-18T12:40:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2023-10-18T12:40:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2023-10-18T12:40:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2023-10-18T12:40:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2023-10-18T12:40:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2023-10-18T12:40:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2023-10-18T12:40:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2023-10-18T12:40:33.436 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2023-10-18T12:40:33.436 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2023-10-18T12:40:33.436 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2023-10-18T12:40:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2023-10-18T12:40:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2023-10-18T12:40:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2023-10-18T12:40:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2023-10-18T12:40:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2023-10-18T12:40:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2023-10-18T12:40:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 109, 2023-10-18T12:40:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2023-10-18T12:40:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 25, 2023-10-18T12:40:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 12, 2023-10-18T12:40:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 12, 2023-10-18T12:40:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2023-10-18T12:40:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2023-10-18T12:40:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 1008, 2023-10-18T12:40:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2023-10-18T12:40:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 293 2023-10-18T12:40:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2023-10-18T12:40:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2023-10-18T12:40:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2023-10-18T12:40:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2023-10-18T12:40:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2023-10-18T12:40:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2023-10-18T12:40:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2023-10-18T12:40:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2023-10-18T12:40:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2023-10-18T12:40:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 48, 2023-10-18T12:40:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2023-10-18T12:40:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 24, 2023-10-18T12:40:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 56525, 2023-10-18T12:40:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2023-10-18T12:40:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2023-10-18T12:40:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 701309, 2023-10-18T12:40:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 160335678, 2023-10-18T12:40:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 701309, 2023-10-18T12:40:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2023-10-18T12:40:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2023-10-18T12:40:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 701309, 2023-10-18T12:40:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 160335678, 2023-10-18T12:40:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2023-10-18T12:40:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2023-10-18T12:40:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2023-10-18T12:40:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2023-10-18T12:40:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2023-10-18T12:40:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 211917, 2023-10-18T12:40:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 1420779450, 2023-10-18T12:40:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 211917, 2023-10-18T12:40:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1420779450, 2023-10-18T12:40:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2023-10-18T12:40:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2023-10-18T12:40:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2023-10-18T12:40:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2023-10-18T12:40:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2023-10-18T12:40:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 211917, 2023-10-18T12:40:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 211917, 2023-10-18T12:40:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 211917, 2023-10-18T12:40:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 211917, 2023-10-18T12:40:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2023-10-18T12:40:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2023-10-18T12:40:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 56525, 2023-10-18T12:40:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 6168641, 2023-10-18T12:40:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 56525, 2023-10-18T12:40:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2023-10-18T12:40:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2023-10-18T12:40:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 1190, 2023-10-18T12:40:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 6168641, 2023-10-18T12:40:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x55a73cf766e0": { 2023-10-18T12:40:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 22762, 2023-10-18T12:40:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2023-10-18T12:40:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 254254, 2023-10-18T12:40:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1401616714, 2023-10-18T12:40:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 254254, 2023-10-18T12:40:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2023-10-18T12:40:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2023-10-18T12:40:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 90983, 2023-10-18T12:40:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1401593952, 2023-10-18T12:40:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:33.473 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-10-18T12:40:33.474 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:32 smithi070 ceph-mon[98639]: pgmap v766: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.7 MiB/s wr, 34 op/s 2023-10-18T12:40:33.755 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-10-18T12:40:33.756 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=14 2023-10-18T12:40:33.756 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=8 2023-10-18T12:40:33.757 DEBUG:tasks.check_counter:Getting stats from e 2023-10-18T12:40:33.757 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-18T12:40:34.314 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:40:35.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:34 smithi152 ceph-mon[134767]: pgmap v767: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2023-10-18T12:40:35.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:34 smithi186 ceph-mon[135253]: pgmap v767: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2023-10-18T12:40:35.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:34 smithi070 ceph-mon[98639]: pgmap v767: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2023-10-18T12:40:35.516 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-10-18T12:40:35.517 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:35.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:35.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:35.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:35.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:35.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:35.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:35.519 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.001232479, 2023-10-18T12:40:35.519 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:35.519 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000300445, 2023-10-18T12:40:35.519 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:35.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:35.520 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:35.520 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:35.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:35.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.521 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:35.522 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:35.522 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:35.522 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:35.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.523 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:35.523 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:35.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.524 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:35.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 6, 2023-10-18T12:40:35.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 208, 2023-10-18T12:40:35.525 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 3264, 2023-10-18T12:40:35.525 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 190016, 2023-10-18T12:40:35.525 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:35.526 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:35.526 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.009833287, 2023-10-18T12:40:35.526 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.006576733, 2023-10-18T12:40:35.527 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.001496588, 2023-10-18T12:40:35.527 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:35.527 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:35.527 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 207, 2023-10-18T12:40:35.528 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.014802132, 2023-10-18T12:40:35.528 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000071507 2023-10-18T12:40:35.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.529 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:35.529 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:35.529 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:35.529 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:35.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.530 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-10-18T12:40:35.530 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 190016 2023-10-18T12:40:35.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.531 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:35.531 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 276, 2023-10-18T12:40:35.531 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 271, 2023-10-18T12:40:35.532 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 760944, 2023-10-18T12:40:35.532 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 162544, 2023-10-18T12:40:35.532 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:35.532 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:35.533 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.036489692, 2023-10-18T12:40:35.533 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.012697941, 2023-10-18T12:40:35.533 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.018545279, 2023-10-18T12:40:35.533 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.006322613, 2023-10-18T12:40:35.534 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:35.534 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 270, 2023-10-18T12:40:35.534 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.009976925, 2023-10-18T12:40:35.535 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000036951 2023-10-18T12:40:35.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.535 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:35.535 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:35.536 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:35.536 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:35.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.536 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 760944, 2023-10-18T12:40:35.537 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 162544 2023-10-18T12:40:35.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.537 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2023-10-18T12:40:35.538 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 0, 2023-10-18T12:40:35.538 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2023-10-18T12:40:35.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.539 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2023-10-18T12:40:35.539 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:35.539 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:35.539 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:35.540 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:35.540 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:35.540 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:35.540 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:35.541 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:35.541 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:35.541 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:35.541 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:35.542 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:35.542 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:35.542 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:35.543 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:35.543 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:35.543 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:35.543 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:35.544 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:35.544 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:35.544 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:35.544 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:35.545 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:35.545 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:35.545 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:35.546 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:35.546 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:35.546 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:35.546 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:35.547 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:35.547 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:35.547 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2023-10-18T12:40:35.547 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:35.548 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:35.548 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:35.548 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:35.549 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 147920, 2023-10-18T12:40:35.549 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 27, 2023-10-18T12:40:35.549 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:35.549 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:35.550 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2023-10-18T12:40:35.550 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2023-10-18T12:40:35.550 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:35.550 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:35.551 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:35.551 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:35.551 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:35.552 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 0, 2023-10-18T12:40:35.552 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:35.552 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:35.552 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:35.553 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2023-10-18T12:40:35.553 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:35.553 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 2, 2023-10-18T12:40:35.554 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:35.554 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2023-10-18T12:40:35.554 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:35.554 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2023-10-18T12:40:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2023-10-18T12:40:35.556 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:35.556 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 278, 2023-10-18T12:40:35.556 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 721952, 2023-10-18T12:40:35.557 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:35.557 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 278, 2023-10-18T12:40:35.557 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2023-10-18T12:40:35.557 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2023-10-18T12:40:35.558 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 278, 2023-10-18T12:40:35.558 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 721952, 2023-10-18T12:40:35.558 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:35.558 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:35.559 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:35.559 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:35.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:35.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:35.560 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-10-18T12:40:35.959 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-10-18T12:40:35.960 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-10-18T12:40:35.960 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-10-18T12:40:35.960 DEBUG:tasks.check_counter:Getting stats from h 2023-10-18T12:40:35.961 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-18T12:40:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:40:37.092 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:36 smithi152 ceph-mon[134767]: pgmap v768: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2023-10-18T12:40:37.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:36 smithi186 ceph-mon[135253]: pgmap v768: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2023-10-18T12:40:37.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:36 smithi070 ceph-mon[98639]: pgmap v768: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2023-10-18T12:40:37.864 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-10-18T12:40:37.865 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:37.865 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:37.865 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:37.865 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:37.866 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:37.866 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:37.866 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:37.867 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.001317011, 2023-10-18T12:40:37.867 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:37.867 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000332408, 2023-10-18T12:40:37.868 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:37.868 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:37.868 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:37.868 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:37.869 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:37.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.869 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:37.870 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:37.870 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:37.870 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:37.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.871 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:37.871 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:37.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.872 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:37.872 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 279, 2023-10-18T12:40:37.872 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 272, 2023-10-18T12:40:37.873 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 763616, 2023-10-18T12:40:37.873 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 162368, 2023-10-18T12:40:37.873 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:37.873 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:37.874 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.037642112, 2023-10-18T12:40:37.874 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.013315352, 2023-10-18T12:40:37.875 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.019227446, 2023-10-18T12:40:37.875 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.006317505, 2023-10-18T12:40:37.875 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:37.875 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 272, 2023-10-18T12:40:37.876 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.017021510, 2023-10-18T12:40:37.876 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000062579 2023-10-18T12:40:37.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.877 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:37.877 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:37.877 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:37.877 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:37.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.878 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 763616, 2023-10-18T12:40:37.878 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 162368 2023-10-18T12:40:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2, 2023-10-18T12:40:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 205, 2023-10-18T12:40:37.880 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 320, 2023-10-18T12:40:37.880 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 189392, 2023-10-18T12:40:37.880 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:37.881 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:37.881 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.009255091, 2023-10-18T12:40:37.881 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.006637218, 2023-10-18T12:40:37.881 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000904067, 2023-10-18T12:40:37.882 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:37.882 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:37.882 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 203, 2023-10-18T12:40:37.883 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.011041828, 2023-10-18T12:40:37.883 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000054393 2023-10-18T12:40:37.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.883 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:37.884 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:37.884 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:37.884 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:37.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.885 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T12:40:37.885 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 189392 2023-10-18T12:40:37.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.886 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2023-10-18T12:40:37.886 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 0, 2023-10-18T12:40:37.887 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2023-10-18T12:40:37.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.887 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2023-10-18T12:40:37.887 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:37.888 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:37.888 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:37.888 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:37.889 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:37.889 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:37.889 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:37.889 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:37.890 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:37.890 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:37.890 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:37.891 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:37.891 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:37.891 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:37.891 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:37.892 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:37.892 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:37.892 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:37.893 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:37.893 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:37.893 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:37.893 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:37.894 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:37.894 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:37.895 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:37.895 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:37.895 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:37.896 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:37.896 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:37.897 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:37.897 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:37.897 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2023-10-18T12:40:37.898 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:37.898 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:37.898 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:37.899 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:37.899 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 147938, 2023-10-18T12:40:37.900 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 28, 2023-10-18T12:40:37.900 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:37.901 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:37.901 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2023-10-18T12:40:37.901 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2023-10-18T12:40:37.902 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:37.902 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:37.903 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:37.903 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:37.903 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:37.904 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 0, 2023-10-18T12:40:37.904 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:37.905 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:37.905 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:37.905 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2023-10-18T12:40:37.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:37.906 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 2, 2023-10-18T12:40:37.906 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:37.907 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2023-10-18T12:40:37.907 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:37.907 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2023-10-18T12:40:37.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:37.908 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:37.908 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:37.908 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2023-10-18T12:40:37.909 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:37.909 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 277, 2023-10-18T12:40:37.909 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 721822, 2023-10-18T12:40:37.909 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:37.910 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 277, 2023-10-18T12:40:37.910 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2023-10-18T12:40:37.910 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2023-10-18T12:40:37.911 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 277, 2023-10-18T12:40:37.911 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 721822, 2023-10-18T12:40:37.911 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:37.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:37.912 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:37.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:37.913 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:37.913 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:37.913 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-10-18T12:40:38.274 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2023-10-18T12:40:38.275 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2023-10-18T12:40:38.275 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2023-10-18T12:40:38.276 DEBUG:tasks.check_counter:Getting stats from k 2023-10-18T12:40:38.277 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-18T12:40:38.833 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T12:40:39.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:38 smithi186 ceph-mon[135253]: pgmap v769: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2023-10-18T12:40:39.245 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:38 smithi152 ceph-mon[134767]: pgmap v769: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2023-10-18T12:40:39.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:38 smithi070 ceph-mon[98639]: pgmap v769: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.7 MiB/s wr, 37 op/s 2023-10-18T12:40:40.132 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-10-18T12:40:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 4, 2023-10-18T12:40:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 4, 2023-10-18T12:40:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 2944, 2023-10-18T12:40:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 448, 2023-10-18T12:40:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.001664214, 2023-10-18T12:40:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000042835, 2023-10-18T12:40:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000762719, 2023-10-18T12:40:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3, 2023-10-18T12:40:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000848408, 2023-10-18T12:40:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000282802 2023-10-18T12:40:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-18T12:40:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T12:40:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 279, 2023-10-18T12:40:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 274, 2023-10-18T12:40:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 761760, 2023-10-18T12:40:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 164416, 2023-10-18T12:40:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.037380943, 2023-10-18T12:40:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.013114384, 2023-10-18T12:40:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.019666013, 2023-10-18T12:40:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.006187580, 2023-10-18T12:40:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 273, 2023-10-18T12:40:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.011113315, 2023-10-18T12:40:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000040708 2023-10-18T12:40:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 761760, 2023-10-18T12:40:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 164416 2023-10-18T12:40:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2, 2023-10-18T12:40:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 209, 2023-10-18T12:40:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 320, 2023-10-18T12:40:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 190096, 2023-10-18T12:40:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.009775705, 2023-10-18T12:40:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.006631845, 2023-10-18T12:40:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.001186857, 2023-10-18T12:40:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 207, 2023-10-18T12:40:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.010518313, 2023-10-18T12:40:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000050813 2023-10-18T12:40:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T12:40:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 190096 2023-10-18T12:40:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2023-10-18T12:40:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 0, 2023-10-18T12:40:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2023-10-18T12:40:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2023-10-18T12:40:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:40.155 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:40.155 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:40.155 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:40.155 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:40.156 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:40.156 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:40.156 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:40.158 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:40.158 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:40.158 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:40.158 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:40.159 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:40.159 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:40.159 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:40.160 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:40.160 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:40.160 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:40.160 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:40.161 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:40.161 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:40.161 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:40.161 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:40.162 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2023-10-18T12:40:40.162 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:40.162 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:40.162 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:40.163 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:40.163 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 147937, 2023-10-18T12:40:40.163 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 28, 2023-10-18T12:40:40.163 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:40.164 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:40.164 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2023-10-18T12:40:40.164 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2023-10-18T12:40:40.164 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:40.165 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:40.165 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:40.165 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:40.165 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:40.166 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 0, 2023-10-18T12:40:40.166 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:40.166 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:40.167 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:40.167 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2023-10-18T12:40:40.167 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:40.167 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 2, 2023-10-18T12:40:40.168 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:40.168 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2023-10-18T12:40:40.168 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:40.169 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2023-10-18T12:40:40.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-10-18T12:40:40.169 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:40.169 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2023-10-18T12:40:40.170 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2023-10-18T12:40:40.170 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2023-10-18T12:40:40.170 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 281, 2023-10-18T12:40:40.171 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 722342, 2023-10-18T12:40:40.171 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:40.171 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 281, 2023-10-18T12:40:40.171 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2023-10-18T12:40:40.172 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2023-10-18T12:40:40.172 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 281, 2023-10-18T12:40:40.172 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 722342, 2023-10-18T12:40:40.172 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2023-10-18T12:40:40.173 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2023-10-18T12:40:40.173 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2023-10-18T12:40:40.173 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2023-10-18T12:40:40.174 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:40.174 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-10-18T12:40:40.174 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-10-18T12:40:40.548 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:40 smithi152 ceph-mon[134767]: pgmap v770: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 29 op/s 2023-10-18T12:40:40.590 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-10-18T12:40:40.590 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-10-18T12:40:40.591 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-10-18T12:40:40.591 DEBUG:tasks.check_counter:Getting stats from c 2023-10-18T12:40:40.591 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-18T12:40:40.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:40 smithi186 ceph-mon[135253]: pgmap v770: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 29 op/s 2023-10-18T12:40:40.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:40 smithi070 ceph-mon[98639]: pgmap v770: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 29 op/s 2023-10-18T12:40:41.173 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:40:42.417 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-18T12:40:42.418 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:42.418 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 281, 2023-10-18T12:40:42.419 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 274, 2023-10-18T12:40:42.419 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 764160, 2023-10-18T12:40:42.419 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 163616, 2023-10-18T12:40:42.420 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:42.420 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:42.420 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.041742172, 2023-10-18T12:40:42.420 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014339799, 2023-10-18T12:40:42.421 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.020624659, 2023-10-18T12:40:42.421 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006930682, 2023-10-18T12:40:42.421 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:42.421 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 274, 2023-10-18T12:40:42.422 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.016536247, 2023-10-18T12:40:42.422 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000060351 2023-10-18T12:40:42.422 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.423 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:42.423 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:42.423 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:42.423 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:42.424 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.424 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 764160, 2023-10-18T12:40:42.424 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 163616 2023-10-18T12:40:42.425 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.425 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:42.425 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2023-10-18T12:40:42.425 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 207, 2023-10-18T12:40:42.426 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 320, 2023-10-18T12:40:42.426 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 189744, 2023-10-18T12:40:42.426 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:42.427 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:42.427 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.011512881, 2023-10-18T12:40:42.427 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008075145, 2023-10-18T12:40:42.427 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001120510, 2023-10-18T12:40:42.428 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:42.428 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:42.428 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 205, 2023-10-18T12:40:42.428 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.011227073, 2023-10-18T12:40:42.429 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000054766 2023-10-18T12:40:42.429 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.429 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:42.429 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:42.430 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:42.430 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:42.430 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T12:40:42.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 189744 2023-10-18T12:40:42.431 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.432 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:42.432 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:42.432 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:42.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:42.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:42.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:42.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:42.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000403447, 2023-10-18T12:40:42.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:42.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000324957, 2023-10-18T12:40:42.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:42.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:42.435 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:42.435 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:42.436 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:42.436 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.436 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:42.436 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:42.437 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:42.437 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:42.437 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.437 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:42.438 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:42.438 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.438 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-18T12:40:42.439 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-10-18T12:40:42.439 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-18T12:40:42.439 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.439 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-18T12:40:42.440 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:42.440 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:42.440 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:42.440 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:42.441 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:42.441 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:42.441 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:42.441 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:42.442 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:42.442 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:42.442 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:42.443 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:42.443 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:42.444 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:42.444 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:42.444 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:42.445 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:42.445 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:42.445 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:42.446 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:42.446 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:42.446 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:42.447 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:42.447 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:42.447 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:42.448 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:42.448 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:42.448 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:42.448 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:42.449 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:42.449 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:42.449 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-18T12:40:42.450 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:42.450 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:42.450 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:42.451 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:42.451 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148000, 2023-10-18T12:40:42.451 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 29, 2023-10-18T12:40:42.451 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:42.452 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:42.452 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-18T12:40:42.453 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-18T12:40:42.453 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:42.454 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:42.454 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:42.455 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:42.455 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:42.456 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-10-18T12:40:42.456 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:42.456 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:42.457 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:42.457 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-18T12:40:42.457 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:42.457 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-10-18T12:40:42.458 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:42.458 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-18T12:40:42.458 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:42.459 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-18T12:40:42.459 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:42.459 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:42.459 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-18T12:40:42.460 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-18T12:40:42.460 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-18T12:40:42.461 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 279, 2023-10-18T12:40:42.461 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 722082, 2023-10-18T12:40:42.462 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:42.462 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 279, 2023-10-18T12:40:42.463 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-18T12:40:42.463 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-18T12:40:42.464 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 279, 2023-10-18T12:40:42.464 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 722082, 2023-10-18T12:40:42.464 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-18T12:40:42.465 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:42.465 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:42.465 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:42.466 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:42.466 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:42.467 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-18T12:40:42.830 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2023-10-18T12:40:42.831 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2023-10-18T12:40:42.831 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2023-10-18T12:40:42.831 DEBUG:tasks.check_counter:Getting stats from f 2023-10-18T12:40:42.832 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-18T12:40:43.219 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:42 smithi186 ceph-mon[135253]: pgmap v771: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-10-18T12:40:43.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:42 smithi070 ceph-mon[98639]: pgmap v771: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-10-18T12:40:43.392 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:40:43.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:42 smithi152 ceph-mon[134767]: pgmap v771: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-10-18T12:40:44.614 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-18T12:40:44.615 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:44.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 278, 2023-10-18T12:40:44.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 273, 2023-10-18T12:40:44.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 761488, 2023-10-18T12:40:44.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 163792, 2023-10-18T12:40:44.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:44.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:44.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.042346523, 2023-10-18T12:40:44.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014842013, 2023-10-18T12:40:44.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.020419293, 2023-10-18T12:40:44.618 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006992010, 2023-10-18T12:40:44.618 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:44.618 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 272, 2023-10-18T12:40:44.619 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012966434, 2023-10-18T12:40:44.619 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000047670 2023-10-18T12:40:44.619 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:44.620 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:44.620 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:44.620 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:44.620 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.621 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 761488, 2023-10-18T12:40:44.621 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 163792 2023-10-18T12:40:44.621 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.621 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:44.622 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 6, 2023-10-18T12:40:44.622 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 210, 2023-10-18T12:40:44.622 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3264, 2023-10-18T12:40:44.622 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 190368, 2023-10-18T12:40:44.623 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:44.623 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:44.623 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.012152840, 2023-10-18T12:40:44.623 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008410768, 2023-10-18T12:40:44.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001562064, 2023-10-18T12:40:44.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:44.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:44.624 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 209, 2023-10-18T12:40:44.625 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.021190817, 2023-10-18T12:40:44.625 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000101391 2023-10-18T12:40:44.625 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.625 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:44.626 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:44.626 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:44.626 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:44.627 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-10-18T12:40:44.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 190368 2023-10-18T12:40:44.627 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.628 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:44.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:44.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:44.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:44.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:44.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:44.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:44.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000438295, 2023-10-18T12:40:44.630 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:44.630 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000369206, 2023-10-18T12:40:44.630 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:44.630 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:44.631 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:44.631 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:44.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:44.638 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.638 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:44.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:44.639 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:44.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:44.639 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.640 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:44.640 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:44.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.640 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-18T12:40:44.641 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-10-18T12:40:44.641 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-18T12:40:44.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.642 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-18T12:40:44.642 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:44.642 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:44.643 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:44.643 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:44.643 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:44.644 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:44.644 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:44.644 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:44.645 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:44.645 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:44.645 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:44.646 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:44.646 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:44.646 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:44.646 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:44.647 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:44.647 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:44.647 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:44.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:44.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:44.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:44.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:44.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:44.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:44.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:44.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:44.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:44.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:44.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:44.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:44.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:44.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-18T12:40:44.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:44.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:44.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:44.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:44.653 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 147920, 2023-10-18T12:40:44.653 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 27, 2023-10-18T12:40:44.654 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:44.654 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:44.654 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-18T12:40:44.654 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-18T12:40:44.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:44.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:44.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:44.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:44.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:44.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-10-18T12:40:44.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:44.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:44.657 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:44.657 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-18T12:40:44.657 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:44.658 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-10-18T12:40:44.658 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:44.658 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-18T12:40:44.658 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:44.659 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-18T12:40:44.659 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:44.659 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:44.659 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-18T12:40:44.660 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-18T12:40:44.660 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-18T12:40:44.660 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 280, 2023-10-18T12:40:44.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 722212, 2023-10-18T12:40:44.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:44.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 280, 2023-10-18T12:40:44.661 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-18T12:40:44.662 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-18T12:40:44.662 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 280, 2023-10-18T12:40:44.662 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 722212, 2023-10-18T12:40:44.663 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-18T12:40:44.663 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:44.663 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:44.663 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:44.664 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:44.664 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:44.664 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-18T12:40:45.045 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:44 smithi186 ceph-mon[135253]: pgmap v772: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 4 op/s 2023-10-18T12:40:45.047 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2023-10-18T12:40:45.047 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2023-10-18T12:40:45.048 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2023-10-18T12:40:45.048 DEBUG:tasks.check_counter:Getting stats from i 2023-10-18T12:40:45.049 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-18T12:40:45.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:44 smithi070 ceph-mon[98639]: pgmap v772: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 4 op/s 2023-10-18T12:40:45.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:44 smithi152 ceph-mon[134767]: pgmap v772: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 4 op/s 2023-10-18T12:40:45.605 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:40:46.808 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-18T12:40:46.809 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:46.809 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:46.810 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:46.810 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:46.810 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:46.811 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:46.811 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:46.811 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001088722, 2023-10-18T12:40:46.811 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:46.812 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000089793, 2023-10-18T12:40:46.812 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:46.812 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:46.812 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:46.813 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:46.813 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:46.813 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.813 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:46.814 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:46.814 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:46.814 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:46.815 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.815 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:46.815 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:46.815 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.816 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:46.816 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 6, 2023-10-18T12:40:46.816 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 209, 2023-10-18T12:40:46.816 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3264, 2023-10-18T12:40:46.817 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 190192, 2023-10-18T12:40:46.817 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:46.817 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:46.817 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010963250, 2023-10-18T12:40:46.818 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.007568145, 2023-10-18T12:40:46.818 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001364363, 2023-10-18T12:40:46.818 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:46.818 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:46.819 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 208, 2023-10-18T12:40:46.819 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.011780549, 2023-10-18T12:40:46.819 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000056637 2023-10-18T12:40:46.819 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.820 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:46.820 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:46.820 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:46.820 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:46.821 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.821 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-10-18T12:40:46.821 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 190192 2023-10-18T12:40:46.822 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.822 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:46.822 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 277, 2023-10-18T12:40:46.822 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 272, 2023-10-18T12:40:46.823 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 761216, 2023-10-18T12:40:46.823 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 163168, 2023-10-18T12:40:46.823 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:46.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:46.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.039920691, 2023-10-18T12:40:46.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.015086836, 2023-10-18T12:40:46.824 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.017548728, 2023-10-18T12:40:46.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006625829, 2023-10-18T12:40:46.825 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:46.825 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 271, 2023-10-18T12:40:46.825 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.016856000, 2023-10-18T12:40:46.826 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000062199 2023-10-18T12:40:46.826 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.826 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:46.826 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:46.827 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:46.827 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:46.827 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.827 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 761216, 2023-10-18T12:40:46.828 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 163168 2023-10-18T12:40:46.828 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.828 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-18T12:40:46.828 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-10-18T12:40:46.829 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-18T12:40:46.829 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.829 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-18T12:40:46.830 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:46.830 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:46.830 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:46.830 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:46.831 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:46.831 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:46.831 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:46.831 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:46.832 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:46.832 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:46.832 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:46.833 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:46.833 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:46.833 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:46.833 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:46.834 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:46.834 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:46.834 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:46.834 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:46.835 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:46.835 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:46.835 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:46.835 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:46.836 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:46.836 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:46.836 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:46.836 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:46.837 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:46.837 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:46.837 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:46.837 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:46.838 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-18T12:40:46.838 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:46.838 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:46.839 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:46.839 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:46.839 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 147898, 2023-10-18T12:40:46.839 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 27, 2023-10-18T12:40:46.840 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:46.840 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:46.840 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-18T12:40:46.841 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-18T12:40:46.841 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:46.841 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:46.841 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:46.842 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:46.842 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:46.842 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-10-18T12:40:46.843 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:46.843 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:46.843 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:46.843 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-18T12:40:46.844 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:46.844 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-10-18T12:40:46.844 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:46.844 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-18T12:40:46.845 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:46.845 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-18T12:40:46.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:46.845 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:46.846 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-18T12:40:46.846 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-18T12:40:46.846 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-18T12:40:46.847 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 279, 2023-10-18T12:40:46.847 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 722082, 2023-10-18T12:40:46.847 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:46.847 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 279, 2023-10-18T12:40:46.848 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-18T12:40:46.848 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-18T12:40:46.848 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 279, 2023-10-18T12:40:46.848 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 722082, 2023-10-18T12:40:46.849 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-18T12:40:46.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:46.849 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:46.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:46.850 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:46.850 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:46.850 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-18T12:40:47.149 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2023-10-18T12:40:47.149 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2023-10-18T12:40:47.150 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2023-10-18T12:40:47.151 DEBUG:tasks.check_counter:Getting stats from l 2023-10-18T12:40:47.151 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-18T12:40:47.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:46 smithi186 ceph-mon[135253]: pgmap v773: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-10-18T12:40:47.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:46 smithi070 ceph-mon[98639]: pgmap v773: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-10-18T12:40:47.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:46 smithi152 ceph-mon[134767]: pgmap v773: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-10-18T12:40:47.710 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T12:40:48.948 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-18T12:40:48.949 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T12:40:48.949 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 282, 2023-10-18T12:40:48.949 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 277, 2023-10-18T12:40:48.949 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 762576, 2023-10-18T12:40:48.950 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 166288, 2023-10-18T12:40:48.950 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:48.950 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-10-18T12:40:48.951 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.043221184, 2023-10-18T12:40:48.951 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.015426631, 2023-10-18T12:40:48.951 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.020763872, 2023-10-18T12:40:48.951 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006954582, 2023-10-18T12:40:48.952 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:48.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 276, 2023-10-18T12:40:48.952 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012877698, 2023-10-18T12:40:48.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000046658 2023-10-18T12:40:48.953 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.953 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:48.953 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:48.954 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:48.954 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:48.955 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.955 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 762576, 2023-10-18T12:40:48.955 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 166288 2023-10-18T12:40:48.955 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.956 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T12:40:48.956 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 6, 2023-10-18T12:40:48.956 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 213, 2023-10-18T12:40:48.956 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3264, 2023-10-18T12:40:48.957 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 190896, 2023-10-18T12:40:48.957 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-18T12:40:48.957 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-18T12:40:48.957 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.012063452, 2023-10-18T12:40:48.958 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008522412, 2023-10-18T12:40:48.958 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001210121, 2023-10-18T12:40:48.958 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:48.959 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:48.959 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 212, 2023-10-18T12:40:48.959 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.013897808, 2023-10-18T12:40:48.960 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000065555 2023-10-18T12:40:48.960 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.960 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:48.960 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:48.961 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:48.961 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:48.961 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.961 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-10-18T12:40:48.962 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 190896 2023-10-18T12:40:48.962 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.962 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T12:40:48.963 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2023-10-18T12:40:48.963 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2023-10-18T12:40:48.963 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2023-10-18T12:40:48.963 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2023-10-18T12:40:48.964 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-10-18T12:40:48.964 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-10-18T12:40:48.965 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000362463, 2023-10-18T12:40:48.965 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2023-10-18T12:40:48.965 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000294118, 2023-10-18T12:40:48.966 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T12:40:48.966 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T12:40:48.966 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:48.967 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:48.967 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:48.967 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.967 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-18T12:40:48.968 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:48.968 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:48.968 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:48.968 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.969 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T12:40:48.969 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T12:40:48.969 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.969 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-18T12:40:48.970 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-10-18T12:40:48.970 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-18T12:40:48.970 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.971 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-18T12:40:48.971 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-18T12:40:48.971 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-18T12:40:48.971 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T12:40:48.972 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-18T12:40:48.972 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T12:40:48.972 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-18T12:40:48.972 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T12:40:48.973 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T12:40:48.973 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T12:40:48.973 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T12:40:48.974 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T12:40:48.974 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-18T12:40:48.974 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T12:40:48.974 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T12:40:48.975 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-18T12:40:48.975 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-18T12:40:48.975 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-18T12:40:48.975 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-18T12:40:48.976 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T12:40:48.976 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T12:40:48.976 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T12:40:48.976 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T12:40:48.977 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T12:40:48.977 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-18T12:40:48.977 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-18T12:40:48.977 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-18T12:40:48.978 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T12:40:48.978 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T12:40:48.978 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-18T12:40:48.978 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-18T12:40:48.979 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-18T12:40:48.979 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-18T12:40:48.979 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T12:40:48.980 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-18T12:40:48.980 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T12:40:48.980 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-18T12:40:48.980 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148704, 2023-10-18T12:40:48.981 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 29, 2023-10-18T12:40:48.981 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-18T12:40:48.981 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-18T12:40:48.981 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-18T12:40:48.982 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-18T12:40:48.982 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-18T12:40:48.982 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-18T12:40:48.983 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-18T12:40:48.983 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-18T12:40:48.983 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-10-18T12:40:48.983 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-10-18T12:40:48.983 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T12:40:48.984 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-18T12:40:48.984 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-18T12:40:48.984 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-18T12:40:48.985 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-10-18T12:40:48.985 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-10-18T12:40:48.985 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-18T12:40:48.986 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-18T12:40:48.986 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-18T12:40:48.986 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-18T12:40:48.986 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-18T12:40:48.987 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T12:40:48.987 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-18T12:40:48.987 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-18T12:40:48.988 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-18T12:40:48.988 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 284, 2023-10-18T12:40:48.988 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 722732, 2023-10-18T12:40:48.989 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-18T12:40:48.989 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 284, 2023-10-18T12:40:48.989 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-18T12:40:48.989 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-18T12:40:48.990 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 284, 2023-10-18T12:40:48.990 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 722732, 2023-10-18T12:40:48.990 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-18T12:40:48.991 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-18T12:40:48.991 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-18T12:40:48.991 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-18T12:40:48.992 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:48.992 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-18T12:40:48.992 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-18T12:40:48.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:48 smithi186 ceph-mon[135253]: pgmap v774: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 3 op/s 2023-10-18T12:40:49.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:48 smithi070 ceph-mon[98639]: pgmap v774: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 3 op/s 2023-10-18T12:40:49.365 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-10-18T12:40:49.366 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-10-18T12:40:49.366 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-10-18T12:40:49.367 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-18T12:40:49.378 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-18T12:40:49.379 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:40:49.379 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:40:49.422 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:48 smithi152 ceph-mon[134767]: pgmap v774: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 3 op/s 2023-10-18T12:40:49.433 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-18T12:40:49.433 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:40:49.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:40:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2296682954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 msgr2=0x7f4cc410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2296682954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 0x7f4cc410d570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b0a0 tx=0x7f4cbc00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2296682954 shutdown_connections 2023-10-18T12:40:49.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2296682954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 0x7f4cc410d570 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2296682954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc41066b0 0x7f4cc4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2296682954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41006a0 0x7f4cc4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2296682954 >> 172.21.15.70:0/2296682954 conn(0x7f4cc40fc1d0 msgr2=0x7f4cc40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:49.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2296682954 shutdown_connections 2023-10-18T12:40:49.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.694+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2296682954 wait complete. 2023-10-18T12:40:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.695+0000 7f4cca1ff700 1 Processor -- start 2023-10-18T12:40:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.695+0000 7f4cca1ff700 1 -- start start 2023-10-18T12:40:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cca1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc41006a0 0x7f4cc419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cca1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 0x7f4cc419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cca1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 0x7f4cc4196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cca1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc41970e0 con 0x7f4cc4101060 2023-10-18T12:40:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cca1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4197250 con 0x7f4cc41006a0 2023-10-18T12:40:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cca1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4197410 con 0x7f4cc41066b0 2023-10-18T12:40:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 0x7f4cc4196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc41006a0 0x7f4cc419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 0x7f4cc4196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.70:33358/0 (socket says 172.21.15.70:33358) 2023-10-18T12:40:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 -- 172.21.15.70:0/2762797308 learned_addr learned my addr 172.21.15.70:0/2762797308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cbbfff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 0x7f4cc419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 -- 172.21.15.70:0/2762797308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc41006a0 msgr2=0x7f4cc419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc41006a0 0x7f4cc419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 -- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 msgr2=0x7f4cc419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 0x7f4cc419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.696+0000 7f4cc3fff700 1 -- 172.21.15.70:0/2762797308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc009d00 con 0x7f4cc41066b0 2023-10-18T12:40:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.697+0000 7f4cc3fff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 0x7f4cc4196910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b070 tx=0x7f4cbc008ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.697+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00e040 con 0x7f4cc41066b0 2023-10-18T12:40:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.697+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cbc01d470 con 0x7f4cc41066b0 2023-10-18T12:40:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.697+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc4197690 con 0x7f4cc41066b0 2023-10-18T12:40:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.697+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0052a0 con 0x7f4cc41066b0 2023-10-18T12:40:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.697+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc41a37f0 con 0x7f4cc41066b0 2023-10-18T12:40:49.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.698+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc4108ca0 con 0x7f4cc41066b0 2023-10-18T12:40:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.700+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4cbc01d5e0 con 0x7f4cc41066b0 2023-10-18T12:40:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.701+0000 7f4cc17fa700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4cb00a1670 0x7f4cb00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.701+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f4cbc0ce6e0 con 0x7f4cc41066b0 2023-10-18T12:40:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.701+0000 7f4cc37fe700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4cb00a1670 0x7f4cb00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.702+0000 7f4cc37fe700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4cb00a1670 0x7f4cb00a3b30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009be0 tx=0x7f4cb4009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.705+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4cbc0976d0 con 0x7f4cc41066b0 2023-10-18T12:40:49.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.910+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cc4197df0 con 0x7f4cc41066b0 2023-10-18T12:40:49.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.912+0000 7f4cc17fa700 1 -- 172.21.15.70:0/2762797308 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+25092 (secure 0 0 0) 0x7f4cbc09acf0 con 0x7f4cc41066b0 2023-10-18T12:40:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:49.915 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":109,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:40:14.381877+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:40:49.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4cb00a1670 msgr2=0x7f4cb00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:49.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4cb00a1670 0x7f4cb00a3b30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009be0 tx=0x7f4cb4009260 comp rx=0 tx=0).stop 2023-10-18T12:40:49.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 msgr2=0x7f4cc4196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:49.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 0x7f4cc4196910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b070 tx=0x7f4cbc008ff0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 shutdown_connections 2023-10-18T12:40:49.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc41066b0 0x7f4cc4196910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4cb00a1670 0x7f4cb00a3b30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc41006a0 0x7f4cc419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 --2- 172.21.15.70:0/2762797308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cc4101060 0x7f4cc419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:49.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 >> 172.21.15.70:0/2762797308 conn(0x7f4cc40fc1d0 msgr2=0x7f4cc40fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:49.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 shutdown_connections 2023-10-18T12:40:49.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:49.914+0000 7f4cca1ff700 1 -- 172.21.15.70:0/2762797308 wait complete. 2023-10-18T12:40:49.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T12:19:58.094929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:24:07.937110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:24:08.930078+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T12:24:12.620813+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T12:40:49.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:40:50.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 -- 172.21.15.70:0/4152757357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 msgr2=0x7f9398106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 --2- 172.21.15.70:0/4152757357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398106b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9388009990 tx=0x7f9388009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 -- 172.21.15.70:0/4152757357 shutdown_connections 2023-10-18T12:40:50.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 --2- 172.21.15.70:0/4152757357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93981010d0 0x7f939810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 --2- 172.21.15.70:0/4152757357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398106b00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 --2- 172.21.15.70:0/4152757357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 0x7f9398100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 -- 172.21.15.70:0/4152757357 >> 172.21.15.70:0/4152757357 conn(0x7f93980fc200 msgr2=0x7f93980fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:50.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 -- 172.21.15.70:0/4152757357 shutdown_connections 2023-10-18T12:40:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.141+0000 7f939e3ff700 1 -- 172.21.15.70:0/4152757357 wait complete. 2023-10-18T12:40:50.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 Processor -- start 2023-10-18T12:40:50.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 -- start start 2023-10-18T12:40:50.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 0x7f9398072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93981010d0 0x7f9398073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93981a4f10 con 0x7f9398106720 2023-10-18T12:40:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93981a5080 con 0x7f9398100710 2023-10-18T12:40:50.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.142+0000 7f939e3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93981a5240 con 0x7f93981010d0 2023-10-18T12:40:50.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f9397fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 0x7f9398072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33350/0 (socket says 172.21.15.70:33350) 2023-10-18T12:40:50.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 -- 172.21.15.70:0/1002796827 learned_addr learned my addr 172.21.15.70:0/1002796827 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:50.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f93977fe700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93981010d0 0x7f9398073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 -- 172.21.15.70:0/1002796827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93981010d0 msgr2=0x7f9398073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93981010d0 0x7f9398073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 -- 172.21.15.70:0/1002796827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 msgr2=0x7f9398072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 0x7f9398072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 -- 172.21.15.70:0/1002796827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880095f0 con 0x7f9398106720 2023-10-18T12:40:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f9397fff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 0x7f9398072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:40:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.143+0000 7f939c99c700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398073570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9390007ee0 tx=0x7f939000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.144+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390017070 con 0x7f9398106720 2023-10-18T12:40:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.144+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f939000f040 con 0x7f9398106720 2023-10-18T12:40:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.144+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93981a5520 con 0x7f9398106720 2023-10-18T12:40:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.144+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390005580 con 0x7f9398106720 2023-10-18T12:40:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.144+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93981a59e0 con 0x7f9398106720 2023-10-18T12:40:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.145+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9398108d10 con 0x7f9398106720 2023-10-18T12:40:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.147+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f939000db20 con 0x7f9398106720 2023-10-18T12:40:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.149+0000 7f93957fa700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93800a15f0 0x7f93800a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.149+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f93900cd230 con 0x7f9398106720 2023-10-18T12:40:50.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.149+0000 7f9397fff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93800a15f0 0x7f93800a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.150+0000 7f9397fff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93800a15f0 0x7f93800a3ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9388006010 tx=0x7f938800b540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:50.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.152+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9390096190 con 0x7f9398106720 2023-10-18T12:40:50.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.355+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9398073f80 con 0x7f9398106720 2023-10-18T12:40:50.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.355+0000 7f93957fa700 1 -- 172.21.15.70:0/1002796827 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93900997b0 con 0x7f9398106720 2023-10-18T12:40:50.359 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-18T12:40:50.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93800a15f0 msgr2=0x7f93800a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93800a15f0 0x7f93800a3ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9388006010 tx=0x7f938800b540 comp rx=0 tx=0).stop 2023-10-18T12:40:50.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 msgr2=0x7f9398073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398073570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9390007ee0 tx=0x7f939000d3b0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 shutdown_connections 2023-10-18T12:40:50.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9398106720 0x7f9398073570 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f93800a15f0 0x7f93800a3ab0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9398100710 0x7f9398072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 --2- 172.21.15.70:0/1002796827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93981010d0 0x7f9398073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 >> 172.21.15.70:0/1002796827 conn(0x7f93980fc200 msgr2=0x7f93980fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 shutdown_connections 2023-10-18T12:40:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.358+0000 7f939e3ff700 1 -- 172.21.15.70:0/1002796827 wait complete. 2023-10-18T12:40:50.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:40:50.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 -- 172.21.15.70:0/833733927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac072c20 msgr2=0x7f4aac073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/833733927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac072c20 0x7f4aac073000 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009990 tx=0x7f4a9c009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 -- 172.21.15.70:0/833733927 shutdown_connections 2023-10-18T12:40:50.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/833733927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aac10cf50 0x7f4aac10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/833733927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac072c20 0x7f4aac073000 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/833733927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac073540 0x7f4aac10ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.579+0000 7f4ab13ff700 1 -- 172.21.15.70:0/833733927 >> 172.21.15.70:0/833733927 conn(0x7f4aac0fc200 msgr2=0x7f4aac0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:50.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.580+0000 7f4ab13ff700 1 -- 172.21.15.70:0/833733927 shutdown_connections 2023-10-18T12:40:50.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.580+0000 7f4ab13ff700 1 -- 172.21.15.70:0/833733927 wait complete. 2023-10-18T12:40:50.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.580+0000 7f4ab13ff700 1 Processor -- start 2023-10-18T12:40:50.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 -- start start 2023-10-18T12:40:50.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aac072c20 0x7f4aac198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 0x7f4aac1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac10cf50 0x7f4aac19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac19dcb0 con 0x7f4aac073540 2023-10-18T12:40:50.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac19de20 con 0x7f4aac10cf50 2023-10-18T12:40:50.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4ab13ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac19dfe0 con 0x7f4aac072c20 2023-10-18T12:40:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4aaa7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 0x7f4aac1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4aaaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aac072c20 0x7f4aac198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4aaa7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 0x7f4aac1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33372/0 (socket says 172.21.15.70:33372) 2023-10-18T12:40:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4aaa7fc700 1 -- 172.21.15.70:0/2059554268 learned_addr learned my addr 172.21.15.70:0/2059554268 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:50.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.581+0000 7f4aab7fe700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac10cf50 0x7f4aac19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aaa7fc700 1 -- 172.21.15.70:0/2059554268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aac072c20 msgr2=0x7f4aac198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aaa7fc700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aac072c20 0x7f4aac198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aaa7fc700 1 -- 172.21.15.70:0/2059554268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac10cf50 msgr2=0x7f4aac19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aaa7fc700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac10cf50 0x7f4aac19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aaa7fc700 1 -- 172.21.15.70:0/2059554268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0095f0 con 0x7f4aac073540 2023-10-18T12:40:50.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aab7fe700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac10cf50 0x7f4aac19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:40:50.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4aaa7fc700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 0x7f4aac1992d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000e9a0 tx=0x7f4aa000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:50.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0009790 con 0x7f4aac073540 2023-10-18T12:40:50.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4aa000cbd0 con 0x7f4aac073540 2023-10-18T12:40:50.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0018680 con 0x7f4aac073540 2023-10-18T12:40:50.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac19e2c0 con 0x7f4aac073540 2023-10-18T12:40:50.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.582+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac1a5b00 con 0x7f4aac073540 2023-10-18T12:40:50.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.583+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac108160 con 0x7f4aac073540 2023-10-18T12:40:50.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.585+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4aa000cd40 con 0x7f4aac073540 2023-10-18T12:40:50.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.587+0000 7f4a93fff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4a940a1670 0x7f4a940a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:50.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.587+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f4aa0014070 con 0x7f4aac073540 2023-10-18T12:40:50.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.587+0000 7f4aaaffd700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4a940a1670 0x7f4a940a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:50.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.587+0000 7f4aaaffd700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4a940a1670 0x7f4a940a3b30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c006010 tx=0x7f4a9c00b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:50.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.588+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4aa0096960 con 0x7f4aac073540 2023-10-18T12:40:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:50 smithi152 ceph-mon[134767]: pgmap v775: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:50.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:50 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2762797308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:50.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:50 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1002796827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:40:50.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:50 smithi186 ceph-mon[135253]: pgmap v775: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:50 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2762797308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:50.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:50 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1002796827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:40:50.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.791+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4aac19a090 con 0x7f4aac073540 2023-10-18T12:40:50.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.792+0000 7f4a93fff700 1 -- 172.21.15.70:0/2059554268 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4aa0099f80 con 0x7f4aac073540 2023-10-18T12:40:50.795 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:40:50.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4a940a1670 msgr2=0x7f4a940a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4a940a1670 0x7f4a940a3b30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c006010 tx=0x7f4a9c00b260 comp rx=0 tx=0).stop 2023-10-18T12:40:50.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 msgr2=0x7f4aac1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:50.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 0x7f4aac1992d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000e9a0 tx=0x7f4aa000ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 shutdown_connections 2023-10-18T12:40:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4a940a1670 0x7f4a940a3b30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aac10cf50 0x7f4aac19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aac072c20 0x7f4aac198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 --2- 172.21.15.70:0/2059554268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4aac073540 0x7f4aac1992d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:50.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 >> 172.21.15.70:0/2059554268 conn(0x7f4aac0fc200 msgr2=0x7f4aac0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:50.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.794+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 shutdown_connections 2023-10-18T12:40:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:50.795+0000 7f4ab13ff700 1 -- 172.21.15.70:0/2059554268 wait complete. 2023-10-18T12:40:50.813 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:50 smithi070 ceph-mon[98639]: pgmap v775: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:50.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:50 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2762797308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:50.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:50 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1002796827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T12:40:50.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:40:51.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 -- 172.21.15.70:0/1205716864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b8106730 msgr2=0x7f79b8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/1205716864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b8106730 0x7f79b8106b10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009990 tx=0x7f79a8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 -- 172.21.15.70:0/1205716864 shutdown_connections 2023-10-18T12:40:51.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/1205716864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b81010e0 0x7f79b810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/1205716864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b8106730 0x7f79b8106b10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/1205716864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79b8100720 0x7f79b8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 -- 172.21.15.70:0/1205716864 >> 172.21.15.70:0/1205716864 conn(0x7f79b80fc1b0 msgr2=0x7f79b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 -- 172.21.15.70:0/1205716864 shutdown_connections 2023-10-18T12:40:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.017+0000 7f79bd9ff700 1 -- 172.21.15.70:0/1205716864 wait complete. 2023-10-18T12:40:51.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.018+0000 7f79bd9ff700 1 Processor -- start 2023-10-18T12:40:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 -- start start 2023-10-18T12:40:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79b8100720 0x7f79b819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b81010e0 0x7f79b819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 0x7f79b81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8197010 con 0x7f79b8106730 2023-10-18T12:40:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8197180 con 0x7f79b81010e0 2023-10-18T12:40:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79bd9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8197340 con 0x7f79b8100720 2023-10-18T12:40:51.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79b77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 0x7f79b81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79b77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 0x7f79b81969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33388/0 (socket says 172.21.15.70:33388) 2023-10-18T12:40:51.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79b77fe700 1 -- 172.21.15.70:0/3584170995 learned_addr learned my addr 172.21.15.70:0/3584170995 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:51.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79b6ffd700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79b8100720 0x7f79b819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.019+0000 7f79b67fc700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b81010e0 0x7f79b819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b77fe700 1 -- 172.21.15.70:0/3584170995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79b8100720 msgr2=0x7f79b819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b77fe700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79b8100720 0x7f79b819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b77fe700 1 -- 172.21.15.70:0/3584170995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b81010e0 msgr2=0x7f79b819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b77fe700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b81010e0 0x7f79b819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b77fe700 1 -- 172.21.15.70:0/3584170995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80095f0 con 0x7f79b8106730 2023-10-18T12:40:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b67fc700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b81010e0 0x7f79b819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:40:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79b77fe700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 0x7f79b81969b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f79b000ef30 tx=0x7f79b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0018070 con 0x7f79b8106730 2023-10-18T12:40:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b000f040 con 0x7f79b8106730 2023-10-18T12:40:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b81975c0 con 0x7f79b8106730 2023-10-18T12:40:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000cdb0 con 0x7f79b8106730 2023-10-18T12:40:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.020+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b81a3870 con 0x7f79b8106730 2023-10-18T12:40:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.021+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b8108d20 con 0x7f79b8106730 2023-10-18T12:40:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.023+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f79b0003a00 con 0x7f79b8106730 2023-10-18T12:40:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.025+0000 7f79bc9fd700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79a00a15f0 0x7f79a00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.025+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f79b00cd090 con 0x7f79b8106730 2023-10-18T12:40:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.025+0000 7f79b6ffd700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79a00a15f0 0x7f79a00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.026+0000 7f79b6ffd700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79a00a15f0 0x7f79a00a3ab0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006010 tx=0x7f79a800b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.028+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f79b0096080 con 0x7f79b8106730 2023-10-18T12:40:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.229+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79b8197e70 con 0x7f79b8106730 2023-10-18T12:40:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.230+0000 7f79bc9fd700 1 -- 172.21.15.70:0/3584170995 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79b00996a0 con 0x7f79b8106730 2023-10-18T12:40:51.232 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-18T12:40:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79a00a15f0 msgr2=0x7f79a00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79a00a15f0 0x7f79a00a3ab0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006010 tx=0x7f79a800b260 comp rx=0 tx=0).stop 2023-10-18T12:40:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 msgr2=0x7f79b81969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 0x7f79b81969b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f79b000ef30 tx=0x7f79b000c3a0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 shutdown_connections 2023-10-18T12:40:51.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8106730 0x7f79b81969b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79b8100720 0x7f79b819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f79a00a15f0 0x7f79a00a3ab0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 --2- 172.21.15.70:0/3584170995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79b81010e0 0x7f79b819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.232+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 >> 172.21.15.70:0/3584170995 conn(0x7f79b80fc1b0 msgr2=0x7f79b80fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.233+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 shutdown_connections 2023-10-18T12:40:51.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.233+0000 7f79bd9ff700 1 -- 172.21.15.70:0/3584170995 wait complete. 2023-10-18T12:40:51.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T12:40:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 -- 172.21.15.70:0/2259073046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c105b30 msgr2=0x7efd7c10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 --2- 172.21.15.70:0/2259073046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c105b30 0x7efd7c10d540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efd7400b0d0 tx=0x7efd7400b3e0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 -- 172.21.15.70:0/2259073046 shutdown_connections 2023-10-18T12:40:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 --2- 172.21.15.70:0/2259073046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c105b30 0x7efd7c10d540 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 --2- 172.21.15.70:0/2259073046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd7c067d40 0x7efd7c068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 --2- 172.21.15.70:0/2259073046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd7c105170 0x7efd7c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 -- 172.21.15.70:0/2259073046 >> 172.21.15.70:0/2259073046 conn(0x7efd7c0767f0 msgr2=0x7efd7c076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 -- 172.21.15.70:0/2259073046 shutdown_connections 2023-10-18T12:40:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.455+0000 7efd841ff700 1 -- 172.21.15.70:0/2259073046 wait complete. 2023-10-18T12:40:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.456+0000 7efd841ff700 1 Processor -- start 2023-10-18T12:40:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.456+0000 7efd841ff700 1 -- start start 2023-10-18T12:40:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd841ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 0x7efd7c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd841ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd7c105170 0x7efd7c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd841ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd7c105b30 0x7efd7c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd841ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c19de00 con 0x7efd7c067d40 2023-10-18T12:40:51.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd841ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c19df70 con 0x7efd7c105170 2023-10-18T12:40:51.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd841ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c19e130 con 0x7efd7c105b30 2023-10-18T12:40:51.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 0x7efd7c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 0x7efd7c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33398/0 (socket says 172.21.15.70:33398) 2023-10-18T12:40:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 -- 172.21.15.70:0/1064524543 learned_addr learned my addr 172.21.15.70:0/1064524543 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd8179a700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd7c105170 0x7efd7c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd8279c700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd7c105b30 0x7efd7c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 -- 172.21.15.70:0/1064524543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd7c105b30 msgr2=0x7efd7c19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd7c105b30 0x7efd7c19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 -- 172.21.15.70:0/1064524543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd7c105170 msgr2=0x7efd7c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd7c105170 0x7efd7c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.457+0000 7efd81f9b700 1 -- 172.21.15.70:0/1064524543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74009d00 con 0x7efd7c067d40 2023-10-18T12:40:51.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.458+0000 7efd81f9b700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 0x7efd7c198ee0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efd7800e9a0 tx=0x7efd7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.458+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78009790 con 0x7efd7c067d40 2023-10-18T12:40:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.458+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd7800cbd0 con 0x7efd7c067d40 2023-10-18T12:40:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.458+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78018680 con 0x7efd7c067d40 2023-10-18T12:40:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.458+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c19e410 con 0x7efd7c067d40 2023-10-18T12:40:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.458+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c1a5c50 con 0x7efd7c067d40 2023-10-18T12:40:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.459+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c101e40 con 0x7efd7c067d40 2023-10-18T12:40:51.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.461+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efd7800cd40 con 0x7efd7c067d40 2023-10-18T12:40:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.462+0000 7efd72ffd700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd680a1670 0x7efd680a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.462+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7efd78014070 con 0x7efd7c067d40 2023-10-18T12:40:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.463+0000 7efd8179a700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd680a1670 0x7efd680a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.463+0000 7efd8179a700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd680a1670 0x7efd680a3b30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efd7c19a430 tx=0x7efd6c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:51.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.466+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd78052100 con 0x7efd7c067d40 2023-10-18T12:40:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:51 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2059554268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:40:51.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:51 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3584170995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:40:51.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.668+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efd7c1106a0 con 0x7efd7c067d40 2023-10-18T12:40:51.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.669+0000 7efd72ffd700 1 -- 172.21.15.70:0/1064524543 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v109) v1 ==== 103+0+11 (secure 0 0 0) 0x7efd78096770 con 0x7efd7c067d40 2023-10-18T12:40:51.671 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-18T12:40:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:51 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2059554268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:40:51.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:51 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3584170995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:40:51.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd680a1670 msgr2=0x7efd680a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd680a1670 0x7efd680a3b30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efd7c19a430 tx=0x7efd6c009260 comp rx=0 tx=0).stop 2023-10-18T12:40:51.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 msgr2=0x7efd7c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 0x7efd7c198ee0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efd7800e9a0 tx=0x7efd7800ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 shutdown_connections 2023-10-18T12:40:51.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd7c105b30 0x7efd7c19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd7c067d40 0x7efd7c198ee0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7efd680a1670 0x7efd680a3b30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 --2- 172.21.15.70:0/1064524543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd7c105170 0x7efd7c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.671+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 >> 172.21.15.70:0/1064524543 conn(0x7efd7c0767f0 msgr2=0x7efd7c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:51.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.672+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 shutdown_connections 2023-10-18T12:40:51.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.672+0000 7efd841ff700 1 -- 172.21.15.70:0/1064524543 wait complete. 2023-10-18T12:40:51.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:40:51.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:51 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2059554268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T12:40:51.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:51 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3584170995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T12:40:51.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 -- 172.21.15.70:0/3661371585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77641006a0 msgr2=0x7f7764100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 --2- 172.21.15.70:0/3661371585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77641006a0 0x7f7764100b20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7754009990 tx=0x7f7754009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 -- 172.21.15.70:0/3661371585 shutdown_connections 2023-10-18T12:40:51.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 --2- 172.21.15.70:0/3661371585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7764101060 0x7f776410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 --2- 172.21.15.70:0/3661371585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77641066b0 0x7f7764106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 --2- 172.21.15.70:0/3661371585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77641006a0 0x7f7764100b20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 -- 172.21.15.70:0/3661371585 >> 172.21.15.70:0/3661371585 conn(0x7f77640fc190 msgr2=0x7f77640fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:51.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 -- 172.21.15.70:0/3661371585 shutdown_connections 2023-10-18T12:40:51.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.890+0000 7f776c3ff700 1 -- 172.21.15.70:0/3661371585 wait complete. 2023-10-18T12:40:51.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.892+0000 7f776c3ff700 1 Processor -- start 2023-10-18T12:40:51.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.892+0000 7f776c3ff700 1 -- start start 2023-10-18T12:40:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.892+0000 7f776c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77641006a0 0x7f776419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776c3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 0x7f776419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776c3ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77641066b0 0x7f7764196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776c3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7764196fd0 con 0x7f7764101060 2023-10-18T12:40:51.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776c3ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7764197140 con 0x7f77641066b0 2023-10-18T12:40:51.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7764197300 con 0x7f77641006a0 2023-10-18T12:40:51.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 0x7f776419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 0x7f776419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33418/0 (socket says 172.21.15.70:33418) 2023-10-18T12:40:51.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 -- 172.21.15.70:0/608300296 learned_addr learned my addr 172.21.15.70:0/608300296 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:51.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776a99c700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77641066b0 0x7f7764196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776a19b700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77641006a0 0x7f776419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 -- 172.21.15.70:0/608300296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77641006a0 msgr2=0x7f776419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77641006a0 0x7f776419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 -- 172.21.15.70:0/608300296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77641066b0 msgr2=0x7f7764196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:51.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77641066b0 0x7f7764196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:51.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 -- 172.21.15.70:0/608300296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540095f0 con 0x7f7764101060 2023-10-18T12:40:51.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776a99c700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77641066b0 0x7f7764196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:40:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.893+0000 7f776999a700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 0x7f776419cd80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7754000c00 tx=0x7f77540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.894+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775401d070 con 0x7f7764101060 2023-10-18T12:40:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.894+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7764197580 con 0x7f7764101060 2023-10-18T12:40:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.894+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7754022470 con 0x7f7764101060 2023-10-18T12:40:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.894+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754017650 con 0x7f7764101060 2023-10-18T12:40:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.894+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77641a3830 con 0x7f7764101060 2023-10-18T12:40:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.895+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7764108c30 con 0x7f7764101060 2023-10-18T12:40:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.897+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f77540225e0 con 0x7f7764101060 2023-10-18T12:40:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.898+0000 7f775b7fe700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77500a15f0 0x7f77500a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.898+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f77540ce8e0 con 0x7f7764101060 2023-10-18T12:40:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.899+0000 7f776a19b700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77500a15f0 0x7f77500a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.899+0000 7f776a19b700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77500a15f0 0x7f77500a3ab0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7760009bd0 tx=0x7f7760009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:51.902+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77540978d0 con 0x7f7764101060 2023-10-18T12:40:52.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.123+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7764101de0 con 0x7f7764101060 2023-10-18T12:40:52.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.124+0000 7f775b7fe700 1 -- 172.21.15.70:0/608300296 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v109) v1 ==== 81+0+884 (secure 0 0 0) 0x7f7754027020 con 0x7f7764101060 2023-10-18T12:40:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1564492170 2023-10-19T12:18:13.515343+0000 2023-10-18T12:40:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2836082256 2023-10-19T12:18:13.515343+0000 2023-10-18T12:40:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/392897085 2023-10-19T12:18:13.515343+0000 2023-10-18T12:40:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1990317867 2023-10-19T12:17:02.382050+0000 2023-10-18T12:40:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2942646472 2023-10-19T12:18:13.515343+0000 2023-10-18T12:40:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1990317867 2023-10-19T12:17:02.382050+0000 2023-10-18T12:40:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/771435827 2023-10-19T12:17:02.382050+0000 2023-10-18T12:40:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1853570886 2023-10-19T12:16:35.958063+0000 2023-10-18T12:40:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4154455091 2023-10-19T12:16:35.958063+0000 2023-10-18T12:40:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1853570886 2023-10-19T12:16:35.958063+0000 2023-10-18T12:40:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4120653611 2023-10-19T12:17:02.382050+0000 2023-10-18T12:40:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/678282316 2023-10-19T12:16:35.958063+0000 2023-10-18T12:40:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3661620656 2023-10-19T12:16:35.958063+0000 2023-10-18T12:40:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2836082256 2023-10-19T12:18:13.515343+0000 2023-10-18T12:40:52.131 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/477422870 2023-10-19T12:17:02.382050+0000 2023-10-18T12:40:52.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.126+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77500a15f0 msgr2=0x7f77500a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:52.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.126+0000 7f776c3ff700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77500a15f0 0x7f77500a3ab0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7760009bd0 tx=0x7f7760009260 comp rx=0 tx=0).stop 2023-10-18T12:40:52.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.126+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 msgr2=0x7f776419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:52.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.126+0000 7f776c3ff700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 0x7f776419cd80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7754000c00 tx=0x7f77540057f0 comp rx=0 tx=0).stop 2023-10-18T12:40:52.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.126+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 shutdown_connections 2023-10-18T12:40:52.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.126+0000 7f776c3ff700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f77500a15f0 0x7f77500a3ab0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:52.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.127+0000 7f776c3ff700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77641066b0 0x7f7764196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:52.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.127+0000 7f776c3ff700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77641006a0 0x7f776419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:52.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.127+0000 7f776c3ff700 1 --2- 172.21.15.70:0/608300296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7764101060 0x7f776419cd80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:52.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.127+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 >> 172.21.15.70:0/608300296 conn(0x7f77640fc190 msgr2=0x7f77640fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:52.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.127+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 shutdown_connections 2023-10-18T12:40:52.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:52.127+0000 7f776c3ff700 1 -- 172.21.15.70:0/608300296 wait complete. 2023-10-18T12:40:52.135 INFO:teuthology.orchestra.run.smithi070.stderr:listed 15 entries 2023-10-18T12:40:52.148 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T12:40:52.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:40:52.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T12:40:52.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:40:52.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:40:52.466 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T12:40:52.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:40:52.511 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-18T12:40:52.597 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T12:40:52.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:40:52.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:40:52.663 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:52 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1064524543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:40:52.664 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:52 smithi070 ceph-mon[98639]: pgmap v776: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:52.664 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:52 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/608300296' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:40:52.664 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:52 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:40:52.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:52 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1064524543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:40:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:52 smithi152 ceph-mon[134767]: pgmap v776: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:52 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/608300296' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:40:52.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:52 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:40:52.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:40:52.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-10-18T12:40:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:52 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1064524543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T12:40:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:52 smithi186 ceph-mon[135253]: pgmap v776: 129 pgs: 129 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:52.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:52 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/608300296' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T12:40:52.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:52 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T12:40:52.794 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-18T12:40:52.811 INFO:tasks.cephadm:Teardown begin 2023-10-18T12:40:52.811 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T12:40:52.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:40:53.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3632942983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1140734b0 msgr2=0x7fa114111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3632942983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1140734b0 0x7fa114111170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa104009990 tx=0x7fa104009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3632942983 shutdown_connections 2023-10-18T12:40:53.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3632942983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1141116b0 0x7fa114113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3632942983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa114072b90 0x7fa114072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3632942983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1140734b0 0x7fa114111170 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.020+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3632942983 >> 172.21.15.70:0/3632942983 conn(0x7fa1140fc170 msgr2=0x7fa1140fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:53.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.021+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3632942983 shutdown_connections 2023-10-18T12:40:53.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.021+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3632942983 wait complete. 2023-10-18T12:40:53.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.021+0000 7fa11b9ff700 1 Processor -- start 2023-10-18T12:40:53.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 -- start start 2023-10-18T12:40:53.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 0x7fa11419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1140734b0 0x7fa11419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1141116b0 0x7fa1141a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1141a20f0 con 0x7fa1140734b0 2023-10-18T12:40:53.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1141a2260 con 0x7fa114072b90 2023-10-18T12:40:53.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11b9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1141a2420 con 0x7fa1141116b0 2023-10-18T12:40:53.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11979b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 0x7fa11419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa118f9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1140734b0 0x7fa11419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11979b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 0x7fa11419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.70:35882/0 (socket says 172.21.15.70:35882) 2023-10-18T12:40:53.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa11979b700 1 -- 172.21.15.70:0/3149708587 learned_addr learned my addr 172.21.15.70:0/3149708587 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:53.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.022+0000 7fa119f9c700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1141116b0 0x7fa1141a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11979b700 1 -- 172.21.15.70:0/3149708587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1141116b0 msgr2=0x7fa1141a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11979b700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1141116b0 0x7fa1141a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11979b700 1 -- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1140734b0 msgr2=0x7fa11419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11979b700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1140734b0 0x7fa11419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11979b700 1 -- 172.21.15.70:0/3149708587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040095f0 con 0x7fa114072b90 2023-10-18T12:40:53.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11979b700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 0x7fa11419d1d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa11000d980 tx=0x7fa11000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:53.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110004220 con 0x7fa114072b90 2023-10-18T12:40:53.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa110007640 con 0x7fa114072b90 2023-10-18T12:40:53.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110005690 con 0x7fa114072b90 2023-10-18T12:40:53.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1141a2700 con 0x7fa114072b90 2023-10-18T12:40:53.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.023+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1141a9f40 con 0x7fa114072b90 2023-10-18T12:40:53.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.024+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1141a2920 con 0x7fa114072b90 2023-10-18T12:40:53.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.026+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa1100447c0 con 0x7fa114072b90 2023-10-18T12:40:53.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.027+0000 7fa10a7fc700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1000a1670 0x7fa1000a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.028+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa1100cd300 con 0x7fa114072b90 2023-10-18T12:40:53.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.028+0000 7fa118f9a700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1000a1670 0x7fa1000a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.028+0000 7fa118f9a700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1000a1670 0x7fa1000a3b30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa11419e720 tx=0x7fa10400b260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:53.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.030+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa110096260 con 0x7fa114072b90 2023-10-18T12:40:53.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.288+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa114103b90 con 0x7fa114072b90 2023-10-18T12:40:53.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.288+0000 7fa10a7fc700 1 -- 172.21.15.70:0/3149708587 <== mon.2 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fa110016020 con 0x7fa114072b90 2023-10-18T12:40:53.291 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:53.291 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:40:53.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1000a1670 msgr2=0x7fa1000a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1000a1670 0x7fa1000a3b30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa11419e720 tx=0x7fa10400b260 comp rx=0 tx=0).stop 2023-10-18T12:40:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 msgr2=0x7fa11419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 0x7fa11419d1d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa11000d980 tx=0x7fa11000dc90 comp rx=0 tx=0).stop 2023-10-18T12:40:53.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 shutdown_connections 2023-10-18T12:40:53.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1141116b0 0x7fa1141a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa1000a1670 0x7fa1000a3b30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa114072b90 0x7fa11419d1d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 --2- 172.21.15.70:0/3149708587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1140734b0 0x7fa11419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 >> 172.21.15.70:0/3149708587 conn(0x7fa1140fc170 msgr2=0x7fa1140fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:53.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 shutdown_connections 2023-10-18T12:40:53.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.291+0000 7fa11b9ff700 1 -- 172.21.15.70:0/3149708587 wait complete. 2023-10-18T12:40:53.297 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:40:53.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:40:53.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 -- 172.21.15.70:0/2215899649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6941066b0 msgr2=0x7fa694106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 --2- 172.21.15.70:0/2215899649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6941066b0 0x7fa694106a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa684009990 tx=0x7fa684009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 -- 172.21.15.70:0/2215899649 shutdown_connections 2023-10-18T12:40:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 --2- 172.21.15.70:0/2215899649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa694101060 0x7fa69410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 --2- 172.21.15.70:0/2215899649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6941066b0 0x7fa694106a90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 --2- 172.21.15.70:0/2215899649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6941006a0 0x7fa694100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 -- 172.21.15.70:0/2215899649 >> 172.21.15.70:0/2215899649 conn(0x7fa6940fc170 msgr2=0x7fa6940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 -- 172.21.15.70:0/2215899649 shutdown_connections 2023-10-18T12:40:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.511+0000 7fa6999ff700 1 -- 172.21.15.70:0/2215899649 wait complete. 2023-10-18T12:40:53.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.512+0000 7fa6999ff700 1 Processor -- start 2023-10-18T12:40:53.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.512+0000 7fa6999ff700 1 -- start start 2023-10-18T12:40:53.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6999ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 0x7fa694198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6999ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa694101060 0x7fa694199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6999ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6941066b0 0x7fa69419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6999ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419cb30 con 0x7fa6941006a0 2023-10-18T12:40:53.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6999ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419cca0 con 0x7fa6941066b0 2023-10-18T12:40:53.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6999ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419ce60 con 0x7fa694101060 2023-10-18T12:40:53.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 0x7fa694198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 0x7fa694198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33458/0 (socket says 172.21.15.70:33458) 2023-10-18T12:40:53.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 -- 172.21.15.70:0/1408816982 learned_addr learned my addr 172.21.15.70:0/1408816982 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:53.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6927fc700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa694101060 0x7fa694199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa6937fe700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6941066b0 0x7fa69419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 -- 172.21.15.70:0/1408816982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa694101060 msgr2=0x7fa694199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa694101060 0x7fa694199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 -- 172.21.15.70:0/1408816982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6941066b0 msgr2=0x7fa69419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6941066b0 0x7fa69419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.513+0000 7fa692ffd700 1 -- 172.21.15.70:0/1408816982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6840095f0 con 0x7fa6941006a0 2023-10-18T12:40:53.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.514+0000 7fa692ffd700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 0x7fa694198d00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa684005910 tx=0x7fa684004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:53.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.514+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68401d070 con 0x7fa6941006a0 2023-10-18T12:40:53.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.514+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69419d0e0 con 0x7fa6941006a0 2023-10-18T12:40:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.514+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa684022470 con 0x7fa6941006a0 2023-10-18T12:40:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.514+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6941a5aa0 con 0x7fa6941006a0 2023-10-18T12:40:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.514+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68402c640 con 0x7fa6941006a0 2023-10-18T12:40:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.515+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa694101de0 con 0x7fa6941006a0 2023-10-18T12:40:53.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.517+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa6840225e0 con 0x7fa6941006a0 2023-10-18T12:40:53.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.518+0000 7fa6989fd700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.518+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fa6840ce660 con 0x7fa6941006a0 2023-10-18T12:40:53.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.519+0000 7fa6927fc700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.519+0000 7fa6927fc700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa69419a210 tx=0x7fa688009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:53.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.522+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa68405d100 con 0x7fa6941006a0 2023-10-18T12:40:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:40:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:40:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:53 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:40:53.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:53 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/3149708587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:53.671 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:40:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:40:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:53 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:40:53.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:53 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/3149708587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:53.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T12:40:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T12:40:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:53 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' 2023-10-18T12:40:53.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:53 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/3149708587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:53.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.767+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa694067530 con 0x7fa6941006a0 2023-10-18T12:40:53.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.768+0000 7fa6989fd700 1 -- 172.21.15.70:0/1408816982 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11273 (secure 0 0 0) 0x7fa684097650 con 0x7fa6941006a0 2023-10-18T12:40:53.771 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14631,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/104240782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":104240782},{"type":"v1","addr":"172.21.15.70:6835","nonce":104240782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14643,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3255312208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3255312208},{"type":"v1","addr":"172.21.15.70:6837","nonce":3255312208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6841/3718886784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6840","nonce":3718886784},{"type":"v1","addr":"172.21.15.70:6841","nonce":3718886784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/339260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":339260253},{"type":"v1","addr":"172.21.15.186:6833","nonce":339260253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24629,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1678353915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1678353915},{"type":"v1","addr":"172.21.15.152:6833","nonce":1678353915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24665,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2963573440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2963573440},{"type":"v1","addr":"172.21.15.152:6835","nonce":2963573440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3580803773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3580803773},{"type":"v1","addr":"172.21.15.186:6835","nonce":3580803773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24685,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/569012458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":569012458},{"type":"v1","addr":"172.21.15.186:6837","nonce":569012458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1543432937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1543432937},{"type":"v1","addr":"172.21.15.152:6837","nonce":1543432937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24697,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2873838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2873838836},{"type":"v1","addr":"172.21.15.186:6839","nonce":2873838836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T12:24:09.485042+0000","modified":"2023-10-18T12:24:34.631959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24719":{"gid":24719,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6839/3607185387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":3607185387},{"type":"v1","addr":"172.21.15.70:6839","nonce":3607185387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1810263953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1810263953},{"type":"v1","addr":"172.21.15.152:6839","nonce":1810263953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T12:40:53.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa67c0a15f0 msgr2=0x7fa67c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa69419a210 tx=0x7fa688009260 comp rx=0 tx=0).stop 2023-10-18T12:40:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 msgr2=0x7fa694198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 0x7fa694198d00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa684005910 tx=0x7fa684004730 comp rx=0 tx=0).stop 2023-10-18T12:40:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 shutdown_connections 2023-10-18T12:40:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6941066b0 0x7fa69419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6941006a0 0x7fa694198d00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 --2- 172.21.15.70:0/1408816982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa694101060 0x7fa694199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 >> 172.21.15.70:0/1408816982 conn(0x7fa6940fc170 msgr2=0x7fa69410bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.770+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 shutdown_connections 2023-10-18T12:40:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.771+0000 7fa6999ff700 1 -- 172.21.15.70:0/1408816982 wait complete. 2023-10-18T12:40:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-18T12:40:53.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:40:53.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.987+0000 7f4b91fff700 1 -- 172.21.15.70:0/3937519856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c1006a0 msgr2=0x7f4b8c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.987+0000 7f4b91fff700 1 --2- 172.21.15.70:0/3937519856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c100b20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009990 tx=0x7f4b80009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 -- 172.21.15.70:0/3937519856 shutdown_connections 2023-10-18T12:40:53.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 --2- 172.21.15.70:0/3937519856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c101060 0x7f4b8c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 --2- 172.21.15.70:0/3937519856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c1066b0 0x7f4b8c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 --2- 172.21.15.70:0/3937519856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c100b20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 -- 172.21.15.70:0/3937519856 >> 172.21.15.70:0/3937519856 conn(0x7f4b8c0fc170 msgr2=0x7f4b8c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:53.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 -- 172.21.15.70:0/3937519856 shutdown_connections 2023-10-18T12:40:53.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.988+0000 7f4b91fff700 1 -- 172.21.15.70:0/3937519856 wait complete. 2023-10-18T12:40:53.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 Processor -- start 2023-10-18T12:40:53.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 -- start start 2023-10-18T12:40:53.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c101060 0x7f4b8c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c1066b0 0x7f4b8c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:53.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c19cb40 con 0x7f4b8c1006a0 2023-10-18T12:40:53.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c19ccb0 con 0x7f4b8c101060 2023-10-18T12:40:53.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b91fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c19ce70 con 0x7f4b8c1066b0 2023-10-18T12:40:53.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b8b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b8b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33466/0 (socket says 172.21.15.70:33466) 2023-10-18T12:40:53.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.989+0000 7f4b8b7fe700 1 -- 172.21.15.70:0/2904548269 learned_addr learned my addr 172.21.15.70:0/2904548269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:53.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8affd700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c101060 0x7f4b8c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8bfff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c1066b0 0x7f4b8c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:53.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8b7fe700 1 -- 172.21.15.70:0/2904548269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c1066b0 msgr2=0x7f4b8c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8b7fe700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c1066b0 0x7f4b8c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8b7fe700 1 -- 172.21.15.70:0/2904548269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c101060 msgr2=0x7f4b8c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8b7fe700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c101060 0x7f4b8c199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8b7fe700 1 -- 172.21.15.70:0/2904548269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800095f0 con 0x7f4b8c1006a0 2023-10-18T12:40:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8affd700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c101060 0x7f4b8c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:40:54.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b8b7fe700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c198d10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e9a0 tx=0x7f4b7c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:54.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c009790 con 0x7f4b8c1006a0 2023-10-18T12:40:54.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c19d150 con 0x7f4b8c1006a0 2023-10-18T12:40:54.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b7c00cbd0 con 0x7f4b8c1006a0 2023-10-18T12:40:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c018680 con 0x7f4b8c1006a0 2023-10-18T12:40:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.990+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c1a5ab0 con 0x7f4b8c1006a0 2023-10-18T12:40:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.991+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c101de0 con 0x7f4b8c1006a0 2023-10-18T12:40:54.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.993+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4b7c00cd40 con 0x7f4b8c1006a0 2023-10-18T12:40:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.995+0000 7f4b88ff9700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b740a15f0 0x7f4b740a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.995+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f4b7c014070 con 0x7f4b8c1006a0 2023-10-18T12:40:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.995+0000 7f4b8affd700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b740a15f0 0x7f4b740a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.995+0000 7f4b8affd700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b740a15f0 0x7f4b740a3ab0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000b2e0 tx=0x7f4b80005fb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:53.997+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4b7c096bb0 con 0x7f4b8c1006a0 2023-10-18T12:40:54.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.204+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b8c19a070 con 0x7f4b8c1006a0 2023-10-18T12:40:54.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.206+0000 7f4b88ff9700 1 -- 172.21.15.70:0/2904548269 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+25092 (secure 0 0 0) 0x7f4b7c09a1d0 con 0x7f4b8c1006a0 2023-10-18T12:40:54.209 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-18T12:40:54.209 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":109,"fsid":"f94fde2a-6daf-11ee-8db6-212e2dc638e7","created":"2023-10-18T12:16:03.804671+0000","modified":"2023-10-18T12:40:14.381877+0000","last_up_change":"2023-10-18T12:23:22.107674+0000","last_in_change":"2023-10-18T12:23:04.669489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T12:19:58.094929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:24:07.937110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T12:24:08.930078+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T12:24:12.620813+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"86cb6d70-44d3-441f-9d82-d24f1167aac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6803","nonce":2979990222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6805","nonce":2979990222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6809","nonce":2979990222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2979990222},{"type":"v1","addr":"172.21.15.70:6807","nonce":2979990222}]},"public_addr":"172.21.15.70:6803/2979990222","cluster_addr":"172.21.15.70:6805/2979990222","heartbeat_back_addr":"172.21.15.70:6809/2979990222","heartbeat_front_addr":"172.21.15.70:6807/2979990222","state":["exists","up"]},{"osd":1,"uuid":"48353e9d-5623-4c9b-b90a-35487c80253c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6801","nonce":1167081028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6803","nonce":1167081028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6807","nonce":1167081028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1167081028},{"type":"v1","addr":"172.21.15.152:6805","nonce":1167081028}]},"public_addr":"172.21.15.152:6801/1167081028","cluster_addr":"172.21.15.152:6803/1167081028","heartbeat_back_addr":"172.21.15.152:6807/1167081028","heartbeat_front_addr":"172.21.15.152:6805/1167081028","state":["exists","up"]},{"osd":2,"uuid":"972ea005-3f12-4646-896e-89622a35b19a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6801","nonce":1687717134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6803","nonce":1687717134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6807","nonce":1687717134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1687717134},{"type":"v1","addr":"172.21.15.186:6805","nonce":1687717134}]},"public_addr":"172.21.15.186:6801/1687717134","cluster_addr":"172.21.15.186:6803/1687717134","heartbeat_back_addr":"172.21.15.186:6807/1687717134","heartbeat_front_addr":"172.21.15.186:6805/1687717134","state":["exists","up"]},{"osd":3,"uuid":"c5d4fd36-c5d9-4b8d-aa7b-c620764c45c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6811","nonce":3863911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6813","nonce":3863911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6817","nonce":3863911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3863911164},{"type":"v1","addr":"172.21.15.70:6815","nonce":3863911164}]},"public_addr":"172.21.15.70:6811/3863911164","cluster_addr":"172.21.15.70:6813/3863911164","heartbeat_back_addr":"172.21.15.70:6817/3863911164","heartbeat_front_addr":"172.21.15.70:6815/3863911164","state":["exists","up"]},{"osd":4,"uuid":"ff61a4d2-db3e-4df1-abbc-daf21ea63965","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6809","nonce":934272440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6811","nonce":934272440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6815","nonce":934272440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":934272440},{"type":"v1","addr":"172.21.15.152:6813","nonce":934272440}]},"public_addr":"172.21.15.152:6809/934272440","cluster_addr":"172.21.15.152:6811/934272440","heartbeat_back_addr":"172.21.15.152:6815/934272440","heartbeat_front_addr":"172.21.15.152:6813/934272440","state":["exists","up"]},{"osd":5,"uuid":"cd152ca6-76f9-4773-887d-9a751fd85319","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6809","nonce":443683147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6811","nonce":443683147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6815","nonce":443683147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":443683147},{"type":"v1","addr":"172.21.15.186:6813","nonce":443683147}]},"public_addr":"172.21.15.186:6809/443683147","cluster_addr":"172.21.15.186:6811/443683147","heartbeat_back_addr":"172.21.15.186:6815/443683147","heartbeat_front_addr":"172.21.15.186:6813/443683147","state":["exists","up"]},{"osd":6,"uuid":"09373159-f65f-4a33-a0cd-7adaba830621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6819","nonce":3765589692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6821","nonce":3765589692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6825","nonce":3765589692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3765589692},{"type":"v1","addr":"172.21.15.70:6823","nonce":3765589692}]},"public_addr":"172.21.15.70:6819/3765589692","cluster_addr":"172.21.15.70:6821/3765589692","heartbeat_back_addr":"172.21.15.70:6825/3765589692","heartbeat_front_addr":"172.21.15.70:6823/3765589692","state":["exists","up"]},{"osd":7,"uuid":"2979cb11-038c-482c-b796-7d8dc3f6da42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6817","nonce":2054103643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6819","nonce":2054103643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6823","nonce":2054103643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2054103643},{"type":"v1","addr":"172.21.15.152:6821","nonce":2054103643}]},"public_addr":"172.21.15.152:6817/2054103643","cluster_addr":"172.21.15.152:6819/2054103643","heartbeat_back_addr":"172.21.15.152:6823/2054103643","heartbeat_front_addr":"172.21.15.152:6821/2054103643","state":["exists","up"]},{"osd":8,"uuid":"ba13df86-7cc5-4b47-8890-c35b0f2b2ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6817","nonce":2454566106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6819","nonce":2454566106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6823","nonce":2454566106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2454566106},{"type":"v1","addr":"172.21.15.186:6821","nonce":2454566106}]},"public_addr":"172.21.15.186:6817/2454566106","cluster_addr":"172.21.15.186:6819/2454566106","heartbeat_back_addr":"172.21.15.186:6823/2454566106","heartbeat_front_addr":"172.21.15.186:6821/2454566106","state":["exists","up"]},{"osd":9,"uuid":"6976bc71-bb5c-49ac-8b46-9b64c66fd4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6827","nonce":4034457990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6829","nonce":4034457990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6833","nonce":4034457990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":4034457990},{"type":"v1","addr":"172.21.15.70:6831","nonce":4034457990}]},"public_addr":"172.21.15.70:6827/4034457990","cluster_addr":"172.21.15.70:6829/4034457990","heartbeat_back_addr":"172.21.15.70:6833/4034457990","heartbeat_front_addr":"172.21.15.70:6831/4034457990","state":["exists","up"]},{"osd":10,"uuid":"310462a3-9997-49ff-ab18-0a1fbc14518d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6825","nonce":4238500329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6827","nonce":4238500329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6831","nonce":4238500329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":4238500329},{"type":"v1","addr":"172.21.15.152:6829","nonce":4238500329}]},"public_addr":"172.21.15.152:6825/4238500329","cluster_addr":"172.21.15.152:6827/4238500329","heartbeat_back_addr":"172.21.15.152:6831/4238500329","heartbeat_front_addr":"172.21.15.152:6829/4238500329","state":["exists","up"]},{"osd":11,"uuid":"3695b252-80cc-489f-a5d5-48f7ce64a1f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6825","nonce":1072168944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6827","nonce":1072168944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6831","nonce":1072168944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1072168944},{"type":"v1","addr":"172.21.15.186:6829","nonce":1072168944}]},"public_addr":"172.21.15.186:6825/1072168944","cluster_addr":"172.21.15.186:6827/1072168944","heartbeat_back_addr":"172.21.15.186:6831/1072168944","heartbeat_front_addr":"172.21.15.186:6829/1072168944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:13.861154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:35.626296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:19:56.060939+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:22.235415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:20:44.086266+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:04.991164+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:30.634187+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:21:52.204256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:12.746171+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:38.263169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:22:59.684551+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T12:23:20.152083+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1564492170":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/392897085":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6801/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/2942646472":"2023-10-19T12:18:13.515343+0000","172.21.15.70:6800/1990317867":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/771435827":"2023-10-19T12:17:02.382050+0000","172.21.15.70:6800/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4154455091":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/1853570886":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/4120653611":"2023-10-19T12:17:02.382050+0000","172.21.15.70:0/678282316":"2023-10-19T12:16:35.958063+0000","172.21.15.70:0/3661620656":"2023-10-19T12:16:35.958063+0000","172.21.15.70:6801/2836082256":"2023-10-19T12:18:13.515343+0000","172.21.15.70:0/477422870":"2023-10-19T12:17:02.382050+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:40:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b740a15f0 msgr2=0x7f4b740a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b740a15f0 0x7f4b740a3ab0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000b2e0 tx=0x7f4b80005fb0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 msgr2=0x7f4b8c198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c198d10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e9a0 tx=0x7f4b7c00ed60 comp rx=0 tx=0).stop 2023-10-18T12:40:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 shutdown_connections 2023-10-18T12:40:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c1066b0 0x7f4b8c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b8c1006a0 0x7f4b8c198d10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f4b740a15f0 0x7f4b740a3ab0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 --2- 172.21.15.70:0/2904548269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b8c101060 0x7f4b8c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 >> 172.21.15.70:0/2904548269 conn(0x7f4b8c0fc170 msgr2=0x7f4b8c072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:54.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 shutdown_connections 2023-10-18T12:40:54.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.209+0000 7f4b91fff700 1 -- 172.21.15.70:0/2904548269 wait complete. 2023-10-18T12:40:54.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T12:40:54.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/2572736679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941066b0 msgr2=0x7fcc94106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 --2- 172.21.15.70:0/2572736679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941066b0 0x7fcc94106a90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009990 tx=0x7fcc84009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/2572736679 shutdown_connections 2023-10-18T12:40:54.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 --2- 172.21.15.70:0/2572736679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc94101060 0x7fcc9410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 --2- 172.21.15.70:0/2572736679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941066b0 0x7fcc94106a90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 --2- 172.21.15.70:0/2572736679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc941006a0 0x7fcc94100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/2572736679 >> 172.21.15.70:0/2572736679 conn(0x7fcc940fc170 msgr2=0x7fcc940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:54.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/2572736679 shutdown_connections 2023-10-18T12:40:54.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.430+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/2572736679 wait complete. 2023-10-18T12:40:54.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.431+0000 7fcc9a1ff700 1 Processor -- start 2023-10-18T12:40:54.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 -- start start 2023-10-18T12:40:54.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 0x7fcc94072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc94101060 0x7fcc94073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc941066b0 0x7fcc94073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941a4eb0 con 0x7fcc941006a0 2023-10-18T12:40:54.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941a5020 con 0x7fcc941066b0 2023-10-18T12:40:54.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc9a1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941a51e0 con 0x7fcc94101060 2023-10-18T12:40:54.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc937fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 0x7fcc94072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc937fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 0x7fcc94072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33496/0 (socket says 172.21.15.70:33496) 2023-10-18T12:40:54.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.432+0000 7fcc937fe700 1 -- 172.21.15.70:0/1605381757 learned_addr learned my addr 172.21.15.70:0/1605381757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:54.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc93fff700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc941066b0 0x7fcc94073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc92ffd700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc94101060 0x7fcc94073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc937fe700 1 -- 172.21.15.70:0/1605381757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc94101060 msgr2=0x7fcc94073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc937fe700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc94101060 0x7fcc94073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc937fe700 1 -- 172.21.15.70:0/1605381757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc941066b0 msgr2=0x7fcc94073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc937fe700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc941066b0 0x7fcc94073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc937fe700 1 -- 172.21.15.70:0/1605381757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc840095f0 con 0x7fcc941006a0 2023-10-18T12:40:54.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc937fe700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 0x7fcc94072af0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc84005910 tx=0x7fcc84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:54.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401d070 con 0x7fcc941006a0 2023-10-18T12:40:54.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc84022470 con 0x7fcc941006a0 2023-10-18T12:40:54.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/1605381757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc941a5460 con 0x7fcc941006a0 2023-10-18T12:40:54.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.433+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84017650 con 0x7fcc941006a0 2023-10-18T12:40:54.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.434+0000 7fcc9a1ff700 1 -- 172.21.15.70:0/1605381757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc941a5990 con 0x7fcc941006a0 2023-10-18T12:40:54.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.435+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc94101de0 con 0x7fcc941006a0 2023-10-18T12:40:54.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.437+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcc840225e0 con 0x7fcc941006a0 2023-10-18T12:40:54.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.438+0000 7fcc90ff9700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc7c0a1670 0x7fcc7c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.438+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9381+0+0 (secure 0 0 0) 0x7fcc840cead0 con 0x7fcc941006a0 2023-10-18T12:40:54.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.439+0000 7fcc92ffd700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc7c0a1670 0x7fcc7c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.439+0000 7fcc92ffd700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc7c0a1670 0x7fcc7c0a3b30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc940741d0 tx=0x7fcc8800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:54.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.441+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc84097ac0 con 0x7fcc941006a0 2023-10-18T12:40:54.604 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1408816982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-mon[98639]: pgmap v777: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:54.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2904548269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1408816982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:54.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-mon[134767]: pgmap v777: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:54.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2904548269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:54.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.687+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcc94067530 con 0x7fcc941006a0 2023-10-18T12:40:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.689+0000 7fcc90ff9700 1 -- 172.21.15.70:0/1605381757 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcc8409b0e0 con 0x7fcc941006a0 2023-10-18T12:40:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.692+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc7c0a1670 msgr2=0x7fcc7c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.692+0000 7fcc7a7fc700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc7c0a1670 0x7fcc7c0a3b30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc940741d0 tx=0x7fcc8800b410 comp rx=0 tx=0).stop 2023-10-18T12:40:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.692+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 msgr2=0x7fcc94072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.692+0000 7fcc7a7fc700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 0x7fcc94072af0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc84005910 tx=0x7fcc84004730 comp rx=0 tx=0).stop 2023-10-18T12:40:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 shutdown_connections 2023-10-18T12:40:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fcc7c0a1670 0x7fcc7c0a3b30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc941066b0 0x7fcc94073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc941006a0 0x7fcc94072af0 secure :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc84005910 tx=0x7fcc84004730 comp rx=0 tx=0).stop 2023-10-18T12:40:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 --2- 172.21.15.70:0/1605381757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc94101060 0x7fcc94073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 >> 172.21.15.70:0/1605381757 conn(0x7fcc940fc170 msgr2=0x7fcc940fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 shutdown_connections 2023-10-18T12:40:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.693+0000 7fcc7a7fc700 1 -- 172.21.15.70:0/1605381757 wait complete. 2023-10-18T12:40:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:40:54.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T12:40:54.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:54 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1408816982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T12:40:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:54 smithi186 ceph-mon[135253]: pgmap v777: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2023-10-18T12:40:54.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:54 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2904548269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T12:40:54.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 -- 172.21.15.70:0/2729739115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 msgr2=0x7fb028100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2729739115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028100ba0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb010009990 tx=0x7fb010009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 -- 172.21.15.70:0/2729739115 shutdown_connections 2023-10-18T12:40:54.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2729739115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0281010e0 0x7fb02810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2729739115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb028106730 0x7fb028106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2729739115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028100ba0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 -- 172.21.15.70:0/2729739115 >> 172.21.15.70:0/2729739115 conn(0x7fb0280fc1b0 msgr2=0x7fb0280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:54.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 -- 172.21.15.70:0/2729739115 shutdown_connections 2023-10-18T12:40:54.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.920+0000 7fb02cfff700 1 -- 172.21.15.70:0/2729739115 wait complete. 2023-10-18T12:40:54.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.921+0000 7fb02cfff700 1 Processor -- start 2023-10-18T12:40:54.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.921+0000 7fb02cfff700 1 -- start start 2023-10-18T12:40:54.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02cfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0281010e0 0x7fb0281992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02cfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb028106730 0x7fb02819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02cfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819dca0 con 0x7fb028100720 2023-10-18T12:40:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02cfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819de10 con 0x7fb028106730 2023-10-18T12:40:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819dfd0 con 0x7fb0281010e0 2023-10-18T12:40:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33518/0 (socket says 172.21.15.70:33518) 2023-10-18T12:40:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 -- 172.21.15.70:0/2022729114 learned_addr learned my addr 172.21.15.70:0/2022729114 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb026d9d700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb028106730 0x7fb02819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb01ffff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0281010e0 0x7fb0281992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 -- 172.21.15.70:0/2022729114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0281010e0 msgr2=0x7fb0281992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0281010e0 0x7fb0281992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 -- 172.21.15.70:0/2022729114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb028106730 msgr2=0x7fb02819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb028106730 0x7fb02819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 -- 172.21.15.70:0/2022729114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100095f0 con 0x7fb028100720 2023-10-18T12:40:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.922+0000 7fb02659c700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028198d80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb01800d950 tx=0x7fb01800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.923+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018009940 con 0x7fb028100720 2023-10-18T12:40:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.923+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb018010460 con 0x7fb028100720 2023-10-18T12:40:54.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.923+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02819e2b0 con 0x7fb028100720 2023-10-18T12:40:54.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.923+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01800f620 con 0x7fb028100720 2023-10-18T12:40:54.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.923+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0281a5af0 con 0x7fb028100720 2023-10-18T12:40:54.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.924+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb028101e60 con 0x7fb028100720 2023-10-18T12:40:54.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.926+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb0180108d0 con 0x7fb028100720 2023-10-18T12:40:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.927+0000 7fb01f7fe700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb0140a1670 0x7fb0140a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.927+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9379+0+0 (secure 0 0 0) 0x7fb0180cd1a0 con 0x7fb028100720 2023-10-18T12:40:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.927+0000 7fb01ffff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb0140a1670 0x7fb0140a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.928+0000 7fb01ffff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb0140a1670 0x7fb0140a3b30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b5c0 tx=0x7fb01001a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:54.929+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb018096100 con 0x7fb028100720 2023-10-18T12:40:55.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a[98615]: 2023-10-18T12:40:54.687+0000 7efcff1ab700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:40:55.105 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g[157317]: ignoring --setuser ceph since I am not root 2023-10-18T12:40:55.106 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g[157317]: ignoring --setgroup ceph since I am not root 2023-10-18T12:40:55.106 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-mds[157361]: -- 172.21.15.70:0/1004728541 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56367fd751e0 con 0x56367fd4cc00 2023-10-18T12:40:55.106 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:40:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g[157317]: starting mds.g at 2023-10-18T12:40:55.172 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: ignoring --setuser ceph since I am not root 2023-10-18T12:40:55.172 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: ignoring --setgroup ceph since I am not root 2023-10-18T12:40:55.173 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-mds[173985]: -- 172.21.15.152:0/3046928700 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aa376a31e0 con 0x55aa37678c00 2023-10-18T12:40:55.173 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:40:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: starting mds.b at 2023-10-18T12:40:55.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.179+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb02819a120 con 0x7fb028100720 2023-10-18T12:40:55.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1605381757' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T12:40:55.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-18T12:40:55.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:40:55.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1605381757' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-18T12:40:55.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: osdmap e110: 12 total, 12 up, 12 in 2023-10-18T12:40:55.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-10-18T12:40:55.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:55 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2022729114' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:40:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.691+0000 7fb01f7fe700 1 -- 172.21.15.70:0/2022729114 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb0180160a0 con 0x7fb028100720 2023-10-18T12:40:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb0140a1670 msgr2=0x7fb0140a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb0140a1670 0x7fb0140a3b30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b5c0 tx=0x7fb01001a040 comp rx=0 tx=0).stop 2023-10-18T12:40:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 msgr2=0x7fb028198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028198d80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb01800d950 tx=0x7fb01800dc60 comp rx=0 tx=0).stop 2023-10-18T12:40:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 shutdown_connections 2023-10-18T12:40:55.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fb0140a1670 0x7fb0140a3b30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb028106730 0x7fb02819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb028100720 0x7fb028198d80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 --2- 172.21.15.70:0/2022729114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0281010e0 0x7fb0281992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 >> 172.21.15.70:0/2022729114 conn(0x7fb0280fc1b0 msgr2=0x7fb02810bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 shutdown_connections 2023-10-18T12:40:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.694+0000 7fb02cfff700 1 -- 172.21.15.70:0/2022729114 wait complete. 2023-10-18T12:40:55.716 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1605381757' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T12:40:55.716 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-18T12:40:55.716 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:40:55.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1605381757' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-18T12:40:55.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: osdmap e110: 12 total, 12 up, 12 in 2023-10-18T12:40:55.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-10-18T12:40:55.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:55 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2022729114' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:40:55.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T12:40:55.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1605381757' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T12:40:55.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-18T12:40:55.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:40:55.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1605381757' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-18T12:40:55.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: osdmap e110: 12 total, 12 up, 12 in 2023-10-18T12:40:55.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-10-18T12:40:55.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:55 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2022729114' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T12:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.920+0000 7fd79a5ff700 1 -- 172.21.15.70:0/1909605685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7941010e0 msgr2=0x7fd79410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.920+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/1909605685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7941010e0 0x7fd79410d5f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00b0a0 tx=0x7fd78c00b3b0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 -- 172.21.15.70:0/1909605685 shutdown_connections 2023-10-18T12:40:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/1909605685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7941010e0 0x7fd79410d5f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/1909605685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd794106730 0x7fd794106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/1909605685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794100720 0x7fd794100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 -- 172.21.15.70:0/1909605685 >> 172.21.15.70:0/1909605685 conn(0x7fd7940fc1b0 msgr2=0x7fd7940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 -- 172.21.15.70:0/1909605685 shutdown_connections 2023-10-18T12:40:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.921+0000 7fd79a5ff700 1 -- 172.21.15.70:0/1909605685 wait complete. 2023-10-18T12:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 Processor -- start 2023-10-18T12:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 -- start start 2023-10-18T12:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 0x7fd79419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7941010e0 0x7fd79419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794106730 0x7fd794198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941991d0 con 0x7fd794100720 2023-10-18T12:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794199340 con 0x7fd7941010e0 2023-10-18T12:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd79a5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794199500 con 0x7fd794106730 2023-10-18T12:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd793fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 0x7fd79419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd793fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 0x7fd79419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33534/0 (socket says 172.21.15.70:33534) 2023-10-18T12:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.922+0000 7fd793fff700 1 -- 172.21.15.70:0/2939221118 learned_addr learned my addr 172.21.15.70:0/2939221118 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd7937fe700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7941010e0 0x7fd79419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd798b9c700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794106730 0x7fd794198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd793fff700 1 -- 172.21.15.70:0/2939221118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794106730 msgr2=0x7fd794198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd793fff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794106730 0x7fd794198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd793fff700 1 -- 172.21.15.70:0/2939221118 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7941010e0 msgr2=0x7fd79419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd793fff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7941010e0 0x7fd79419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd793fff700 1 -- 172.21.15.70:0/2939221118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c009d00 con 0x7fd794100720 2023-10-18T12:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd793fff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 0x7fd79419e9f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd78400e9a0 tx=0x7fd78400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:55.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784009790 con 0x7fd794100720 2023-10-18T12:40:55.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7941997e0 con 0x7fd794100720 2023-10-18T12:40:55.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd784010430 con 0x7fd794100720 2023-10-18T12:40:55.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784018680 con 0x7fd794100720 2023-10-18T12:40:55.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.923+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7941a59e0 con 0x7fd794100720 2023-10-18T12:40:55.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.924+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd794101e60 con 0x7fd794100720 2023-10-18T12:40:55.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.928+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd7840105a0 con 0x7fd794100720 2023-10-18T12:40:55.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.929+0000 7fd7917fa700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd77c0a15f0 0x7fd77c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:55.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.929+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9379+0+0 (secure 0 0 0) 0x7fd784014070 con 0x7fd794100720 2023-10-18T12:40:55.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.930+0000 7fd7937fe700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd77c0a15f0 0x7fd77c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:55.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.930+0000 7fd7937fe700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd77c0a15f0 0x7fd77c0a3ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd794199ce0 tx=0x7fd788009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:55.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:55.932+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd784096750 con 0x7fd794100720 2023-10-18T12:40:56.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.133+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7940675a0 con 0x7fd794100720 2023-10-18T12:40:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-18T12:40:56.672 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-18T12:40:56.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: Cluster is now healthy 2023-10-18T12:40:56.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2022729114' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-18T12:40:56.673 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.152:6838/3773419096,v1:172.21.15.152:6839/3773419096] up:boot 2023-10-18T12:40:56.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: mds.? [v2:172.21.15.70:6838/2973829837,v1:172.21.15.70:6839/2973829837] up:boot 2023-10-18T12:40:56.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: fsmap 12 up:standby 2023-10-18T12:40:56.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T12:40:56.674 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T12:40:56.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: pgmap v779: 129 pgs: 129 active+clean; 775 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 151 KiB/s wr, 3 op/s 2023-10-18T12:40:56.675 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:56 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2939221118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:56.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.697+0000 7fd7917fa700 1 -- 172.21.15.70:0/2939221118 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd7840260d0 con 0x7fd794100720 2023-10-18T12:40:56.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.700+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd77c0a15f0 msgr2=0x7fd77c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:56.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.700+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd77c0a15f0 0x7fd77c0a3ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd794199ce0 tx=0x7fd788009260 comp rx=0 tx=0).stop 2023-10-18T12:40:56.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.700+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 msgr2=0x7fd79419e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:56.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.700+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 0x7fd79419e9f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd78400e9a0 tx=0x7fd78400ecb0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 shutdown_connections 2023-10-18T12:40:56.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794106730 0x7fd794198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794100720 0x7fd79419e9f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7fd77c0a15f0 0x7fd77c0a3ab0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 --2- 172.21.15.70:0/2939221118 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7941010e0 0x7fd79419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 >> 172.21.15.70:0/2939221118 conn(0x7fd7940fc1b0 msgr2=0x7fd7940fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:56.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 shutdown_connections 2023-10-18T12:40:56.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.701+0000 7fd79a5ff700 1 -- 172.21.15.70:0/2939221118 wait complete. 2023-10-18T12:40:56.707 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-18T12:40:56.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-18T12:40:56.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-18T12:40:56.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: Cluster is now healthy 2023-10-18T12:40:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2022729114' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-18T12:40:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.152:6838/3773419096,v1:172.21.15.152:6839/3773419096] up:boot 2023-10-18T12:40:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: mds.? [v2:172.21.15.70:6838/2973829837,v1:172.21.15.70:6839/2973829837] up:boot 2023-10-18T12:40:56.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: fsmap 12 up:standby 2023-10-18T12:40:56.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T12:40:56.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T12:40:56.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: pgmap v779: 129 pgs: 129 active+clean; 775 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 151 KiB/s wr, 3 op/s 2023-10-18T12:40:56.721 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:56 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2939221118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:56.736 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-18T12:40:56.736 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-18T12:40:56.736 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: Cluster is now healthy 2023-10-18T12:40:56.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2022729114' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-18T12:40:56.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.152:6838/3773419096,v1:172.21.15.152:6839/3773419096] up:boot 2023-10-18T12:40:56.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: mds.? [v2:172.21.15.70:6838/2973829837,v1:172.21.15.70:6839/2973829837] up:boot 2023-10-18T12:40:56.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: fsmap 12 up:standby 2023-10-18T12:40:56.738 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T12:40:56.738 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: from='mgr.14214 172.21.15.70:0/255358476' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T12:40:56.738 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: pgmap v779: 129 pgs: 129 active+clean; 775 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 151 KiB/s wr, 3 op/s 2023-10-18T12:40:56.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:56 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2939221118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:56.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T12:40:56.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.955+0000 7f33a99ff700 1 -- 172.21.15.70:0/1430752361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a4072b90 msgr2=0x7f33a4072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:56.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.955+0000 7f33a99ff700 1 --2- 172.21.15.70:0/1430752361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a4072b90 0x7f33a4072f70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f33940098f0 tx=0x7f3394009c00 comp rx=0 tx=0).stop 2023-10-18T12:40:56.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 -- 172.21.15.70:0/1430752361 shutdown_connections 2023-10-18T12:40:56.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 --2- 172.21.15.70:0/1430752361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a4111730 0x7f33a4113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 --2- 172.21.15.70:0/1430752361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a4072b90 0x7f33a4072f70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 --2- 172.21.15.70:0/1430752361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a40734b0 0x7f33a41111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 -- 172.21.15.70:0/1430752361 >> 172.21.15.70:0/1430752361 conn(0x7f33a40fc1b0 msgr2=0x7f33a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:56.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 -- 172.21.15.70:0/1430752361 shutdown_connections 2023-10-18T12:40:56.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.956+0000 7f33a99ff700 1 -- 172.21.15.70:0/1430752361 wait complete. 2023-10-18T12:40:56.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.958+0000 7f33a99ff700 1 Processor -- start 2023-10-18T12:40:56.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.958+0000 7f33a99ff700 1 -- start start 2023-10-18T12:40:56.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.959+0000 7f33a99ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4072b90 0x7f33a419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:56.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.959+0000 7f33a99ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 0x7f33a419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:56.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.959+0000 7f33a99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a4111730 0x7f33a41a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:56.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.959+0000 7f33a99ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41a21d0 con 0x7f33a40734b0 2023-10-18T12:40:56.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.959+0000 7f33a99ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41a2340 con 0x7f33a4072b90 2023-10-18T12:40:56.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.959+0000 7f33a99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41a2500 con 0x7f33a4111730 2023-10-18T12:40:56.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.960+0000 7f33a27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 0x7f33a419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:56.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.960+0000 7f33a27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 0x7f33a419d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33554/0 (socket says 172.21.15.70:33554) 2023-10-18T12:40:56.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.960+0000 7f33a27fc700 1 -- 172.21.15.70:0/913694673 learned_addr learned my addr 172.21.15.70:0/913694673 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:56.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.961+0000 7f33a27fc700 1 -- 172.21.15.70:0/913694673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a4111730 msgr2=0x7f33a41a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:56.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.961+0000 7f33a27fc700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a4111730 0x7f33a41a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.961+0000 7f33a27fc700 1 -- 172.21.15.70:0/913694673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4072b90 msgr2=0x7f33a419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:56.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.961+0000 7f33a27fc700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4072b90 0x7f33a419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:56.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.961+0000 7f33a27fc700 1 -- 172.21.15.70:0/913694673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394009520 con 0x7f33a40734b0 2023-10-18T12:40:56.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.961+0000 7f33a27fc700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 0x7f33a419d840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f339000e9c0 tx=0x7f339000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:56.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.962+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000cb50 con 0x7f33a40734b0 2023-10-18T12:40:56.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.962+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3390004cf0 con 0x7f33a40734b0 2023-10-18T12:40:56.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.962+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390005560 con 0x7f33a40734b0 2023-10-18T12:40:56.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.962+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a41a27e0 con 0x7f33a40734b0 2023-10-18T12:40:56.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.962+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a41aa020 con 0x7f33a40734b0 2023-10-18T12:40:56.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.964+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4103c10 con 0x7f33a40734b0 2023-10-18T12:40:56.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.967+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f339000ccb0 con 0x7f33a40734b0 2023-10-18T12:40:56.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.970+0000 7f33a89fd700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f33880a1670 0x7f33880a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:56.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.970+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8825+0+0 (secure 0 0 0) 0x7f3390014070 con 0x7f33a40734b0 2023-10-18T12:40:56.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.970+0000 7f33a2ffd700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f33880a1670 0x7f33880a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:56.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.971+0000 7f33a2ffd700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f33880a1670 0x7f33880a3b30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f339400c010 tx=0x7f3394009f50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:56.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:56.975+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3390096560 con 0x7f33a40734b0 2023-10-18T12:40:57.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.207+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33a419e5b0 con 0x7f33a40734b0 2023-10-18T12:40:57.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.703+0000 7f33a89fd700 1 -- 172.21.15.70:0/913694673 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3390099b80 con 0x7f33a40734b0 2023-10-18T12:40:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.707+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f33880a1670 msgr2=0x7f33880a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.707+0000 7f33a99ff700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f33880a1670 0x7f33880a3b30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f339400c010 tx=0x7f3394009f50 comp rx=0 tx=0).stop 2023-10-18T12:40:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.707+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 msgr2=0x7f33a419d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.707+0000 7f33a99ff700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 0x7f33a419d840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f339000e9c0 tx=0x7f339000ed80 comp rx=0 tx=0).stop 2023-10-18T12:40:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 shutdown_connections 2023-10-18T12:40:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a4111730 0x7f33a41a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f33880a1670 0x7f33880a3b30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4072b90 0x7f33a419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 --2- 172.21.15.70:0/913694673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33a40734b0 0x7f33a419d840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 >> 172.21.15.70:0/913694673 conn(0x7f33a40fc1b0 msgr2=0x7f33a40fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 shutdown_connections 2023-10-18T12:40:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.708+0000 7f33a99ff700 1 -- 172.21.15.70:0/913694673 wait complete. 2023-10-18T12:40:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-18T12:40:57.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-18T12:40:57.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 -- 172.21.15.70:0/2682069133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8106730 msgr2=0x7f25e8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:57.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/2682069133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8106730 0x7f25e8106b10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009990 tx=0x7f25d8009ca0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 -- 172.21.15.70:0/2682069133 shutdown_connections 2023-10-18T12:40:57.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/2682069133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e81010e0 0x7f25e810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/2682069133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8106730 0x7f25e8106b10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/2682069133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25e8100720 0x7f25e8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 -- 172.21.15.70:0/2682069133 >> 172.21.15.70:0/2682069133 conn(0x7f25e80fc1b0 msgr2=0x7f25e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:57.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 -- 172.21.15.70:0/2682069133 shutdown_connections 2023-10-18T12:40:57.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.944+0000 7f25ec9ff700 1 -- 172.21.15.70:0/2682069133 wait complete. 2023-10-18T12:40:57.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.945+0000 7f25ec9ff700 1 Processor -- start 2023-10-18T12:40:57.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.945+0000 7f25ec9ff700 1 -- start start 2023-10-18T12:40:57.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25ec9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 0x7f25e819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:57.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25ec9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25e81010e0 0x7f25e819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:57.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25ec9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106730 0x7f25e8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:57.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25ec9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8199350 con 0x7f25e8100720 2023-10-18T12:40:57.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25ec9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81994c0 con 0x7f25e81010e0 2023-10-18T12:40:57.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25ec9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8199680 con 0x7f25e8106730 2023-10-18T12:40:57.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 0x7f25e819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:57.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 0x7f25e819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59262/0 (socket says 172.21.15.70:59262) 2023-10-18T12:40:57.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 -- 172.21.15.70:0/1790497152 learned_addr learned my addr 172.21.15.70:0/1790497152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-18T12:40:57.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e5d9b700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25e81010e0 0x7f25e819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:57.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e6d9d700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106730 0x7f25e8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:57.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 -- 172.21.15.70:0/1790497152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106730 msgr2=0x7f25e8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:57.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106730 0x7f25e8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 -- 172.21.15.70:0/1790497152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25e81010e0 msgr2=0x7f25e819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25e81010e0 0x7f25e819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.946+0000 7f25e659c700 1 -- 172.21.15.70:0/1790497152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80095f0 con 0x7f25e8100720 2023-10-18T12:40:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.947+0000 7f25e659c700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 0x7f25e819ea70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.947+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d801d070 con 0x7f25e8100720 2023-10-18T12:40:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.947+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d8022470 con 0x7f25e8100720 2023-10-18T12:40:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.947+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8017650 con 0x7f25e8100720 2023-10-18T12:40:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.947+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e8199900 con 0x7f25e8100720 2023-10-18T12:40:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.947+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81a59f0 con 0x7f25e8100720 2023-10-18T12:40:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.948+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8101e60 con 0x7f25e8100720 2023-10-18T12:40:57.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.950+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f25d80225e0 con 0x7f25e8100720 2023-10-18T12:40:57.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.952+0000 7f25d77fe700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25d00a15f0 0x7f25d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:40:57.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.952+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8295+0+0 (secure 0 0 0) 0x7f25d80ce360 con 0x7f25e8100720 2023-10-18T12:40:57.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.952+0000 7f25e5d9b700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25d00a15f0 0x7f25d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:40:57.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.953+0000 7f25e5d9b700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25d00a15f0 0x7f25d00a3ab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f25e819a280 tx=0x7f25dc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:40:57.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:57.956+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25d8097790 con 0x7f25e8100720 2023-10-18T12:40:57.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:57 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/2939221118' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T12:40:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:57 smithi186 ceph-mon[135253]: osdmap e111: 12 total, 12 up, 12 in 2023-10-18T12:40:57.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:57 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/913694673' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:58.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:57 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/2939221118' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T12:40:58.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:57 smithi070 ceph-mon[98639]: osdmap e111: 12 total, 12 up, 12 in 2023-10-18T12:40:58.105 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:57 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/913694673' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:58.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:57 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/2939221118' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T12:40:58.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:57 smithi152 ceph-mon[134767]: osdmap e111: 12 total, 12 up, 12 in 2023-10-18T12:40:58.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:57 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/913694673' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.181+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25e819a020 con 0x7f25e8100720 2023-10-18T12:40:58.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.709+0000 7f25d77fe700 1 -- 172.21.15.70:0/1790497152 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v113) v1 ==== 178+0+0 (secure 0 0 0) 0x7f25d809adb0 con 0x7f25e8100720 2023-10-18T12:40:58.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.715+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25d00a15f0 msgr2=0x7f25d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.715+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25d00a15f0 0x7f25d00a3ab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f25e819a280 tx=0x7f25dc009260 comp rx=0 tx=0).stop 2023-10-18T12:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.715+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 msgr2=0x7f25e819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.715+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 0x7f25e819ea70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d8005740 comp rx=0 tx=0).stop 2023-10-18T12:40:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 shutdown_connections 2023-10-18T12:40:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106730 0x7f25e8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25e8100720 0x7f25e819ea70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.70:6800/223035468,v1:172.21.15.70:6801/223035468] conn(0x7f25d00a15f0 0x7f25d00a3ab0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 --2- 172.21.15.70:0/1790497152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25e81010e0 0x7f25e819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 >> 172.21.15.70:0/1790497152 conn(0x7f25e80fc1b0 msgr2=0x7f25e810bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 shutdown_connections 2023-10-18T12:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-18T12:40:58.716+0000 7f25ec9ff700 1 -- 172.21.15.70:0/1790497152 wait complete. 2023-10-18T12:40:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data_ec' removed 2023-10-18T12:40:58.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:40:58.821 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:40:58.868 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:40:58.904 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-18T12:40:58.905 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-18T12:40:58.949 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-18T12:40:58.968 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-18T12:40:58.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/913694673' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T12:40:58.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:58 smithi186 ceph-mon[135253]: osdmap e112: 12 total, 12 up, 12 in 2023-10-18T12:40:58.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:58 smithi186 ceph-mon[135253]: pgmap v782: 33 pgs: 33 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:40:58.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:40:58 smithi186 ceph-mon[135253]: from='client.? 172.21.15.70:0/1790497152' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:58.989 INFO:tasks.cephadm:Stopping all daemons... 2023-10-18T12:40:58.990 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-18T12:40:58.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a 2023-10-18T12:40:59.051 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/913694673' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T12:40:59.051 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:58 smithi070 ceph-mon[98639]: osdmap e112: 12 total, 12 up, 12 in 2023-10-18T12:40:59.051 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:58 smithi070 ceph-mon[98639]: pgmap v782: 33 pgs: 33 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:40:59.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:58 smithi070 ceph-mon[98639]: from='client.? 172.21.15.70:0/1790497152' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:58 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/913694673' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T12:40:59.172 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:58 smithi152 ceph-mon[134767]: osdmap e112: 12 total, 12 up, 12 in 2023-10-18T12:40:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:58 smithi152 ceph-mon[134767]: pgmap v782: 33 pgs: 33 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T12:40:59.173 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:40:58 smithi152 ceph-mon[134767]: from='client.? 172.21.15.70:0/1790497152' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T12:40:59.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:59 smithi070 systemd[1]: Stopping Ceph mon.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:40:59.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a[98615]: 2023-10-18T12:40:59.521+0000 7efd049b6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:40:59.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:40:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a[98615]: 2023-10-18T12:40:59.521+0000 7efd049b6700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-18T12:41:00.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Oct 18 12:41:00 smithi070 bash[189116]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-a 2023-10-18T12:41:00.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.a.service' 2023-10-18T12:41:00.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:00.587 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-18T12:41:00.588 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-18T12:41:00.588 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.b 2023-10-18T12:41:00.922 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:41:00 smithi152 systemd[1]: Stopping Ceph mon.b for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:01.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:41:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:41:00.774+0000 7f32e80f0700 -1 client.24115: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2023-10-18T12:41:01.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:41:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:41:00.776+0000 7f32e80f0700 -1 client.24115: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2023-10-18T12:41:01.247 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:41:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-b[134742]: 2023-10-18T12:41:00.989+0000 7f8dccb8d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:01.248 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:41:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-b[134742]: 2023-10-18T12:41:00.989+0000 7f8dccb8d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-18T12:41:01.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Oct 18 12:41:01 smithi152 bash[179259]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-b 2023-10-18T12:41:01.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.b.service' 2023-10-18T12:41:01.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:01.782 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-18T12:41:01.782 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-18T12:41:01.783 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.c 2023-10-18T12:41:02.188 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:41:01 smithi186 systemd[1]: Stopping Ceph mon.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:02.189 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:41:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-c[135223]: 2023-10-18T12:41:02.184+0000 7f6ec6621700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:02.189 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:41:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-c[135223]: 2023-10-18T12:41:02.184+0000 7f6ec6621700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-18T12:41:02.828 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:41:02 smithi186 bash[179496]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mon-c 2023-10-18T12:41:02.828 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:41:02 smithi186 systemd[1]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.c.service: Succeeded. 2023-10-18T12:41:02.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 18 12:41:02 smithi186 systemd[1]: Stopped Ceph mon.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:41:02.830 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mon.c.service' 2023-10-18T12:41:02.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:02.910 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-18T12:41:02.911 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-18T12:41:02.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x 2023-10-18T12:41:03.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:41:02 smithi070 systemd[1]: Stopping Ceph mgr.x for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:03.689 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:41:03 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:41:03.412+0000 7f3395787700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:03.690 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:41:03 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x[98885]: 2023-10-18T12:41:03.412+0000 7f3395787700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-18T12:41:04.105 INFO:journalctl@ceph.mgr.x.smithi070.stdout:Oct 18 12:41:03 smithi070 bash[189265]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-x 2023-10-18T12:41:04.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.x.service' 2023-10-18T12:41:04.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:04.273 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-18T12:41:04.274 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-18T12:41:04.275 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.y 2023-10-18T12:41:04.672 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:41:04 smithi152 systemd[1]: Stopping Ceph mgr.y for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:04.982 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:41:04 smithi152 bash[179399]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-y 2023-10-18T12:41:05.161 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Oct 18 12:41:04 smithi152 systemd[1]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-18T12:41:05.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.y.service' 2023-10-18T12:41:05.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:05.204 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-18T12:41:05.204 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-18T12:41:05.205 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.z 2023-10-18T12:41:05.637 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:41:05 smithi186 systemd[1]: Stopping Ceph mgr.z for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:05.914 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:41:05 smithi186 bash[179641]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mgr-z 2023-10-18T12:41:06.096 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 18 12:41:05 smithi186 systemd[1]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-18T12:41:06.099 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mgr.z.service' 2023-10-18T12:41:06.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:06.137 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-18T12:41:06.137 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-18T12:41:06.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.0 2023-10-18T12:41:06.605 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:41:06 smithi070 systemd[1]: Stopping Ceph osd.0 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:07.105 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:41:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0[124289]: 2023-10-18T12:41:06.670+0000 7f9c31041700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:07.105 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:41:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0[124289]: 2023-10-18T12:41:06.670+0000 7f9c31041700 -1 osd.0 113 *** Got signal Terminated *** 2023-10-18T12:41:07.106 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:41:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0[124289]: 2023-10-18T12:41:06.670+0000 7f9c31041700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:12.858 INFO:journalctl@ceph.osd.0.smithi070.stdout:Oct 18 12:41:12 smithi070 bash[189410]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-0 2023-10-18T12:41:14.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.0.service' 2023-10-18T12:41:14.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:14.577 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-18T12:41:14.577 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-18T12:41:14.578 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.1 2023-10-18T12:41:14.922 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:41:14 smithi152 systemd[1]: Stopping Ceph osd.1 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:15.422 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:41:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1[140962]: 2023-10-18T12:41:14.943+0000 7f4cc645c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:15.422 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:41:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1[140962]: 2023-10-18T12:41:14.943+0000 7f4cc645c700 -1 osd.1 113 *** Got signal Terminated *** 2023-10-18T12:41:15.423 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:41:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1[140962]: 2023-10-18T12:41:14.943+0000 7f4cc645c700 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:20.921 INFO:journalctl@ceph.osd.1.smithi152.stdout:Oct 18 12:41:20 smithi152 bash[179550]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-1 2023-10-18T12:41:21.963 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.1.service' 2023-10-18T12:41:22.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:22.005 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-18T12:41:22.005 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-18T12:41:22.005 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.2 2023-10-18T12:41:22.375 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:41:22 smithi186 systemd[1]: Stopping Ceph osd.2 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:22.719 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:41:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2[142282]: 2023-10-18T12:41:22.371+0000 7ff726027700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:22.720 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:41:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2[142282]: 2023-10-18T12:41:22.371+0000 7ff726027700 -1 osd.2 113 *** Got signal Terminated *** 2023-10-18T12:41:22.720 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:41:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2[142282]: 2023-10-18T12:41:22.371+0000 7ff726027700 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:28.145 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 18 12:41:28 smithi186 bash[179785]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-2 2023-10-18T12:41:29.507 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.2.service' 2023-10-18T12:41:29.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:29.547 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-18T12:41:29.547 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-18T12:41:29.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.3 2023-10-18T12:41:30.060 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:41:29 smithi070 systemd[1]: Stopping Ceph osd.3 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:30.355 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:41:30 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-3[132819]: 2023-10-18T12:41:30.057+0000 7f5dfa1e0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:30.355 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:41:30 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-3[132819]: 2023-10-18T12:41:30.057+0000 7f5dfa1e0700 -1 osd.3 113 *** Got signal Terminated *** 2023-10-18T12:41:30.355 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:41:30 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-3[132819]: 2023-10-18T12:41:30.057+0000 7f5dfa1e0700 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:33.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:33 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:33.092+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:34.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:34 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:34.088+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:34.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:34.255+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:35.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:34.805+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:35.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:35 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:35.045+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:35.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:35.297+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:35.765 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:35 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:35.474+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:36.105 INFO:journalctl@ceph.osd.3.smithi070.stdout:Oct 18 12:41:35 smithi070 bash[189688]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-3 2023-10-18T12:41:36.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:35.830+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:36.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:36 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:36.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:36.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:36.282+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:36.773 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:36 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:36.510+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:37.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:36.872+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:37.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:37 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:37.069+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:37.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:37 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:37.460+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:37.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.3.service' 2023-10-18T12:41:37.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:37 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:37.295+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:37.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:37.776 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-18T12:41:37.777 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-18T12:41:37.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.4 2023-10-18T12:41:38.064 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:37 smithi152 systemd[1]: Stopping Ceph osd.4 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:38.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:37 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:37.916+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:38.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:38 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:38.061+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:38.422 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:38 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:41:38.120+0000 7f62c91b0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:38.423 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:38 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:41:38.120+0000 7f62c91b0700 -1 osd.4 113 *** Got signal Terminated *** 2023-10-18T12:41:38.423 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:38 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:41:38.120+0000 7f62c91b0700 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:38.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:38 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:38.300+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:38.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:38 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:38.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:39.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:38 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:38.891+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:39.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:39 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:39.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:39.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:39 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:39.332+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:39.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:39 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:39.410+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:40.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:39 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:39.897+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:40.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:40 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:40.035+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:40.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:40 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:40.329+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:40.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:40 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:40.378+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:41.040 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:40 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:40.712+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:41.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:40 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:40.851+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:41.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:41 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:41.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:41.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:41 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:41.314+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:41.760 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:41 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:41:41.429+0000 7f62c37a2700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.611980+0000 front 2023-10-18T12:41:19.612011+0000 (oldest deadline 2023-10-18T12:41:40.712164+0000) 2023-10-18T12:41:41.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:41 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:41.364+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:42.048 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:41 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:41.757+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:42.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:41 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:41.871+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:42.421 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:42 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:42.045+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:42.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:42 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:42.356+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:42.759 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:42 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4[149674]: 2023-10-18T12:41:42.454+0000 7f62c37a2700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.611980+0000 front 2023-10-18T12:41:19.612011+0000 (oldest deadline 2023-10-18T12:41:40.712164+0000) 2023-10-18T12:41:42.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:42 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:42.359+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:43.068 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:42 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:42.756+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:43.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:42 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:42.917+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:43.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:42 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:42.917+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:43.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:43 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:43.065+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:43.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:43 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:43.065+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:43.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:43 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:43.311+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:43.795 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:43 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:43.366+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:43.795 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:43 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:43.366+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:44.085 INFO:journalctl@ceph.osd.4.smithi152.stdout:Oct 18 12:41:43 smithi152 bash[179825]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-4 2023-10-18T12:41:44.086 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:43 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:43.749+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:44.104 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:43 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:43.792+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:44.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:43 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:43.932+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:44.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:43 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:43.932+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:44.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:44 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:44.082+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:44.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:44 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:44.082+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:44.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:44 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:44.303+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:44.825 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:44 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:44.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:44.826 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:44 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:44.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:45.045 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:44 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:44.762+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:45.046 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:45 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:45.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:45.046 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:45 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:45.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:45.104 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:44 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:44.823+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:45.182 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.4.service' 2023-10-18T12:41:45.218 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:44 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:44.830+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.153078+0000 front 2023-10-18T12:41:19.153179+0000 (oldest deadline 2023-10-18T12:41:43.853154+0000) 2023-10-18T12:41:45.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:44 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:44.975+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:45.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:44 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:44.975+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:45.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:45.222 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-18T12:41:45.223 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-18T12:41:45.223 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.5 2023-10-18T12:41:45.589 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:45 smithi186 systemd[1]: Stopping Ceph osd.5 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:45.589 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:45.586+0000 7fb82629f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:45.590 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:45.335+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:45.590 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:45.335+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:45.794 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:45 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:45.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:45.795 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:45 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:45.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:45.879 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:45.586+0000 7fb82629f700 -1 osd.5 113 *** Got signal Terminated *** 2023-10-18T12:41:45.879 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:45.586+0000 7fb82629f700 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:46.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:45 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:45.791+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:46.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:45 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:45.810+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:46.172 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:46 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:46.015+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:46.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:46 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:46.015+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:46.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:45.876+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.153078+0000 front 2023-10-18T12:41:19.153179+0000 (oldest deadline 2023-10-18T12:41:43.853154+0000) 2023-10-18T12:41:46.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:45.997+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:46.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:45 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:45.997+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:46.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:46 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:46.372+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:46.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:46 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:46.372+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:46.827 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:46 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:46.412+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:46.828 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:46 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:46.412+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:47.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:46 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:46.825+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:47.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:46 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:46.856+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:47.172 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:47 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:47.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:47.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:47 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:47.049+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:47.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:46 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:46.861+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.153078+0000 front 2023-10-18T12:41:19.153179+0000 (oldest deadline 2023-10-18T12:41:43.853154+0000) 2023-10-18T12:41:47.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:47.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:47.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:47.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:47.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:47.373+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:47.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:47.373+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:47.795 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:47 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:47.448+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:47.795 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:47 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:47.448+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:48.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:47 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:47.792+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:48.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:47 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:47.830+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:48.172 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:48 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:48.069+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:48.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:48 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:48.069+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:48.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:48 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:48.069+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:48.218 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:47.827+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.153078+0000 front 2023-10-18T12:41:19.153179+0000 (oldest deadline 2023-10-18T12:41:43.853154+0000) 2023-10-18T12:41:48.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:47.981+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:48.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:47 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:47.981+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:48.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:48 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:48.357+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:48.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:48 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:48.357+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:48.840 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:48.435+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:48.840 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:48.435+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:48.841 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:48.435+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:49.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:48.837+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:49.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:48.837+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:49.106 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:48 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:48.785+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:49.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:48 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:48.863+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.153078+0000 front 2023-10-18T12:41:19.153179+0000 (oldest deadline 2023-10-18T12:41:43.853154+0000) 2023-10-18T12:41:49.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:48 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:48.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:49.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:48 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:48.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:49.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:49 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:49.102+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:49.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:49 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:49.102+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:49.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:49 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:49.102+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:49.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:49 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:49.398+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:49.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:49 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:49.398+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:49.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:49 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:49.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:49.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:49 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:49.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:49.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:49 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:49.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:50.101 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:49 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:49.834+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:50.102 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:49 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:49.834+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:50.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:49 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:49.901+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.153078+0000 front 2023-10-18T12:41:19.153179+0000 (oldest deadline 2023-10-18T12:41:43.853154+0000) 2023-10-18T12:41:50.219 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:49 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5[150350]: 2023-10-18T12:41:49.901+0000 7fb820891700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.853696+0000 front 2023-10-18T12:41:23.853722+0000 (oldest deadline 2023-10-18T12:41:49.753879+0000) 2023-10-18T12:41:50.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:50 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:50.007+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:50.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:50 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:50.007+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:50.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:49 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:49.864+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:50.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:49 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:49.864+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:50.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:50 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:50.098+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:50.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:50 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:50.098+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:50.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:50 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:50.098+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:50.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:50 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:50.386+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:50.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:50 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:50.386+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:50.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:50 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:50.386+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:50.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:50 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:50.369+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:50.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:50 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:50.369+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:50.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:50 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:50.369+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:51.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:50 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:50.870+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:51.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:50 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:50.870+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:51.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:51 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:51.087+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:51.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:51 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:51.087+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:51.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:51 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:51.087+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:51.341 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:50 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:50.884+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:51.342 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:50 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:50.884+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:51.346 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:51 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:51.051+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:51.346 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:51 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:51.051+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:51.604 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:51 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:51.339+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:51.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:51 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:51.339+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:51.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:51 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:51.339+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:51.719 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 18 12:41:51 smithi186 bash[180056]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-5 2023-10-18T12:41:51.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:51 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:51.422+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:51.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:51 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:51.422+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:51.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:51 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:51.422+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:52.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:51 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:51.912+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:52.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:51 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:51.912+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:52.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:52.065+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:52.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:52.065+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:52.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:52.065+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:52.341 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:51 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:51.859+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:52.342 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:51 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:51.859+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:52.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:52.023+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:52.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:52.023+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:52.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:52 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:52.339+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:52.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:52 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:52.339+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:52.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:52 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:52.339+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:52.838 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:52.466+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:52.839 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:52.466+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:52.839 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:52.466+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:52.845 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.5.service' 2023-10-18T12:41:52.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:41:52.885 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-18T12:41:52.885 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-18T12:41:52.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.6 2023-10-18T12:41:53.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:52.869+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:53.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:52 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:52.869+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:53.172 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:53.028+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:53.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:53.028+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:53.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:53.028+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:53.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:52.986+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:53.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:52.986+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:53.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:52 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:52.986+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:53.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:52 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:52.884+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:53.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:52 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:52.884+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:53.356 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:52 smithi070 systemd[1]: Stopping Ceph osd.6 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:41:53.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:53 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:53.422+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:53.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:53 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:53.422+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:53.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:53 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:53.422+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:53.855 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:53.401+0000 7f1810cb2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:41:53.855 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:53.401+0000 7f1810cb2700 -1 osd.6 113 *** Got signal Terminated *** 2023-10-18T12:41:53.856 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:53.401+0000 7f1810cb2700 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:41:53.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:53.365+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:53.857 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:53.365+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:53.857 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:53.365+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:54.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:53.909+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:54.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:53.909+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:54.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:54.004+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:54.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:54.004+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:54.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:54.004+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:54.354 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:53.853+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:54.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:53 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:53.853+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:54.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:54.017+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:54.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:54.017+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:54.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:54.017+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:54.846 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:54.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:54.847 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:54.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:54.847 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:54.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:54.968 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:54.470+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:54.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:54.470+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:54.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:54 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:54.470+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:55.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:54.844+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:55.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:54 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:54.844+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:55.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:54.900+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:55.172 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:54.900+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:55.172 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:54.963+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:55.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:54.963+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:55.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:54 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:54.963+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:55.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:55.056+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:55.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:55.056+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:55.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:55.056+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:55.846 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:55.382+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:55.847 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:55.382+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:55.847 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:55.382+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:55.968 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:55.471+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:55.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:55.471+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:55.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:55 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:55.471+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:56.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:55.844+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:56.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:55.844+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:56.105 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:55 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:55.844+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.623331+0000 front 2023-10-18T12:41:31.623362+0000 (oldest deadline 2023-10-18T12:41:55.723469+0000) 2023-10-18T12:41:56.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:55.937+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:56.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:55.937+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:56.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:55.949+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:56.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:55.949+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:56.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:55.949+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:56.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:56.083+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:56.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:56.083+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:56.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:56.083+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:56.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:56.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:56.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:56.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:56.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:56.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:56.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:56.402+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:41:56.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:56.493+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:56.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:56.493+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:56.970 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:56 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:56.493+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:57.354 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:56.890+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:57.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:56.890+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:57.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:56 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:56.890+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.623331+0000 front 2023-10-18T12:41:31.623362+0000 (oldest deadline 2023-10-18T12:41:55.723469+0000) 2023-10-18T12:41:57.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:56 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:56.969+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:57.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:56 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:56.969+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:57.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:56 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:56.969+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:41:57.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:56 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:56.959+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:57.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:56 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:56.959+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:57.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:56 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:56.959+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:57.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:57.039+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:57.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:57.039+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:57.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:57.039+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:57.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:57.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:57.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:57.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:57.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:57.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:57.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:57.415+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:41:57.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:57.481+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:57.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:57.481+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:57.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:57.481+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:58.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:57.908+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.918484+0000 front 2023-10-18T12:41:18.918514+0000 (oldest deadline 2023-10-18T12:41:43.618645+0000) 2023-10-18T12:41:58.355 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:57.908+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.120404+0000 front 2023-10-18T12:41:24.120434+0000 (oldest deadline 2023-10-18T12:41:48.820568+0000) 2023-10-18T12:41:58.356 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:57 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6[141245]: 2023-10-18T12:41:57.908+0000 7f18092a0700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.623331+0000 front 2023-10-18T12:41:31.623362+0000 (oldest deadline 2023-10-18T12:41:55.723469+0000) 2023-10-18T12:41:58.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:57 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:57.996+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:58.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:57 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:57.996+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:58.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:57 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:57.996+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:41:58.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:57 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:57.922+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:58.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:57 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:57.922+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:58.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:57 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:57.922+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:58.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:57.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:58.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:57.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:58.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:57 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:57.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:58.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:58.439+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:58.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:58.440+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:58.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:58.440+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:58.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:58 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:58.440+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:41:58.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:58 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:58.478+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:58.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:58 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:58.478+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:58.969 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:58 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:58.478+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:59.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:59.036+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:41:59.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:59.036+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:41:59.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:59.036+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:41:59.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:58.960+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:41:59.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:58.960+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:41:59.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:58.960+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:41:59.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:58.960+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:41:59.434 INFO:journalctl@ceph.osd.6.smithi070.stdout:Oct 18 12:41:59 smithi070 bash[189969]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-6 2023-10-18T12:41:59.462 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:59.039+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:41:59.463 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:59.039+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:41:59.463 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:59.040+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:41:59.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:59.460+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:41:59.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:59.460+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:41:59.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:41:59.460+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:41:59.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:59.431+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:41:59.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:59.431+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:41:59.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:59.431+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:41:59.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:41:59 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:41:59.431+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:00.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:59.998+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:00.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:59.998+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:00.423 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:41:59.998+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:00.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:59.974+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:00.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:59.974+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:00.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:59.974+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:00.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:41:59 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:41:59.974+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:00.455 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:59.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:00.455 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:59.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:00.456 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:41:59 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:41:59.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:00.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:00.405+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:00.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:00.405+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:00.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:00.405+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:00.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:00 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:00.405+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:00.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:00.452+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:00.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:00.452+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:00.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:00.452+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:00.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:00.452+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:01.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.6.service' 2023-10-18T12:42:01.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:01.195 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-18T12:42:01.195 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-18T12:42:01.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.7 2023-10-18T12:42:01.228 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:00.959+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:01.228 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:00.959+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:01.228 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:00 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:00.959+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:01.229 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:01.010+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:01.229 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:01.010+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:01.230 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:01.010+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:01.230 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:01.010+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:01.427 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:00.984+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:01.427 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:00.984+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:01.428 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:00 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:00.984+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:01.481 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 systemd[1]: Stopping Ceph osd.7 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:01.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:01 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:01.424+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:01.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:01 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:01.424+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:01.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:01 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:01.424+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:01.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:01 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:01.424+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:01.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:01 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:01.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:01.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:01 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:01.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:01.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:01 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:01.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:01.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:01 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:01.424+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:01.922 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:01.541+0000 7f04cc783700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:01.922 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:01.541+0000 7f04cc783700 -1 osd.7 113 *** Got signal Terminated *** 2023-10-18T12:42:01.923 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:01.541+0000 7f04cc783700 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:42:02.394 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:01.999+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:02.394 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:01.999+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:02.395 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:01.999+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:02.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:01.966+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:02.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:01.966+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:02.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:01.966+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:02.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:02.008+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:02.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:02.008+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:02.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:02.008+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:02.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:02.008+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:02.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:02.392+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:02.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:02.392+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:02.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:02.392+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:02.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:02.392+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:02.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:02 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:02.384+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:02.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:02 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:02.384+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:02.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:02 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:02.384+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:02.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:02 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:02.384+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:03.390 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:02.975+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:03.391 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:02.975+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:03.391 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:02 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:02.975+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:03.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:02.919+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:03.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:02.919+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:03.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:02 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:02.919+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:03.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:03.046+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:03.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:03.046+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:03.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:03.046+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:03.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:03.046+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:03.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:03.388+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:03.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:03.388+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:03.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:03.388+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:03.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:03.388+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:03.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:03 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:03.364+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:03.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:03 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:03.364+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:03.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:03 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:03.364+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:03.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:03 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:03.364+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:04.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:03.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:04.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:03.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:04.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:03.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:04.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:03 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:03.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:04.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:03.962+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:04.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:03.962+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:04.423 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:03 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:03.962+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:04.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:04.011+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:04.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:04.011+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:04.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:04.011+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:04.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:04.011+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:04.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:04.340+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:04.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:04.340+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:04.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:04.340+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:04.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:04.340+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:04.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:04 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:04.366+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:04.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:04 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:04.366+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:04.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:04 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:04.366+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:04.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:04 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:04.366+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:04.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:04.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:05.343 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:04.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:05.344 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:04 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:04.995+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:05.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:04.953+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:05.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:04.953+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:05.423 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:04 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:04.953+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:05.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:05.060+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:05.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:05.060+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:05.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:05.060+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:05.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:05.060+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:05.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:05 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:05.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:05.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:05 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:05.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:05.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:05 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:05.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:05.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:05 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:05.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:05.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:05.397+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:05.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:05.397+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:05.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:05.397+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:05.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:05 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:05.397+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:06.347 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.028+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:06.347 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.028+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:06.348 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.028+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:06.348 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.028+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:06.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:05.980+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.937124+0000 front 2023-10-18T12:41:18.937095+0000 (oldest deadline 2023-10-18T12:41:40.637243+0000) 2023-10-18T12:42:06.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:05.980+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.738531+0000 front 2023-10-18T12:41:24.738349+0000 (oldest deadline 2023-10-18T12:41:48.838358+0000) 2023-10-18T12:42:06.423 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:05 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7[158188]: 2023-10-18T12:42:05.980+0000 7f04c6d75700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:31.139753+0000 front 2023-10-18T12:41:31.139723+0000 (oldest deadline 2023-10-18T12:41:56.439641+0000) 2023-10-18T12:42:06.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:06 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:06.012+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:06.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:06 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:06.012+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:06.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:06 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:06.012+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:06.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:06 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:06.012+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:06.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:06.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:06.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:06.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:06.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:06.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:06.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:06.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:06.854 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:06.398+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:06.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:06.398+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:06.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:06.398+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:06.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:06 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:06.398+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:07.376 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:07.377 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:07.377 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:07.377 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:06.991+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:07.422 INFO:journalctl@ceph.osd.7.smithi152.stdout:Oct 18 12:42:07 smithi152 bash[180094]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-7 2023-10-18T12:42:07.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.031+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:07.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.031+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:07.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.031+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:07.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.031+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:07.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:07 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:07.374+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:07.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:07 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:07.374+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:07.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:07 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:07.374+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:07.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:07 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:07.374+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:07.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:07.353+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:07.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:07.353+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:07.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:07.353+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:07.856 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:07 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:07.353+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:08.172 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.992+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:08.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.992+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:08.173 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.992+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:08.174 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.992+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:08.174 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:07 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:07.992+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:08.371 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.004+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:08.372 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.004+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:08.372 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.004+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:08.372 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.004+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:08.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:08 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:08.349+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:08.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:08 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:08.349+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:08.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:08 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:08.349+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:08.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:08 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:08.349+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:08.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.7.service' 2023-10-18T12:42:08.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:08.704 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-18T12:42:08.705 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-18T12:42:08.705 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.8 2023-10-18T12:42:08.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:08.368+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:08.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:08.368+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:08.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:08.368+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:08.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:08.368+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:09.068 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:08 smithi186 systemd[1]: Stopping Ceph osd.8 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:09.069 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.976+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:09.069 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.976+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:09.069 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.976+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:09.070 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:08 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:08.976+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:09.347 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.065+0000 7fe150719700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:09.348 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.065+0000 7fe150719700 -1 osd.8 113 *** Got signal Terminated *** 2023-10-18T12:42:09.348 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.065+0000 7fe150719700 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:42:09.349 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:09.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:08 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:08.986+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:09.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:08 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:08.986+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:09.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:08 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:08.986+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:09.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:08 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:08.986+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:09.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:08 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:08.986+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:09.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:09.345+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:09.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:09.345+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:09.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:09.345+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:09.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:09.345+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:09.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:09 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:09.345+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:09.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:09.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:09.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:09.345+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:10.221 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:09.938+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:10.221 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:09.938+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:10.222 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:09.938+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:10.222 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:09 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:09.938+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:10.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:10 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:10.027+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:10.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:10 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:10.027+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:10.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:10 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:10.027+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:10.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:10 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:10.027+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:10.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:10 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:10.027+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:10.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:10.337+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:10.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:10.337+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:10.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:10.337+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:10.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:10.337+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:10.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:10 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:10.337+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:10.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:10.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:10.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:10.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:10.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:10.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:10.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:10.341+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:11.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:10.965+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:11.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:10.965+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:11.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:10.965+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:11.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:10 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:10.965+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:11.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:11 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:11.018+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:11.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:11 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:11.018+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:11.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:11 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:11.018+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:11.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:11 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:11.018+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:11.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:11 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:11.018+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:11.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:11.316+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:11.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:11.316+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:11.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:11.316+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:11.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:11.316+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:11.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:11 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:11.316+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:11.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:11.387+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:11.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:11.387+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:11.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:11.387+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:11.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:11.387+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:12.374 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:11.977+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:12.374 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:11.977+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:12.375 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:11.977+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:12.375 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:11.977+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:12.375 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:11.977+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:12.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:12 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:12.062+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:12.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:12 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:12.062+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:12.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:12 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:12.062+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:12.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:12 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:12.062+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:12.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:12 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:12.062+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:12.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:12.292+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:12.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:12.292+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:12.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:12.292+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:12.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:12.292+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:12.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:12 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:12.292+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:12.718 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:12.371+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:12.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:12.371+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:12.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:12.371+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:12.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:12.371+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:13.386 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:12.966+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:13.387 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:12.966+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:13.387 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:12.966+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:13.387 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:12.966+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:13.388 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:12 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:12.966+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:13.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:13 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:13.038+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:13.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:13 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:13.038+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:13.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:13 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:13.038+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:13.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:13 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:13.038+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:13.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:13 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:13.038+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:13.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:13.251+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:13.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:13.251+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:13.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:13.251+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:13.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:13.251+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:13.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:13 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:13.251+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:13.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:13.382+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.347020+0000 front 2023-10-18T12:41:10.347007+0000 (oldest deadline 2023-10-18T12:41:33.847020+0000) 2023-10-18T12:42:13.719 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:13.383+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:19.148383+0000 front 2023-10-18T12:41:19.148302+0000 (oldest deadline 2023-10-18T12:41:44.448335+0000) 2023-10-18T12:42:13.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:13.383+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:24.448927+0000 front 2023-10-18T12:41:24.448858+0000 (oldest deadline 2023-10-18T12:41:49.748981+0000) 2023-10-18T12:42:13.720 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8[158236]: 2023-10-18T12:42:13.383+0000 7fe14ad0b700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.251348+0000 front 2023-10-18T12:41:34.251327+0000 (oldest deadline 2023-10-18T12:42:00.151040+0000) 2023-10-18T12:42:14.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:14.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:14.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:14.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:14.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:14.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:14.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:14.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:14.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:14.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:14.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:14 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:14.037+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:14.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:14 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:14.002+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:14.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:14 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:14.002+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:14.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:14 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:14.002+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:14.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:14 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:14.002+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:14.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:14 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:14.002+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:14.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:14.296+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:14.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:14.296+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:14.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:14.296+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:14.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:14.296+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:14.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:14.296+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:14.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:14 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:14.296+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:14.739 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 18 12:42:14 smithi186 bash[180324]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-8 2023-10-18T12:42:15.311 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:15.038+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:15.312 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:15.038+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:15.312 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:15.038+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:15.312 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:15.038+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:15.313 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:15.038+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:15.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:15 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:15.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:15.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:15 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:15.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:15.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:15 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:15.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:15.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:15 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:15.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:15.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:15 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:15.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:15.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:15 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:15.041+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:15.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:15.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:15.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:15.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:15.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:15.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:15.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:15.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:15.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:15.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:15.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:15 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:15.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:16.001 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.8.service' 2023-10-18T12:42:16.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:16.083 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-18T12:42:16.083 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-18T12:42:16.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.9 2023-10-18T12:42:16.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:16 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:16.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:16.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:16 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:16.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:16.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:16 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:16.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:16.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:16 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:16.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:16.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:16 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:16.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:16.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:16 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:16.055+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:16.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.000+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:16.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.000+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:16.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.000+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:16.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.000+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:16.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.000+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:16.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 systemd[1]: Stopping Ceph osd.9 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:16.627 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:16.627 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:16.628 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:16.628 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:16.628 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:16.629 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:17.105 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.604+0000 7fe5cbc71700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:17.105 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.604+0000 7fe5cbc71700 -1 osd.9 113 *** Got signal Terminated *** 2023-10-18T12:42:17.105 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:16 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:16.604+0000 7fe5cbc71700 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:42:17.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:17 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:17.026+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:17.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:17 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:17.026+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:17.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:17 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:17.026+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:17.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:17 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:17.026+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:17.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:17 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:17.026+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:17.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:17 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:17.026+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:17.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.989+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:17.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.989+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:17.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.989+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:17.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.989+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:17.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:16 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:16.989+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:17.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:17.260+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:17.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:17.260+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:17.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:17.260+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:17.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:17.260+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:17.607 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:17.260+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:17.607 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:17 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:17.260+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:18.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:18 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:18.059+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:18.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:18 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:18.059+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:18.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:18 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:18.059+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:18.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:18 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:18.059+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:18.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:18 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:18.059+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:18.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:18 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:18.059+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:18.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:17 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:17.979+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:18.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:17 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:17.979+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:18.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:17 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:17.979+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:18.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:17 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:17.979+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:18.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:17 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:17.979+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:18.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:18.294+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:18.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:18.294+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:18.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:18.294+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:18.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:18.294+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:18.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:18.294+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:18.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:18 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:18.294+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:19.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:18 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:18.957+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:19.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:18 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:18.957+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:19.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:18 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:18.957+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:19.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:18 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:18.957+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:19.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:18 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:18.957+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:19.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:19 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:19.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:19.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:19 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:19.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:19.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:19 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:19.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:19.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:19 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:19.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:19.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:19 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:19.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:19.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:19 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:19.048+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:19.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:19 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:19.313+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:19.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:19 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:19.313+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:19.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:19 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:19.313+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:19.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:19 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:19.313+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:19.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:19 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:19.313+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:19.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:19 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:19.313+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:20.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:20 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:20.090+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:20.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:20 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:20.090+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:20.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:20 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:20.090+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:20.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:20 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:20.090+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:20.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:20 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:20.090+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:20.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:20 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:20.090+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:20.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:19 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:19.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:20.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:19 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:19.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:20.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:19 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:19.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:20.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:19 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:19.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:20.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:19 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:19.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:20.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:20.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:20.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:20.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:20.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:20.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:20.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:20.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:20.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:20.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:20.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:20 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:20.283+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:21.218 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:20 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:20.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:21.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:20 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:20.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:21.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:20 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:20.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:21.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:20 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:20.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:21.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:20 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:20.948+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:21.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:21.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:21.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:21.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:21.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:21.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:21.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:21 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:21.089+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:21.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:21 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:21.242+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:09.269765+0000 front 2023-10-18T12:41:09.269793+0000 (oldest deadline 2023-10-18T12:41:35.169935+0000) 2023-10-18T12:42:21.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:21 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:21.242+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.571861+0000 front 2023-10-18T12:41:18.571978+0000 (oldest deadline 2023-10-18T12:41:43.271945+0000) 2023-10-18T12:42:21.605 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:21 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:21.242+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:23.272622+0000 front 2023-10-18T12:41:23.272501+0000 (oldest deadline 2023-10-18T12:41:47.972707+0000) 2023-10-18T12:42:21.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:21 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:21.242+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:33.873999+0000 front 2023-10-18T12:41:33.874032+0000 (oldest deadline 2023-10-18T12:41:55.574157+0000) 2023-10-18T12:42:21.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:21 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:21.242+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.477079+0000 front 2023-10-18T12:41:42.477098+0000 (oldest deadline 2023-10-18T12:42:08.377045+0000) 2023-10-18T12:42:21.606 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:21 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9[149187]: 2023-10-18T12:42:21.242+0000 7fe5c5a62700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:48.377794+0000 front 2023-10-18T12:41:48.377862+0000 (oldest deadline 2023-10-18T12:42:14.277649+0000) 2023-10-18T12:42:22.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:22.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:22.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:22.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:22.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:22.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:22.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:22 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:22.079+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:22.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:21.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:22.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:21.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:22.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:21.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:22.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:21.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:22.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:21 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:21.983+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:22.855 INFO:journalctl@ceph.osd.9.smithi070.stdout:Oct 18 12:42:22 smithi070 bash[190240]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-9 2023-10-18T12:42:23.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:23.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:23.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:23.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:23.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:23.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:23.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:23 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:23.101+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:23.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:22.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:23.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:22.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:23.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:22.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:23.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:22.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:23.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:22 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:22.982+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:24.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.9.service' 2023-10-18T12:42:24.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:24.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:24.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:24.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:24.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:24.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:24.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.124+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:24.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:24.012+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:24.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:24.012+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:24.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:24.012+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:24.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:24.012+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:24.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:24.012+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:24.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:24 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:24.012+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:24.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:24.473 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-18T12:42:24.473 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-18T12:42:24.474 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.10 2023-10-18T12:42:24.824 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 systemd[1]: Stopping Ceph osd.10 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:25.079 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.822+0000 7fc60c5d4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:25.079 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.822+0000 7fc60c5d4700 -1 osd.10 113 *** Got signal Terminated *** 2023-10-18T12:42:25.080 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:24 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:24.822+0000 7fc60c5d4700 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:42:25.080 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:25.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:25.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:25.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:25.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:25.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:25.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:25 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:25.077+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:25.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:25.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:25.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:25.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:25.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:25.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:25.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:25.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:25.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:25.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:25.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:25 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:25.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:26.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:26.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:26.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:26.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:26.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:26.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:26.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:26 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:26.058+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:26.468 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:26.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:26.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:26.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:26.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:26.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:26.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:26.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:26.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:26.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:26.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:26 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:26.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:27.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:27.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:27.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:27.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:27.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:27.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:27.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:27 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:27.094+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:27.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:27.037+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:27.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:27.037+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:27.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:27.037+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:27.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:27.037+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:27.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:27.037+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:27.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:27 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:27.037+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:28.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:28.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:28.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:28.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:28.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:28.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:28.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:28 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:28.120+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:28.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:28.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:28.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:28.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:28.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:28.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:28.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:28 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:28.036+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:29.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:10.250822+0000 front 2023-10-18T12:41:10.250641+0000 (oldest deadline 2023-10-18T12:41:32.550803+0000) 2023-10-18T12:42:29.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:17.251735+0000 front 2023-10-18T12:41:17.251929+0000 (oldest deadline 2023-10-18T12:41:42.552113+0000) 2023-10-18T12:42:29.422 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:22.552870+0000 front 2023-10-18T12:41:22.552924+0000 (oldest deadline 2023-10-18T12:41:47.852632+0000) 2023-10-18T12:42:29.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6814 osd.3 since back 2023-10-18T12:41:34.755197+0000 front 2023-10-18T12:41:34.755115+0000 (oldest deadline 2023-10-18T12:41:58.254900+0000) 2023-10-18T12:42:29.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:42.355749+0000 front 2023-10-18T12:41:42.355782+0000 (oldest deadline 2023-10-18T12:42:07.055936+0000) 2023-10-18T12:42:29.423 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:49.356975+0000 front 2023-10-18T12:41:49.357155+0000 (oldest deadline 2023-10-18T12:42:13.456982+0000) 2023-10-18T12:42:29.424 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:29 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10[166823]: 2023-10-18T12:42:29.097+0000 7fc605bc4700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:53.457492+0000 front 2023-10-18T12:41:53.457471+0000 (oldest deadline 2023-10-18T12:42:20.458138+0000) 2023-10-18T12:42:29.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:29.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:29.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:29.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:29.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:29.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:29.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:29 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:29.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:30.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:30.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:30.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:30.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:30.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:30.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:30.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:30 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:30.018+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:30.672 INFO:journalctl@ceph.osd.10.smithi152.stdout:Oct 18 12:42:30 smithi152 bash[180366]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-10 2023-10-18T12:42:31.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:31.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:31.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:31.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:31.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:31.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:31.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.006+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:31.573 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.10.service' 2023-10-18T12:42:31.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:31.614 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-18T12:42:31.614 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-18T12:42:31.615 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.11 2023-10-18T12:42:31.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 systemd[1]: Stopping Ceph osd.11 for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:32.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.970+0000 7f1013b28700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:32.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.970+0000 7f1013b28700 -1 osd.11 113 *** Got signal Terminated *** 2023-10-18T12:42:32.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:31 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:31.970+0000 7f1013b28700 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T12:42:32.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:32.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:32.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:32.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:32.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:32.472 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:32.472 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:32 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:32.044+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:33.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:33.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:33.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:33.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:33.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:33.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:33.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:33 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:33.085+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:34.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:34.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:34.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:34.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:34.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:34.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:34.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:34 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:34.115+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:35.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:35.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:35.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:35.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:35.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:35.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:35.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:35 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:35.075+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:36.469 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2023-10-18T12:41:08.162916+0000 front 2023-10-18T12:41:08.162900+0000 (oldest deadline 2023-10-18T12:41:34.062783+0000) 2023-10-18T12:42:36.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2023-10-18T12:41:18.664668+0000 front 2023-10-18T12:41:18.664739+0000 (oldest deadline 2023-10-18T12:41:42.764752+0000) 2023-10-18T12:42:36.470 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-18T12:41:26.865670+0000 front 2023-10-18T12:41:26.865690+0000 (oldest deadline 2023-10-18T12:41:52.165902+0000) 2023-10-18T12:42:36.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2023-10-18T12:41:37.967943+0000 front 2023-10-18T12:41:37.967979+0000 (oldest deadline 2023-10-18T12:42:03.868016+0000) 2023-10-18T12:42:36.471 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-10-18T12:41:47.269396+0000 front 2023-10-18T12:41:47.269413+0000 (oldest deadline 2023-10-18T12:42:11.969457+0000) 2023-10-18T12:42:36.472 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.70:6822 osd.6 since back 2023-10-18T12:41:57.671572+0000 front 2023-10-18T12:41:57.671601+0000 (oldest deadline 2023-10-18T12:42:23.471750+0000) 2023-10-18T12:42:36.472 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:36 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11[167017]: 2023-10-18T12:42:36.112+0000 7f100c116700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2023-10-18T12:42:05.772988+0000 front 2023-10-18T12:42:05.772884+0000 (oldest deadline 2023-10-18T12:42:27.472757+0000) 2023-10-18T12:42:37.969 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 18 12:42:37 smithi186 bash[180590]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-osd-11 2023-10-18T12:42:39.112 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@osd.11.service' 2023-10-18T12:42:39.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:39.153 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-18T12:42:39.154 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-18T12:42:39.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.a 2023-10-18T12:42:39.605 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:42:39 smithi070 systemd[1]: Stopping Ceph mds.a for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:40.105 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:42:39 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-a[157724]: 2023-10-18T12:42:39.665+0000 7f9c0d155700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:40.105 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:42:39 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-a[157724]: 2023-10-18T12:42:39.665+0000 7f9c0d155700 -1 mds.a *** got signal Terminated *** 2023-10-18T12:42:41.330 INFO:journalctl@ceph.mds.a.smithi070.stdout:Oct 18 12:42:41 smithi070 bash[190510]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-a 2023-10-18T12:42:41.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.a.service' 2023-10-18T12:42:41.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:41.719 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-18T12:42:41.719 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-18T12:42:41.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.d 2023-10-18T12:42:42.105 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:42:41 smithi070 systemd[1]: Stopping Ceph mds.d for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:42.605 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:42:42 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-d[156716]: 2023-10-18T12:42:42.205+0000 7f6881f82700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:42.605 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:42:42 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-d[156716]: 2023-10-18T12:42:42.205+0000 7f6881f82700 -1 mds.d *** got signal Terminated *** 2023-10-18T12:42:44.355 INFO:journalctl@ceph.mds.d.smithi070.stdout:Oct 18 12:42:44 smithi070 bash[190647]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-d 2023-10-18T12:42:44.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.d.service' 2023-10-18T12:42:44.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:44.513 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-18T12:42:44.514 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-18T12:42:44.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.g 2023-10-18T12:42:44.855 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:42:44 smithi070 systemd[1]: Stopping Ceph mds.g for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:45.355 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:42:44 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g[157317]: 2023-10-18T12:42:44.997+0000 7ff2d323f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:45.355 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:42:44 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g[157317]: 2023-10-18T12:42:44.997+0000 7ff2d323f700 -1 mds.g *** got signal Terminated *** 2023-10-18T12:42:47.355 INFO:journalctl@ceph.mds.g.smithi070.stdout:Oct 18 12:42:47 smithi070 bash[190792]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-g 2023-10-18T12:42:47.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.g.service' 2023-10-18T12:42:47.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:47.587 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-18T12:42:47.588 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-18T12:42:47.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.j 2023-10-18T12:42:47.855 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:42:47 smithi070 systemd[1]: Stopping Ceph mds.j for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:48.355 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:42:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-j[156322]: 2023-10-18T12:42:48.072+0000 7f122c0fe700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:48.355 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:42:48 smithi070 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-j[156322]: 2023-10-18T12:42:48.072+0000 7f122c0fe700 -1 mds.j *** got signal Terminated *** 2023-10-18T12:42:52.508 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:42:52 smithi070 bash[190936]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-j 2023-10-18T12:42:52.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.j.service' 2023-10-18T12:42:52.768 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:42:52 smithi070 systemd[1]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.j.service: Succeeded. 2023-10-18T12:42:52.768 INFO:journalctl@ceph.mds.j.smithi070.stdout:Oct 18 12:42:52 smithi070 systemd[1]: Stopped Ceph mds.j for f94fde2a-6daf-11ee-8db6-212e2dc638e7. 2023-10-18T12:42:52.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:52.791 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-18T12:42:52.792 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-18T12:42:52.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.b 2023-10-18T12:42:53.154 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:42:52 smithi152 systemd[1]: Stopping Ceph mds.b for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:53.422 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:42:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: 2023-10-18T12:42:53.150+0000 7f8595d85700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:53.422 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:42:53 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b[173961]: 2023-10-18T12:42:53.150+0000 7f8595d85700 -1 mds.b *** got signal Terminated *** 2023-10-18T12:42:55.109 INFO:journalctl@ceph.mds.b.smithi152.stdout:Oct 18 12:42:55 smithi152 bash[180628]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-b 2023-10-18T12:42:55.298 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.b.service' 2023-10-18T12:42:55.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:55.335 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-18T12:42:55.336 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-18T12:42:55.336 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.e 2023-10-18T12:42:55.672 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:42:55 smithi152 systemd[1]: Stopping Ceph mds.e for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:56.172 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:42:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-e[173271]: 2023-10-18T12:42:55.725+0000 7f4d3fdd8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:56.172 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:42:55 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-e[173271]: 2023-10-18T12:42:55.725+0000 7f4d3fdd8700 -1 mds.e *** got signal Terminated *** 2023-10-18T12:42:58.278 INFO:journalctl@ceph.mds.e.smithi152.stdout:Oct 18 12:42:58 smithi152 bash[180768]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-e 2023-10-18T12:42:58.468 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.e.service' 2023-10-18T12:42:58.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:42:58.503 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-18T12:42:58.504 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-18T12:42:58.504 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.h 2023-10-18T12:42:58.895 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:42:58 smithi152 systemd[1]: Stopping Ceph mds.h for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:42:59.172 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:42:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-h[173614]: 2023-10-18T12:42:58.891+0000 7f69f6cc5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:42:59.172 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:42:58 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-h[173614]: 2023-10-18T12:42:58.891+0000 7f69f6cc5700 -1 mds.h *** got signal Terminated *** 2023-10-18T12:43:00.922 INFO:journalctl@ceph.mds.h.smithi152.stdout:Oct 18 12:43:00 smithi152 bash[180917]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-h 2023-10-18T12:43:01.028 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.h.service' 2023-10-18T12:43:01.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:43:01.064 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-18T12:43:01.064 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-18T12:43:01.065 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.k 2023-10-18T12:43:01.422 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:43:01 smithi152 systemd[1]: Stopping Ceph mds.k for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:43:01.922 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:43:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-k[172935]: 2023-10-18T12:43:01.427+0000 7fc9b8710700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:43:01.922 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:43:01 smithi152 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-k[172935]: 2023-10-18T12:43:01.427+0000 7fc9b8710700 -1 mds.k *** got signal Terminated *** 2023-10-18T12:43:05.883 INFO:journalctl@ceph.mds.k.smithi152.stdout:Oct 18 12:43:05 smithi152 bash[181062]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-k 2023-10-18T12:43:05.988 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.k.service' 2023-10-18T12:43:06.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:43:06.022 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-18T12:43:06.022 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-18T12:43:06.023 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.c 2023-10-18T12:43:06.386 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:43:06 smithi186 systemd[1]: Stopping Ceph mds.c for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:43:06.719 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:43:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-c[173890]: 2023-10-18T12:43:06.381+0000 7f7aacbb7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:43:06.719 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:43:06 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-c[173890]: 2023-10-18T12:43:06.381+0000 7f7aacbb7700 -1 mds.c *** got signal Terminated *** 2023-10-18T12:43:10.718 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 18 12:43:10 smithi186 bash[180867]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-c 2023-10-18T12:43:10.744 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.c.service' 2023-10-18T12:43:10.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:43:10.782 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-18T12:43:10.782 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-18T12:43:10.782 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.f 2023-10-18T12:43:11.151 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:43:10 smithi186 systemd[1]: Stopping Ceph mds.f for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:43:11.468 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:43:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-f[173548]: 2023-10-18T12:43:11.147+0000 7f9894b04700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:43:11.469 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:43:11 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-f[173548]: 2023-10-18T12:43:11.147+0000 7f9894b04700 -1 mds.f *** got signal Terminated *** 2023-10-18T12:43:12.516 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 18 12:43:12 smithi186 bash[181006]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-f 2023-10-18T12:43:12.709 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.f.service' 2023-10-18T12:43:12.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:43:12.745 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-18T12:43:12.745 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-18T12:43:12.746 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.i 2023-10-18T12:43:12.969 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:43:12 smithi186 systemd[1]: Stopping Ceph mds.i for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:43:13.468 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:43:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-i[174233]: 2023-10-18T12:43:13.121+0000 7f5fd206f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:43:13.469 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:43:13 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-i[174233]: 2023-10-18T12:43:13.121+0000 7f5fd206f700 -1 mds.i *** got signal Terminated *** 2023-10-18T12:43:14.980 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 18 12:43:14 smithi186 bash[181148]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-i 2023-10-18T12:43:15.170 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.i.service' 2023-10-18T12:43:15.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:43:15.204 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-18T12:43:15.205 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-18T12:43:15.205 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.l 2023-10-18T12:43:15.469 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:43:15 smithi186 systemd[1]: Stopping Ceph mds.l for f94fde2a-6daf-11ee-8db6-212e2dc638e7... 2023-10-18T12:43:15.969 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:43:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-l[173198]: 2023-10-18T12:43:15.570+0000 7efe1bcd5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T12:43:15.969 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:43:15 smithi186 ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-l[173198]: 2023-10-18T12:43:15.570+0000 7efe1bcd5700 -1 mds.l *** got signal Terminated *** 2023-10-18T12:43:19.999 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 18 12:43:19 smithi186 bash[181287]: ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7-mds-l 2023-10-18T12:43:20.136 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f94fde2a-6daf-11ee-8db6-212e2dc638e7@mds.l.service' 2023-10-18T12:43:20.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T12:43:20.169 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-18T12:43:20.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-18T12:43:20.566 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:43:23.775 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-18T12:43:24.083 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:43:25.711 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-18T12:43:26.017 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:43:27.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:43:27.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:43:27.698 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T12:43:27.728 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-18T12:43:27.730 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi070/crash 2023-10-18T12:43:27.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash -- . 2023-10-18T12:43:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-18T12:43:27.819 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2023-10-18T12:43:27.822 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi152/crash 2023-10-18T12:43:27.823 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash -- . 2023-10-18T12:43:27.871 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-18T12:43:27.871 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2023-10-18T12:43:27.873 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi186/crash 2023-10-18T12:43:27.874 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash -- . 2023-10-18T12:43:27.931 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-18T12:43:27.932 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2023-10-18T12:43:27.933 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-18T12:43:27.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-18T12:43:28.012 INFO:teuthology.orchestra.run.smithi070.stderr:grep: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph.log: No such file or directory 2023-10-18T12:43:28.017 INFO:tasks.cephadm:Compressing logs... 2023-10-18T12:43:28.018 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T12:43:28.022 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T12:43:28.025 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T12:43:28.053 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-18T12:43:28.053 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-18T12:43:28.054 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.5.log 2023-10-18T12:43:28.054 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.2.log 2023-10-18T12:43:28.054 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.11.log 2023-10-18T12:43:28.055 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.z.log 2023-10-18T12:43:28.055 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.8.log 2023-10-18T12:43:28.055 INFO:teuthology.orchestra.run.smithi186.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-18T12:43:28.056 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.z.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.f.log 2023-10-18T12:43:28.056 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.c.log 2023-10-18T12:43:28.060 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.f.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.c.log: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.f.log.gz 2023-10-18T12:43:28.060 INFO:teuthology.orchestra.run.smithi186.stderr: 94.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.c.log.gz 2023-10-18T12:43:28.061 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.i.log 2023-10-18T12:43:28.061 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.c.log 2023-10-18T12:43:28.062 INFO:teuthology.orchestra.run.smithi186.stderr: 93.0% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-18T12:43:28.063 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-18T12:43:28.063 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-18T12:43:28.063 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.i.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-18T12:43:28.066 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.i.log.gz 2023-10-18T12:43:28.066 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.l.log 2023-10-18T12:43:28.067 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.10.log 2023-10-18T12:43:28.068 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.h.log 2023-10-18T12:43:28.069 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.e.log 2023-10-18T12:43:28.069 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.10.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.4.log 2023-10-18T12:43:28.069 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.1.log 2023-10-18T12:43:28.070 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-18T12:43:28.072 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.k.log 2023-10-18T12:43:28.072 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-18T12:43:28.072 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.1.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.7.log 2023-10-18T12:43:28.073 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.b.log 2023-10-18T12:43:28.073 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.b.log 2023-10-18T12:43:28.073 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.7.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.y.log 2023-10-18T12:43:28.073 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.h.log.gz 2023-10-18T12:43:28.074 INFO:teuthology.orchestra.run.smithi186.stderr: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.l.log.gz 2023-10-18T12:43:28.074 INFO:teuthology.orchestra.run.smithi152.stderr: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.e.log.gz 2023-10-18T12:43:28.075 INFO:teuthology.orchestra.run.smithi152.stderr: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.k.log.gz 2023-10-18T12:43:28.077 INFO:teuthology.orchestra.run.smithi152.stderr: 92.9% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-18T12:43:28.085 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log : No such file or directory 2023-10-18T12:43:28.085 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-18T12:43:28.086 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-18T12:43:28.087 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.0.log 2023-10-18T12:43:28.088 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.g.log 2023-10-18T12:43:28.089 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.a.log 2023-10-18T12:43:28.090 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.g.log: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-18T12:43:28.091 INFO:teuthology.orchestra.run.smithi070.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-18T12:43:28.091 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.6.log 2023-10-18T12:43:28.092 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.3.log 2023-10-18T12:43:28.093 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.d.log 2023-10-18T12:43:28.095 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.x.log 2023-10-18T12:43:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.9.log 2023-10-18T12:43:28.101 INFO:teuthology.orchestra.run.smithi070.stderr: 94.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.a.log.gz 2023-10-18T12:43:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.x.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-18T12:43:28.102 INFO:teuthology.orchestra.run.smithi070.stderr: 94.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.d.log.gz 2023-10-18T12:43:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.j.log 2023-10-18T12:43:28.104 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.a.log 2023-10-18T12:43:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.j.log: /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.j.log.gz 2023-10-18T12:43:28.114 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-18T12:43:28.465 INFO:teuthology.orchestra.run.smithi070.stderr: 90.0% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-18T12:43:28.622 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.c.log.gz 2023-10-18T12:43:28.732 INFO:teuthology.orchestra.run.smithi152.stderr: 91.0% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-18T12:43:28.811 INFO:teuthology.orchestra.run.smithi152.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-18T12:43:29.109 INFO:teuthology.orchestra.run.smithi152.stderr: 91.9% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.b.log.gz 2023-10-18T12:43:30.526 INFO:teuthology.orchestra.run.smithi070.stderr: 91.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mon.a.log.gz 2023-10-18T12:43:46.889 INFO:teuthology.orchestra.run.smithi186.stderr: 91.4% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.11.log.gz 2023-10-18T12:43:48.223 INFO:teuthology.orchestra.run.smithi070.stderr: 95.4% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.g.log.gz 2023-10-18T12:43:50.210 INFO:teuthology.orchestra.run.smithi070.stderr: 91.8% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.0.log.gz 2023-10-18T12:43:51.149 INFO:teuthology.orchestra.run.smithi186.stderr: 91.3% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.8.log.gz 2023-10-18T12:43:55.012 INFO:teuthology.orchestra.run.smithi070.stderr: 91.3% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.6.log.gz 2023-10-18T12:43:56.107 INFO:teuthology.orchestra.run.smithi186.stderr: 91.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.2.log.gz 2023-10-18T12:43:56.797 INFO:teuthology.orchestra.run.smithi070.stderr: 91.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.3.log.gz 2023-10-18T12:43:57.679 INFO:teuthology.orchestra.run.smithi070.stderr: 91.4% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.9.log.gz 2023-10-18T12:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-18T12:43:57.683 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m29.624s 2023-10-18T12:43:57.683 INFO:teuthology.orchestra.run.smithi070.stderr:user 2m1.166s 2023-10-18T12:43:57.683 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m9.176s 2023-10-18T12:44:02.057 INFO:teuthology.orchestra.run.smithi186.stderr: 91.3% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.5.log.gz 2023-10-18T12:44:02.059 INFO:teuthology.orchestra.run.smithi186.stderr: 2023-10-18T12:44:02.059 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m34.017s 2023-10-18T12:44:02.059 INFO:teuthology.orchestra.run.smithi186.stderr:user 1m40.986s 2023-10-18T12:44:02.060 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m3.310s 2023-10-18T12:44:04.916 INFO:teuthology.orchestra.run.smithi152.stderr: 91.9% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.1.log.gz 2023-10-18T12:44:08.531 INFO:teuthology.orchestra.run.smithi152.stderr: 91.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.10.log.gz 2023-10-18T12:44:14.753 INFO:teuthology.orchestra.run.smithi152.stderr: 91.6% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.7.log.gz 2023-10-18T12:44:15.366 INFO:teuthology.orchestra.run.smithi152.stderr: 91.5% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-osd.4.log.gz 2023-10-18T12:48:13.224 INFO:teuthology.orchestra.run.smithi152.stderr: 93.4% -- replaced with /var/log/ceph/f94fde2a-6daf-11ee-8db6-212e2dc638e7/ceph-mds.b.log.gz 2023-10-18T12:48:13.243 INFO:teuthology.orchestra.run.smithi152.stderr: 2023-10-18T12:48:13.243 INFO:teuthology.orchestra.run.smithi152.stderr:real 4m45.201s 2023-10-18T12:48:13.243 INFO:teuthology.orchestra.run.smithi152.stderr:user 5m26.968s 2023-10-18T12:48:13.244 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m12.250s 2023-10-18T12:48:13.244 INFO:tasks.cephadm:Archiving logs... 2023-10-18T12:48:13.245 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi070/log 2023-10-18T12:48:13.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T12:48:30.562 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi152/log 2023-10-18T12:48:30.563 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T12:49:23.702 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi186/log 2023-10-18T12:49:23.703 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T12:49:41.137 INFO:tasks.cephadm:Removing cluster... 2023-10-18T12:49:41.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --force 2023-10-18T12:49:41.536 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:49:42.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --force 2023-10-18T12:49:43.503 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:49:44.341 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f94fde2a-6daf-11ee-8db6-212e2dc638e7 --force 2023-10-18T12:49:44.645 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: f94fde2a-6daf-11ee-8db6-212e2dc638e7 2023-10-18T12:49:45.270 INFO:tasks.cephadm:Removing cephadm ... 2023-10-18T12:49:45.270 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-18T12:49:45.320 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-18T12:49:45.359 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-18T12:49:45.377 INFO:tasks.cephadm:Teardown complete 2023-10-18T12:49:45.377 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T12:49:45.390 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T12:49:45.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T12:49:45.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T12:49:45.402 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T12:49:45.535 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T12:49:45.546 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T12:49:45.547 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:49:45.552 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:49:45.554 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:49:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-10-18T12:49:45.570 INFO:teuthology.orchestra.run.smithi152.stderr:bash: ntpq: command not found 2023-10-18T12:49:45.573 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:49:45.574 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-10-18T12:49:45.574 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 24 +2457us[+2608us] +/- 48ms 2023-10-18T12:49:45.574 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -2113us[-1961us] +/- 64ms 2023-10-18T12:49:45.575 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -1203us[-1051us] +/- 87ms 2023-10-18T12:49:45.575 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:49:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:bash: ntpq: command not found 2023-10-18T12:49:45.596 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:49:45.596 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2023-10-18T12:49:45.596 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 22 +2443us[+2620us] +/- 48ms 2023-10-18T12:49:45.597 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -2133us[-1956us] +/- 64ms 2023-10-18T12:49:45.597 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -1247us[-1070us] +/- 87ms 2023-10-18T12:49:45.597 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:49:45.599 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:49:45.599 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-10-18T12:49:45.599 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 18 +2664us[+2664us] +/- 48ms 2023-10-18T12:49:45.600 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1916us[-1890us] +/- 64ms 2023-10-18T12:49:45.600 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 84 -1017us[ -998us] +/- 86ms 2023-10-18T12:49:45.600 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:49:45.601 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T12:49:45.611 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T12:49:45.612 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T12:49:45.642 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T12:49:45.819 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T12:49:46.000 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T12:49:46.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T12:49:46.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:49:46.175 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T12:49:46.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:49:46.212 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T12:49:46.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:49:46.245 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T12:49:46.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T12:49:46.273 INFO:teuthology.task.internal:Duration was 3043.578748 seconds 2023-10-18T12:49:46.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T12:49:46.284 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T12:49:46.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T12:49:46.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T12:49:46.294 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T12:49:46.341 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T12:49:46.384 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T12:49:46.402 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T12:49:46.732 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T12:49:46.732 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2023-10-18T12:49:46.733 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T12:49:46.791 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2023-10-18T12:49:46.792 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T12:49:46.855 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-10-18T12:49:46.855 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T12:49:46.877 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T12:49:46.878 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T12:49:46.882 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T12:49:46.886 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T12:49:46.982 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T12:49:46.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T12:49:46.987 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T12:49:46.989 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T12:49:47.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T12:49:47.170 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T12:49:47.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T12:49:47.240 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T12:49:47.278 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T12:49:47.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T12:49:47.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:49:47.325 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:49:47.327 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:49:47.372 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-10-18T12:49:47.373 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2023-10-18T12:49:47.388 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2023-10-18T12:49:47.424 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:49:47.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:49:47.466 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:49:47.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:49:47.483 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:49:47.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:49:47.499 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T12:49:47.512 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T12:49:47.513 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi070 2023-10-18T12:49:47.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T12:49:47.599 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi152 2023-10-18T12:49:47.600 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T12:49:47.639 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832/remote/smithi186 2023-10-18T12:49:47.640 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T12:49:47.684 INFO:teuthology.task.internal:Removing archive directory... 2023-10-18T12:49:47.684 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T12:49:47.689 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T12:49:47.692 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T12:49:47.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T12:49:47.744 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T12:49:47.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T12:49:47.757 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T12:49:47.757 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T12:49:47.773 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T12:49:47.775 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T12:49:47.789 INFO:teuthology.orchestra.run.smithi152.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 12:49 /home/ubuntu/cephtest 2023-10-18T12:49:47.791 INFO:teuthology.orchestra.run.smithi186.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 12:49 /home/ubuntu/cephtest 2023-10-18T12:49:47.812 INFO:teuthology.orchestra.run.smithi070.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 12:49 /home/ubuntu/cephtest 2023-10-18T12:49:47.815 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T12:49:47.829 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T12:49:47.853 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T12:49:47.878 DEBUG:teuthology.nuke:shortname: smithi070 2023-10-18T12:49:47.878 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T12:49:47.901 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 11:50:08.724412', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T12:49:47.905 DEBUG:teuthology.nuke:shortname: smithi152 2023-10-18T12:49:47.906 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T12:49:47.933 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 11:50:08.728216', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T12:49:47.934 DEBUG:teuthology.nuke:shortname: smithi186 2023-10-18T12:49:47.934 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T12:49:47.965 INFO:teuthology.orchestra.console.smithi070:Power off 2023-10-18T12:49:47.966 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T12:49:47.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 11:50:08.726455', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T12:49:47.988 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2023-10-18T12:49:47.989 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:48.005 INFO:teuthology.orchestra.console.smithi152:Power off 2023-10-18T12:49:48.005 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T12:49:48.019 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-18T12:49:48.028 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2023-10-18T12:49:48.029 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:48.047 INFO:teuthology.orchestra.console.smithi186:Power off 2023-10-18T12:49:48.048 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T12:49:48.058 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2023-10-18T12:49:48.071 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-10-18T12:49:48.072 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:48.097 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-18T12:49:52.020 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:52.060 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:52.099 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:52.131 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-18T12:49:52.170 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2023-10-18T12:49:52.209 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-18T12:49:56.136 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:56.171 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:56.211 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:49:56.246 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-10-18T12:49:56.282 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2023-10-18T12:49:56.321 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-10-18T12:49:56.349 INFO:teuthology.orchestra.console.smithi070:Power off completed 2023-10-18T12:49:56.422 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-10-18T12:50:00.286 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:50:00.397 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2023-10-18T12:50:00.498 INFO:teuthology.orchestra.console.smithi152:Power off completed 2023-10-18T12:50:00.605 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 3043.578747987747 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi070 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7420d7f8e55ceea786dcbf536c338ce920c21eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7cf3bacb860a46b0aa8a70058575c397 status: fail success: false 2023-10-18T12:50:00.606 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T12:50:00.691 INFO:teuthology.run:FAIL